Sample Ratio Mismatch

A sample ratio mismatch (SRM) occurs when the actual traffic split between your variations is significantly different from the expected split. For example, if your experiment is configured as 50/50 but you see 60% of visitors in the control and 40% in the variant, something is systematically skewing which variation visitors end up in. This means the experiment data cannot be trusted.

The SRM guardrail

A vs B shows an SRM guardrail warning on the results page when it detects a statistically significant imbalance in your traffic split. The warning has two severity levels:

Yellow warning — the split is noticeably off but not extreme. Proceed with caution and investigate.
Red warning — the split is severely imbalanced. The results cannot be trusted. Stop the experiment and fix the underlying problem before continuing.

The guardrail uses a chi-squared test to determine whether the observed traffic distribution is consistent with the expected distribution given the configured traffic split. A p-value below 0.01 triggers the yellow warning; below 0.001 triggers the red.

Why SRM matters

If your traffic split is skewed, the two groups are not equivalent. Any difference in conversion rates between them might be because of the variation, or it might be because one group has a systematically different composition of visitors. You cannot tell which. An SRM makes the experiment's results meaningless — you cannot conclude that the variation caused any observed difference.

Common causes

Variation code crashes for some visitors

If your variant variation has a JavaScript error that crashes for a subset of visitors, those visitors may be excluded from the experiment (the snippet catches the error and deactivates the experiment for that visitor). This disproportionately removes visitors from the variant, reducing its count relative to the control.

How to investigate:Open DevTools → Console on the target page and force yourself into the variant:

1avsb.forceVariation('exp_abc123', 'var_variant_a');

Look for any JavaScript errors in the console after the variation is injected. Any error here could be the cause of the SRM.

Bot traffic affecting one variation more than others

Web crawlers, monitoring bots, and automated scripts may be consistently bucketed into one variation. This inflates one variation's visitor count without producing real human conversions. Most bots do not store cookies, so they may be assigned a new variation on every visit and could be landing disproportionately in certain buckets depending on how they cycle through.

How to investigate: Look at the visitors per variation over time. Bot traffic often shows up as spikes of visits with zero conversions during off-hours.

Redirecting visitors away from a variation

If your variation includes a redirect (sending visitors to a different URL after being bucketed), the exposure event may fire before the redirect, but the redirect prevents the visitor from seeing the actual variation experience. This can cause SRM because some visitors are counted as exposed but never actually participated.

Fix: If you are testing a redirect, structure the experiment so the exposure event fires only after the redirect completes, or use a redirect experiment type.

Visitors being re-bucketed due to cookie issues

If visitors' cookies are being cleared frequently (for example, due to strict browser privacy settings, cookie-blocking policies, or short cookie expiry), the same visitor may be bucketed multiple times. The random bucketing is independent each time, so over many re-bucketing events, the distribution should even out — but in the short term, this can cause SRM.

How to investigate: Check whether the SRM correlates with a particular browser (Safari with ITP, Firefox with Enhanced Tracking Protection) or a particular traffic segment.

How to investigate any SRM

Compare visitor counts per variation — look at the raw numbers on the results page. Is the imbalance consistent over time, or did it spike at a specific point?
Check for JavaScript errors — open DevTools and force yourself into each variation. Look for any console errors.
Look for patterns in the excess traffic — if nearly all the excess traffic is in the control, it suggests the variant is crashing or being deactivated for some visitors. If the excess is spread randomly, it may be a bucketing or cookie issue.
Check for deployment timing — did the SRM appear right after you changed the variation code or traffic splits? Changing traffic splits mid-experiment always causes SRM; it is best to set the splits once and not change them.

If you change the traffic allocation percentages after an experiment is already running, you will almost certainly get an SRM. The visitors bucketed before the change were distributed according to the old percentages; visitors after the change are distributed according to the new ones. The combined dataset looks skewed because it is a mix of two different distributions. Avoid changing traffic splits after launch. If you must change them, consider stopping the experiment, resetting the data, and starting fresh.

It can be tempting to proceed with an experiment despite an SRM warning, especially if the results look good. Resist this temptation. An SRM means the experimental groups are not equivalent, which means the results might be a statistical artefact rather than a real effect. Fix the underlying problem first, then re-run the experiment.