Experiment Error Log

The Experiment Error Log surfaces JavaScript errors and warnings that the A vs B snippet captures while your experiment is running. It appears on the Results page whenever at least one error or warning has been captured, and stays hidden on clean experiments so it never adds noise to healthy runs.

What the error log shows

The log tracks two severity levels:

Error — a JavaScript exception or a failed variation apply that likely broke the variation for the affected visitor.
Warning — a non-fatal issue, such as a CSS selector that could not be matched. The variation may still have run, but not as intended.

Within each severity level there are three error types:

JS Exception — an unhandled JavaScript error thrown inside variation code or project JavaScript.
Apply Failed — the snippet attempted to apply a variation (DOM modification or JS execution) and the operation threw before completing.
Selector Miss — a CSS selector targeted by a visual variation could not be found in the page DOM at application time.

How the error rate is calculated

The header row of the error log shows a per-variation error rate. The rate is computed as:

errorVisitors ÷ exposedVisitors × 100

where errorVisitors is the count of distinct visitors who had at least one error captured on the current shipped code (keyed to the code hash published at the time of the last deploy), and exposedVisitors is the count of distinct visitors exposed to that variation since the last deploy.

When a variation has fewer than 50 exposed visitors on the current code, the rate denominator is too small to be meaningful. In that case the UI shows the raw affected visitor count instead of a percentage, and displays a dash (—) where the rate would appear. This avoids misleading 100% error rates on very new deploys.

Reading and expanding an error group

Errors are grouped by fingerprint — a client-computed hash of the error type, message, and stack frame locations. Each group row shows:

Severity badge (Error or Warning)
Error type (JS Exception, Apply Failed, or Selector Miss)
Which variation triggered it
Truncated error message
Total occurrence count and distinct affected visitor count
When it was last seen

Click a group row to expand it. The expanded panel shows:

Sample stack trace— a representative stack captured with the first (or most recent) occurrence. Long stacks can be expanded with "Show more."
By browser — occurrence counts broken down by browser user-agent family.
By page — occurrence counts broken down by the page path where the error was captured (query strings are stripped — see Privacy below).
First seen and last seen timestamps.

Filtering by severity

The toolbar at the top of the log provides three tabs:

All — shows every captured group regardless of severity.
Errors — limits the list to error-severity groups only.
Warnings — limits the list to warning-severity groups only.

You can also filter by a specific variation using the dropdown next to the tabs. The view refetches automatically a short time after you change either filter.

The log section starts expanded when there are any error-severity groups, and collapsed when there are only warnings. This means critical errors are always visible above the fold on the Results page without requiring a click.

Downloading the error log

The toolbar includes CSV and JSON download buttons. Both respect the current severity and variation filters, so you can export exactly the subset you are looking at. The download is a direct file from the server — your browser will prompt you to save it.

When a variation is shipping with errors on its current code, a banner appears at the top of the Results page — above everything else — so you see it the moment you open the experiment. It names the affected variation(s) and their error rate (or affected-visitor count under the small-sample floor), with two actions:

View error log — jumps straight to the log section lower on the page.
Dismiss — acknowledges the banner for the current code. It stays dismissed until a new, distinct problem appears on that same code, at which point it re-appears so a fresh issue is never silently hidden. Re-publishing the variation always clears it (a code change starts a fresh measurement window).

The banner is driven by errors only — warnings notify (below) but never raise the banner. Experiments with active errors also show a red Errors pill next to their status on the experiment list.

Alerts & notifications

A background reconciler checks running experiments every couple of minutes. The first time a variation starts erroring on its current code, A vs B sends an alert immediately, then at most one reminder per hour while the problem persists. Alerts fan out through two channels:

In-app bell + email to the person who launched the experiment (falling back to whoever scheduled it, then an org admin), so the alert always has an owner.
Your org's integrations — Slack, Microsoft Teams, Jira, and outgoing webhooks all receive the event if configured. Like SRM and guardrail alerts, code-error alerts are a safety event: they bypass notification keyword gating and mute rules so a real breakage is never filtered out.

Errors always alert. Warnings (selector misses) only escalate to a notification once they affect a meaningful share of traffic — a rate of 5% or more, or 25 or more affected visitors. Below that they are still recorded in the log, just not pushed.

Auto-pause on errors

Open an experiment's Analysis settings to enable Auto-pause on errors. When enabled, A vs B automatically pauses the experiment if any variation's error rate on the current code crosses your threshold (default 25%). Pausing drops the experiment from the published datafile, so the snippet stops serving the broken variation right away, and the alert tells you it was auto-paused.

Unlike the statistical-analysis settings — which lock once an experiment launches — the auto-pause toggle and threshold stay editable for running experiments, because their whole job is to protect a live test. A threshold only trips on a measured rate, so a variation under the small-sample floor (fewer than 50 exposed visitors) is never auto-paused on thin data.

Errors in the preview panel

Errors captured while a visitor (including you) uses a preview link also flow into the error log. This makes the log useful as a QA surface during the pre-launch review phase — you can open a preview link, interact with your variation, and then check the Results page to see whether any JS exceptions were triggered.

What we capture and privacy

The snippet captures the error message, stack trace, error type, and the page path (with query strings stripped before transmission). It never captures DOM content, form values, cookies, localStorage, or any personally identifiable information. Stack frames reference line and column numbers in your bundled JavaScript — not user data.

The page path recorded in the "By page" breakdown is the location.pathname at the moment the error was captured, with the query string (?…) removed. If your app encodes user identifiers in path segments (e.g. /users/12345), those will appear in the breakdown. Use path-based identifiers with care, or configure your server-side routing to keep sensitive identifiers out of URL paths.