Feature Flag Concepts

Feature flags let developers control feature visibility from the A vs B dashboard without deploying new code. A flag is evaluated in your application code and returns a value that determines whether a feature is shown, which variant to display, or what configuration to use.

Project Types

A vs B supports two project types. When creating a project, you choose one:

• Web Experimentation — Visual A/B tests with a snippet that injects changes into web pages. For marketing and growth teams.
• Feature Flags — Code-native flag evaluation via an npm SDK. For developers and product engineers.

Each project type has its own dashboard, navigation, and workflow. Both share the same organization, audiences, scheduling, and analytics infrastructure.

Flag Types

The flag type is chosen at creation and cannot be changed afterward.

• Boolean — Simple on/off. Returns true or false.
• String — Returns one of several string values (e.g., "blue", "green").
• Number — Returns a numeric value (e.g., 30 for a timeout).
• JSON — Returns a JSON object for complex configuration.

Variations

Each flag has variations — the possible values it can return. Boolean flags always have exactly two variations (true and false). String, number, and JSON flags can have unlimited named variations.

One variation is designated the default — served when the flag is off or when no rules match.

Environments

Feature flag projects support multiple environments (Development, Staging, Production, etc.). Each environment:

• Has its own SDK key and datafile
• Has independent on/off state and targeting rules
• Must be published separately for changes to go live

Draft and publish

Edits to a flag's rules, environment toggle, default variation, scheduling, and overrides are staged in a draft until you click Save in the bottom bar. Toggling a rule on or off, deleting it, duplicating it, or editing its targeting all stage into the same draft — nothing reaches your users until you publish. Click Revert to discard the entire draft and go back to the last published state.

Rule Types

Rules are configured per-environment and evaluated top-to-bottom (first match wins). There are two types:

• A/B Test — Split users across variations with configurable traffic allocation, attach metrics, and get statistical results. When a winner is found, conclude the test.
• Targeted Delivery — Everyone matching the targeting conditions gets a specific variation. No randomization, no metrics. Use for feature rollouts, kill switches, or serving a winner after an A/B test concludes.

Evaluation Order

When a flag is evaluated, the SDK checks in this order:

1. Individual overrides — specific user IDs forced to specific variations
2. A/B Test rules — traffic-split experiments with metric tracking
3. Targeted Delivery rules — serve a fixed variation to matching users
4. Default variation — returned when no rules match

Flag Statuses

Statuses are auto-calculated from the flag's state:

• Draft — Off in all environments, never been enabled
• Active — On in at least one environment
• Paused — Was previously active, now off everywhere
• Archived — Manually archived, removed from datafiles

Stale Flag Detection

Flags are automatically marked stalewhen they haven't been modified in 2+ weeks and their rules serve a single variation at 100%. Stale indicators help teams identify flags that should be cleaned up (code reference removed, flag archived).

Targeting Attributes

Feature flag rules use audiences to control which users see which variation. Audiences are built from conditions, and the primary condition type for feature flags is the Custom Attribute.

Attributes come in two categories:

• Auto-detected — 11 built-in attributes (browser, OS, device type, country, etc.) that the SDK captures automatically. Some require passing userAgent or ip in server-side SDKs.
• Custom — Application-specific properties (userId, plan, team role) that you register in Audiences > Attributes and pass when evaluating flags.

Register and manage attributes in your project's Audiences > Attributes tab. Each attribute has a key, display name, type (String, Number, Boolean, or JSON), and optional suggested values for string attributes.

Events and Metrics

For A/B test rules, you define custom events in the dashboard and track them from code using client.track(). Events support two metric types:

• Conversion — Did the user perform the action? (binary)
• Numeric — What was the value? (e.g., revenue)

Analysis section (A/B test rules)

Every A/B test rule has an Analysis section in the rule editor with two controls:

• Stats engine— the statistical method used to decide whether a result is real. In V1 this is always Bayesian.
• Variance reduction — Auto or Off. Auto applies CUPED on numeric metrics when the pre-experiment data supports it; Off always reports raw numbers.

Both controls default to the project's Analysis Defaults at rule-creation time. You can override either before the rule first goes live. Once the rule has been enabled (even briefly), both fields are locked — the same rule as experiments, for the same reason: switching analyses after exposures have been collected would invalidate the results.

See Variance Reduction (CUPED) for a deeper explanation of what the adjustment does and when Auto applies it. The same math, eligibility rules, and label conventions apply to both experiments and feature-flag A/B test rules.

Flag Rule Results modal

The Flag Rule Results modal (opened from the rule's three-dot menu) shows an analysis labelat the top — the same Bayesian · Auto (CUPED applied) / Bayesian · Off /Bayesian · Auto (no adjustment — reason) summary you see on the experiment Results page.

When CUPED was applied to a metric, an ⓘmarker appears next to that metric's improvement cell. Hover it to see the CUPED weight (θ) and the percentage variance reduction for that metric. Metrics where CUPED declined (conversion-rate metrics in V1, or continuous metrics without enough pre-period history) report the raw numbers and explain the decline in the analysis label's tooltip.