FeatBit

Feature Experimentation

A promising feature still needs a real comparison.

An indoor route or a short indoor list? Give travelers a stable 50/50 split, connect exposure to meaningful measures, and inspect the effect and uncertainty before making either experience the default.

Try FeatBit See the solution
AB

A practical walkthrough with rainy-day-rescue

The story · An indoor route or a short list? Let the comparison answer.

The travel app has two rainy-day ideas. Which one keeps travelers engaged?

An ordered indoor route offers a ready-made plan. A short indoor list gives travelers more choice. The team compares the experiences with an even traffic split, stay duration as the primary metric, and key button clicks as a guardrail.

Recorded demo case · Real product configuration

Inside FeatBit · 3 steps

Split the traffic. Measure what happens next.

Follow a rainy-day travel experiment from a 50/50 feature flag split to an evidence-based release decision.

Recorded demo · This travel-app experiment remains inconclusive. Its observations illustrate the measurement workflow.

  1. 01Step 01 / 03

    Split users between two real experiences.

    The travel app can suggest an ordered indoor route or a short indoor list. In rainy-day-rescue, configure a 50/50 percentage rollout and dispatch by keyId so the same identity receives a stable assignment.

    Indoor route / list
    50% / 50%
    Dispatch by
    keyId

    The feature flag controls the experience each user receives; the experiment measures what follows.

    Inside FeatBitRecorded demo
    Swipe to inspect
    Feature flag → Targeting → 50/50 rollout

    Eligible travelers

    Route or list · 50/50

    Stable feature flag assignment comes first

  2. 02Step 02 / 03

    Connect the flag to the measures that matter.

    Bind rainy-day-rescue in Exposure. The hypothesis is that an indoor list keeps people engaged without reducing key button clicks. Choose stay duration as the primary metric and key button clicks as the guardrail, using the same user identity for exposure and outcome events.

    Primary
    Stay duration · higher is better
    Guardrail
    Key button clicks · decrease is bad

    Both the served variation and its later outcomes belong to the same comparison.

    Inside FeatBitRecorded demo
    Swipe to inspect
    Exposure → Linked feature flag
    Inside FeatBitRecorded demo
    Swipe to inspect
    Exposure → Primary metric and guardrail

    Served variation

    Duration + key actions

    One primary measure, with a guardrail

  3. 03Step 03 / 03

    Read the effect, uncertainty, and guardrail together.

    The recorded analysis has 778 control and 722 treatment samples. The primary metric shows +0.9% relative lift, with a 95% credible interval from −3.4% to +5.1%. The interval still includes no improvement. Review the posterior and the guardrail before choosing the next release action.

    Primary result
    Inconclusive
    Guardrail lift
    +5.4% · interval −12.7% to +23.5%
    Guardrail status
    Inconclusive · monitor

    This run supports continued observation, not a winning-variant claim or an automatic full release.

    Inside FeatBitRecorded demo
    Swipe to inspect
    Measuring → Full analysis and posterior distribution
    Inside FeatBitRecorded demo
    Swipe to inspect
    Measuring → Guardrail result

    Effect + uncertainty

    Continue, stop, or ship

    An inconclusive result is a reason to keep learning

The next decision

Keep learning when the interval still includes no improvement.

This recorded run remains inconclusive. Review sample quality, the primary effect and its uncertainty, and the guardrail together before continuing, changing the idea, or expanding the release.

Build your own release in FeatBit

Keep allocation and analysis sampling distinct

The feature flag controls which experience users receive. Analysis sampling controls which observations enter the run. Keep identities consistent, inspect sample quality, and avoid changing the decision criteria after seeing a result.

From idea to implementation

Build your release path.

Explore the release decision workflowDesign useful experiment metricsConfigure a percentage rollout

Keep the release moving.

All solutions

Feature Impact Monitoring

Explore the Release Health design: connect metric sources, define guard conditions, send webhook alerts on threshold breaches, and investigate with release context.

Segment-Based Releases

Try a search rewrite with an opt-in beta or a regional pilot. Use the same limited audience across related feature flags.

Progressive Rollouts

Start an AI-assisted checkout with a small canary. Review errors and completion before more users receive the candidate.