Skip to main content

Settings Reference

All settings can be included when creating or updating a split test. Only name, mode, and variants are required for creation; everything else is optional with sensible defaults.

Core

ParameterTypeDefaultDescription
namestringrequiredHuman-readable test name
statusstringrunningrunning or paused
start_datedatetimenowISO 8601 datetime
end_datedatetimenullISO 8601 datetime

Duration & Auto-Winner

ParameterTypeDefaultDescription
durationintegernullTest duration in days
auto_select_winnerbooleanfalseAutomatically declare a winner
winner_selection_criteriaarraynullMetrics to evaluate, e.g. ["aov", "add_to_carts_pct"]
winner_selection_confidenceintegernullConfidence threshold (1–100)
sample_size_limitintegernullStop after this many total views
baselinestringnullNode code of the baseline variant

Statistical Configuration

ParameterTypeDefaultDescription
significance_leveldecimal0.05Alpha (0.001–0.5)
statistical_powerdecimal0.81 - beta (0.1–0.99)
minimum_detectable_effectdecimal5MDE in percent (0.1–100)
multiple_testing_correctionstringbonferroninone, bonferroni, holm, fdr
statistical_methodstringfrequentistfrequentist or bayesian
primary_metricstringconversion_rateMain metric to optimize
practical_significance_thresholddecimal2Minimum meaningful improvement (%)
require_practical_significancebooleantrueRequire practical significance
min_sample_size_per_variantinteger2000Minimum views before declaring winner
max_sample_size_per_variantintegernullStop variant at this many views
min_duration_daysintegernullMinimum test duration
max_duration_daysintegernullAuto-stop after this many days
enable_sequential_testingbooleanfalseAllow peeking with alpha spending
alpha_spending_functionstringnulle.g. obrien-fleming
guardrail_metricsarraynullMetrics that must not degrade
enable_automated_winner_selectionbooleanfalseAuto-declare when thresholds met
automated_decision_thresholddecimal0.95Probability threshold for auto-declaration

Bayesian Settings

Used when statistical_method is bayesian.
ParameterTypeDefaultDescription
bayesian_prior_meandecimalnullPrior mean
bayesian_prior_variancedecimalnullPrior variance
bayesian_credible_intervaldecimal0.95Credible interval (0.5–0.99)

Advanced

ParameterTypeDefaultDescription
enable_variance_reductionbooleanfalseCUPED-style variance reduction
covariate_metricsstringnullComma-separated covariate metric names
enable_segmentation_analysisbooleanfalseSegment results by dimensions
segmentation_dimensionsarraynulle.g. ["device", "country"]
statistical_notesstringnullFree-text notes

Custom Goals

ParameterTypeDescription
goalsarrayArray of goal objects
goals.*.namestringDisplay name (e.g. “Purchase”)
goals.*.valuestringEvent identifier matching a page interaction (e.g. purchase_complete). Alphanumeric, underscores, and dashes only.

Full Settings Example

{
  "mode": "page_top",
  "page_id": 456,
  "name": "Homepage hero test",
  "variants": [
    { "name": "Control", "weight": 50 },
    { "name": "Variant B", "weight": 50, "page_id": 789 }
  ],
  "duration": 14,
  "auto_select_winner": true,
  "winner_selection_confidence": 95,
  "sample_size_limit": 10000,
  "significance_level": 0.05,
  "statistical_power": 0.8,
  "minimum_detectable_effect": 5,
  "statistical_method": "frequentist",
  "primary_metric": "conversion_rate",
  "min_sample_size_per_variant": 2000,
  "goals": [
    { "name": "Add to Cart", "value": "add_to_cart" },
    { "name": "Purchase", "value": "purchase_complete" }
  ]
}

Allowed Node Types

For Mode 2 variant chains, each node in the nodes array must have a type from this whitelist:
TypeData FieldsInputsOutputsConstraint
pagevalue: page ID (integer)11Funnel context only
page_variantvalue: page ID (string), load_page_events: boolean (optional)11
url_redirectvalue: URL (string)10Terminal node
product_check(none)12
product_check_productvalue: product code (string)11Must follow product_check
product_check_all(none)11Must follow product_check
script_rulescript_rule: JS expression (string)12
set_variablevariables: key-value object11
add_tagvalue: tag name (string)11

Example: Node chain with product check

{
  "variants": [
    {
      "name": "Product Path",
      "weight": 50,
      "nodes": [
        { "type": "product_check" },
        { "type": "product_check_product", "value": "premium-bundle" },
        { "type": "page", "value": 500 }
      ],
      "next_node_code": "abc123def456gh78"
    }
  ]
}