Success Threshold

The success threshold is the smallest change you’d consider worth shipping for. It’s optional — but when set, VEKTIS uses it to make sure a result isn’t just statistically significant, but also large enough to matter to your business.

Statistical vs. practical significance

VEKTIS computes two scores in parallel and takes the lower of the two as the final impact score:

Score	Asks	When it matters
Statistical	Is the change big enough vs. natural variation to be a real signal?	Always — anchored in your baseline data
Practical	Is the change at least as big as your success threshold?	Only when a threshold is set — otherwise statistical confidence is the only gate

If a result clears the statistical bar but the change is much smaller than your threshold, the practical score pulls the final impact score down — preventing technically-significant-but-tiny changes from being celebrated.

When to set it

Situation	Recommendation
You know the rough business value of a change in the metric’s units	Set it — e.g., “5 percentage points of conversion is a real win”
The metric is volatile and noisy	Set it — keeps statistically-significant-but-meaningless changes out of green
You’re unsure what counts as meaningful yet	Leave it blank — let statistical confidence drive scoring while you learn
You want every signal surfaced, regardless of magnitude	Leave it blank — no practical-significance gate applies

Tip

You can set or change the threshold at any time. There’s no penalty for starting blank and adding it later, or revising it as you learn what “meaningful” looks like for this metric.

Practical guidance

You see	What it means	What to do
Significant Impact (green), threshold cleared	Both bars met — strong evidence and the change is large enough to matter	Celebrate
Confirmed Impact (green), threshold cleared	Likely real and meaningful	Keep measuring
Score lower than expected, threshold set high	Practical score is pulling the impact score down	Re-check whether your threshold is realistic for this metric
No threshold set	Only statistical confidence is gating the score	Consider setting one once you know what “meaningful” looks like