Finding Statistical Significance for A/B Testing

Options
Eric.P
Eric.P Member
in Dashboards

Hi there,

I have a dashboard for reporting on the results of my A/B tests. You select what experiment you want to look at from the filters, and the KPIs are populated for each variant, as well as a Lift/Decline for different metrics which is calculated through different Beast Mode Calculated Metrics.

My question is, is it possible to dynamically return statistical significance for these different metrics? It would have to take into account whatever filters are applied on the page. Since Beast Mode doesn't have statistical functions, I anticipate I'll have to plug in Python somehow.

Thanks!

Answers

  • MichelleH
    MichelleH Coach

    @Eric.P Depending on which type of statistical test you are running, you could potentially recreate the base p-value calculation in beast mode. Admittedly, that would be a pretty complex calculation, though I do see that there is a standard deviation beast mode function.

    image.png

Categories