Hello, I need to build a Power Calculator in Python for a priori analysis in A/B testing. This tool should be able to identify a relationship between %Lift and Holdout while showing the Power change (please refer to the attached picture (Output_Example) as an example for visualization that needs to be built, but disregard the Campaign duration and Media Weight sliders).
- Confidence Level (Alpha = 0.1)
- Conversion Rate Control - array of values
- Conversion Rate Test = Conversion Rate Control*(1+Lift)
- Lift Range : 1, 2, 2.5, 3, 4, 5, 6, 7, 8, 9, 10, 12.5, 15, 20, 25, 30, 35, 40
- Holdout Range: 0.01, 0.02, 0.03, 0.04, 0.05, 0.1, 0.15, 0.2, 0.25, 0.3, 0.35, 0.4, 0.45, 0.5
- Total Group Size - array of values
For reference, you should be using the pwr.2p2n.test() from R, where:
n1 = Total Group Size*(1-Holdout)
n2 = Total Group Size*Holdout
Anticipated output: Python code with functions that can be applied to a range of different scenarios (Total Group Size and Conversion Rate Control will differ, based on proposed scenarios - please refer to Input_Scenarios file). The Power Calculator should be generating a heatmap similar to Output_Example, but values for sliders are provided as inputs above. Add commentary to the Python code, so it could be explained and reused.
Please let me know if you could help me with this task, anticipated time for completion and budget.