How many users the test needs
hardHow many users the test needs
Duolingo A/B Testing Interview Question
A new onboarding screen is going to be tested. Ten percent of new signups currently finish their first lesson, and the team wants to be able to detect a two percentage point lift at 80% power and the usual 5% significance level. About a thousand people sign up a day. How many signups does each arm need?
Asked of
- Data Scientist
- Product Analyst
- Data Analyst
- Business Analyst