The lift the test can promise

Netflix A/B Testing Interview Question

A new home screen row was tested on 200,000 sessions in each arm. Five percent of control sessions added a title to the list, against 5.3 percent of treatment. Engineering will only keep the row if it is worth a quarter of a percentage point.