A/B Testing with Matched Pairs Design

• Analytical Framework
• Business Decision Making based on A/B Testing
• Univariate, Bivariate Exploration of Data
• A/B Testing with Small Size Control Units (Matched Pairs Design)
• Principal component analysis (PCA) technique

According to the A/B Test Comparisons, the gross margin of the treatment stores in Central and West regions showed 30.42% and 21.9% growth at a significance level of 99.99% and 99.98% over the control stores, respectively. The expected impact of gross margin per week from Central region is USD 590.13, higher than that from West: USD 344.5. In both regions, the time comparison plots apparently show the improved sales trend after the implementation of the test. The treatment and control units have roughly similar sales during the comparative period on the left, but after the experiment began, there was a considerable increase in the profits of the treatment stores. The overall average lift as a result from launching the updated offerings would be 26.61% per store per week, approximately USD 467.309 increasing per unit per week. Introducing the new menu would improve the weekly gross margin, higher than 18% which is the cut-off that the management suggested, thus the company should roll out the new menu to all stores.

AB Trend Tool - Alteryx
AB Controls Tool - Alteryx
AB Analysis Tool - Alteryx
Principal component analysis - sklearn