This project evaluates the causal impact of advertisement exposure on user purchase behavior using a large-scale randomized A/B experiment. The goal is to determine whether displaying advertisements increases conversion rates compared to a Public Service Announcement (PSA) control condition.
The project demonstrates a complete experimentation workflow including statistical testing, Bayesian inference, business impact analysis, and deployment of an interactive analytics dashboard.
Experiment Validation: The experiment was first validated to ensure reliable inference by checking randomization integrity and verifying that traffic allocation across experimental groups was consistent.
Conversion Rate Analysis: The primary success metric is conversion rate, defined as the proportion of users who completed a purchase. Conversion rates were compared across treatment and control groups to estimate the impact of advertisement exposure.
Statistical Testing: A two-proportion z-test was conducted to determine whether the observed difference in conversion rates between the ad and PSA groups is statistically significant.
Bayesian Inference: A Bayesian Beta–Binomial model was used to estimate posterior distributions of conversion rates and compute the probability that advertisements outperform the control condition.
Business Impact Evaluation: Statistical results were translated into business metrics such as absolute lift, relative improvement in conversion rate, and estimated incremental conversions generated by advertisement exposure.
An interactive dashboard built with Streamlit allows exploration of experiment outcomes and user behavior patterns. The dashboard provides visual analysis of conversion rates, treatment lift, and Bayesian posterior distributions, enabling intuitive interpretation of the experiment results.
The application is containerized using Docker and deployed on Google Cloud Run. A GitHub Actions CI/CD pipeline automatically builds and deploys the application whenever updates are pushed to the repository.
Python • Pandas • NumPy • Statsmodels • Matplotlib • Streamlit • Docker • GitHub Actions • Google Cloud Run
Estimated impact:
- Absolute lift: ~0.77 percentage points
- Relative lift: ~43% improvement
The results indicate that advertisement exposure significantly increases the likelihood of user conversion.