The diagram above shows a research project I designed. The beauty of it is that it followed two week development sprints which allowed the business the freedom to end or continue the experiment as they saw fit.
This experiment design relied heavily on the customer journey and special requirements of the company. Customers were subscribers to a weekly service, so the two week sprint accounted for two purchase cycles. The company felt strongly that customers needed to opt-in to the experiment. They were also very nervous about losing customers because of the new experience. I built a proxy into the research design to safely account for potential customer losses.
Through the planning and execution process, the diagram was used to communicate with everyone involved from the developers to the CEO. In practice, the company got statistically significant results from only one sprint iteration.
Stage 6 shows a mid-sprint phone call to participants. In practice, we sent out a survey. Here is a sampling of results that shows how I code qualitative data.
Other research methods were used before rolling out the final changes. The data collected from the research altered the original plans. The results of the changes were an increase in Average Order Sale (AOS) by $1.03 meaning nearly $1 million in increased revenue.