Predictive analysis best practices

To project future revenue predict the growth and decay

Predict future customer growth and revenue by using a diffusion model like PH to project the growth curves. Then use a model like SBG or BTYD to project cohort retention decay curves. Put the projections into an expanded cohort and sum.

Prediction data comes from the same place as the level set

Prediction data comes from the master table just like the 108.30.10 Data Analysis - Structural analysis, but the data must be aggregated into another table in a very specific format for the models to work.

Always eyeball predictions and make sure they make sense

Put projections side-by-side with the historical numbers and make sure they make sense. If they are significantly different then the projections are probably not right. - Can look at this by putting KPIs next to each other. Avg churn, AOV, etc

Models aren’t straightforward

• The bottleneck for prediction is tweaking the models to reflect reality and make sense. You can’t just run the model and use the output because it will often not make sense in the real world.
• In addition to the base models there are “twists”. For example a third party company called Zodiak has 25 twists on the BGBB model.
  • Models could potentially be automated to apply all the known “twists” so the most appropriate one can be chosen

Nice-to-haves (but not must-haves)

• Quantile / confidence intervals / error bars – ranges – are not a “must have” but they are a great feature that is appreciated by all parties. Confidence is important.
• We wanted to always do up/down-case and base case for everything when possible, but never really got around to it in practice. It’s hard to keep everything straight. See the section 108.40.30.10 Data Analysis - Data output validation is vital