Perhaps the data scientists have a sense of humor? Considering it comes from inside Facebook, we couldn’t help but notice the title of this latest white paper, “Prepare for the Unexpected.” (Hmm. No comment…)
The subtitle is a lot more precise: “A Guide to Testing and Learning with Incrementality Measurement.”
What is it? A 16-page document to assist marketers to address six common problems identified by Facebook and a team of experts from Lyft, Netflix, eBay, and Booking.com:
- Your Test Doesn’t Have Enough Statistical Power
- One of Your Test Groups Has Some Outliers
- Your Variables Are Not Isolated
- People in Your Test and Control Groups Cross Paths
- Some Tests Will Have Effects Beyond an Initial User Interaction
- You Can’t Track Everything You Want
Leave a Reply