How to find Statistical Significance in AdWords Reports


AdWords or any performance oriented marketing platform is all about numbers. All your actions are based on how historically campaigns, ad groups, keywords have performed. This data however can be quite deceiving if interpreted incorrectly. This article explains the common mistakes a marketer makes while dealing with data and more specifically lack of significant data.

Statistically Significant Data

Different industries follow different methods to calculate statistical significance. Here is a wikipedia article on statistical significance. Statistical significance is extremely important in adwords to make any changes to your campaigns.

Lets say as a marketer, you are given the task of reducing the cost per conversion for your client and the report looks like something given below.

Campaigns with statistical significance

Nabu has 3 campaigns running for his hotel. The AdWords report for the last 30 days is as follows

Campaign Clicks Cost Cost per Conversion Conversions
Hotel 1 200 100 5 20
Hotel 2 300 180 6 30
Hotel 3 100 100 10 10

Looking at the above data, you made the following conclusion.

Hotel 1 and Hotel 2 Campaigns are almost running at the same cost per conversion but Hotel 3 is running at a cost per conversion which is much higher than the other 2. This usually means that there is good amount of improvement that can be done in reducing the cost per conversion of Hotel 3 campaign, Which would lead to overall reduction in cost per conversion.

The above is a perfectly valid argument as there is enough data available to arrive at this conclusion. Now take a look at the following report.

Campaigns with no statistical significance

Campaign Clicks Cost Cost per Conversion Conversions
Hotel 1 200 100 5 20
Hotel 2 300 180 6 30
Hotel 3 50 20 10 2

Here one cannot say with confidence that Hotel 3 is not performing well because of the lack of data. There have just been 2 conversions and this data is too low to arrive at any meaningful conclusion

Cheat sheet for statistical significance

As a rule of thumb, look for campaigns with more than 10 conversions for any given period before coming to a conclusion. For campaigns with less than 10 conversions, it makes lot more sense to look at a longer time period. if you were looking at last 30 days, have a look at last 90 days and see if the conversions is reaching 10. If you still cannot get to 10 conversions it is better to wait for more time before jumping to any conclusions.

About the author

Salil Panikkaveettil

CEO and co founder of AdNabu. Exploring the intersection of data and marketing

Follow Us!