Wednesday, February 27, 2013

How Bad Did Analytics Bugs Fool Facebook Pages? Reach Looked Down 14% When It Wasn’t

The link to the full story is here

Months of mysterious Insights bugs duped Facebook Page owners into thinking their reach was plummeting. The median Page’s analytics showed it reached 14.39 percent fewer people than it actually did, according to early data and graphs from EdgeRank Checker, a Facebook analytics startup that gave us the first look. The bugs may have caused people to mistakenly undervalue their Pages, change strategies or buy ads to make up for “lost” reach.

Facebook announced the bugs last week. During its efforts to speed up its iOS and Android apps in August and December, it accidentally stripped out the markers used by Page Insights to count impressions. This and other bugs led several metrics to be underreported. Fixes and bug-prevention systems are now in place, and accurate data began flowing into Page Insights yesterday, making today the earliest opportunity to get a concrete sense of the glitch’s impact.

EdgeRank Checker analyzed 1,000 Pages who posted both last week when Insights was still wrong, and yesterday with reach reported accurately. How much reach was underreported depends a bit on the Page’s fan count, as those with 500,000 to 1 million Likes were led farther astray.

A Page with 750,000 fans would have been reaching 47,000 fans per post (6.24 percent), according to the broken analytics, when in reality they were reaching 78,000 (10.35 percent). That means Insights was underreporting reach by a massive 66 percent. The impact was more subtle for a Page with 100,000 fans. The buggy Insights would have shown them reaching 10,000 of their fans per post when they were actually reaching 11,000, with reach underreported by 10.2 percent.

The most stunning part of the data was that the median Page’s viral reach according to Insights went up 275 percent after the fix. EdgeRank Checker‘s CEO Chad Wittman believes Facebook also fixed a bug that caused viral reach to exclude the impressions from posts re-shared by a Page’s fans.

No comments:

Post a Comment