Facebook Ads 80% Bot Claim, Examined! Why You Shouldn’t Lose Faith Just Yet

WHOA! Eighty percent bot clicks on Facebook Ads? Yikes!

Limited Run’s allegations of Facebook ad traffic being 80 percent bots has made major news publications, from the New York Times to CNBC. Responses from the industry have ranged from: “Holy Cow! GM pulls out, now this? I’m outta here, too!” to, “Big whoop. Show me the data.”

Hold on, dear marketers, before you go jumping the ship let’s back this Limited Run truck up to reality.

AimClear has served about a trillionty-gazzilion Facebook ad impressions since 2007, the inception of Facebook Advertising, and have not encountered 80 percent bot clicks. Even before data, we sense it. The KPIs speak for themselves.

The Limited Run claim might be read by some as laced with hyperbole. To illustrate our past experience, and research Limited’s claims, we performed our own Facebook Bot Study at aimClear.

Here are aimClear’s high-level findings. According to our data:

The number of visits Facebook reports from ads appears to be nearly accurate, even if the visitors are bots. That’s, umm… comforting.Possible bot traffic from the USA is much lower than Limited’s study purported.Internationally, possible FB Ads bot traffic is higher than it is in the USA but much lower than Limited reports.

Read on for details of our study and what it revealed.

Holes in Previous Claims

No one quite knows how long Limited Run’s Facebook stint was, how much they spent, where and who they targeted; we only know what they told us in regards to how they identified bots.

Here’s what we assume: Limited Run looked into their access logs. They pulled out Facebook traffic but did not say if it was for their paid campaigns or organic. We assume they are talking about Facebook Ads traffic because that’s the foundation of their claim.

They also looked at overall traffic (again, without saying if it was overall paid and/or organic) and assumed that only 2 percent real users disable Javascript. This assumption jibes with Yahoo’s study of users and may or may not be true in the country mix Limited Run markets to. We don’t know. They haven’t revealed that data. Perhaps they should. It’s significant.

The entire basis of Limited Run’s study surrounds the assumption that total Javascript disabled traffic should be about 2 percent. After deducting the estimated 2 percent of assumed Javascript disablers, they then inferred that approximately 80 percent of visitors were bots. This assumption may be inherently flawed.

Javascript Only 2%?

Looking at aimClear’s total organic traffic from every source, which excludes Facebook ads and what we can see in Google Analytics, approximately 63 percent of our total organic site visitors have Javascript disabled and is not declared as a bot, spider, or crawler.

Is the Javascript disabled traffic 2 percent, or is it closer to 60? Are they real users with Javascript disabled, or are they bots? It is likely that some or many of these are malicious crawlers for various reasons and 2 percent or more are real with Javascript disabled. It is possible that more of the organic universe disables Javascript than 2 percent and that certain percentage variations would apply to various countries.

The point is, Limited Run is comparing Facebook ads traffic to real users (assumed at 2 percent), not total traffic (which we know can be a much higher percentage). Using 2 percent as a benchmark may be a flawed start to this analysis.

Case in point: our data says that the majority of our organic traffic has Javascript disabled. The potential flaw in Limited Run’s thinking is that they are comparing Javascript disabled Facebook traffic to Javascript disabled total real users.

The Test

We set up our website to record all traffic and tested a handful of our branding target segments. Since the specifics (e.g., geo targeting) of Limited Run’s campaign were not revealed, we split the test up. One group of campaigns was targeted only the U.S. The other group held the same targeting, same ads, but used the same geo-targeting Facebook’s Promoted Post targets by default, swapping the U.S. for the notorious bot-ridden, Turkey (bot-bait, if you will):

We logged roughly 950 visits over a four-day period from August 3 to 6. We then took and sliced off the first and last day so that we could accurately match our daily data with the daily data Facebook reported. Over those two days we had a total of 765 visits. Those are the visits that we studied.

Bot-Testing

AimClear used a Javascript test to determine potential bots versus real users. We recorded every visit, logged as much visitor data as possible, and then asked every visitor to execute a jQuery post and recorded those visits as well. By comparing the two numbers and filtering out any user-agent that declares itself a bot (spider or crawler) we were able to determine the total percentage of visits without Javascript enabled.

Time and Date Stamps

It’s also incredibly easy to accidently assume Facebook is misreporting ad traffic if time zone differences between Facebook reporting and server logs are not taken in to consideration. In our test Facebook reported ad clicks daily based on Central Standard Time (CST) whereas our server logged visits in Greenwich Mean Time (GMT).

To accurately study the data between our server and Facebook we needed to shift our recorded dates back 5 hours. Depending on the server and Facebook ads configuration the date shift could be much larger.

If you don’t correct for the time zone difference, especially in smaller size samples, it is likely that your reporting will be heavily skewed, if not dead wrong.

The Results

Within a few hours of launching the bot-bait international ads, we had bot clicks. And some solicitations.

When we analyzed Facebook paid traffic, we found that approximately 19 percent of U.S. traffic and 30 percent of international traffic has Javascript disabled and falls into the potential bot bucket. This percentage is much cleaner from Facebook ads than outside the wall, where 63 percent of organic site visitors arrive with Javascript disabled.

High Level FindingsThe number of visits Facebook reports from ad appears to be nearly accurate, even if the visitors are bots.USA Bot traffic could be as high as 17 percent if the 2 percent rule holds. Our guess is that the number is at least slightly lower.Internationally, bot traffic could be as high as 28 percent, with the same caveats.However, international clicks can cost a quarter as much as the USA or less, so the bot percentage may be somewhere between acceptable and better than America net.Limited Run’s conclusions are way off the data we see. They reported a total of about 80 percent bot traffic. They need to show some more data to hold the attention of anyone other than skittish social media paparazzi and freaked out investors.

We call on Limited Run to show a lot more data, including country segmentation and ratios,. Further, we challenge them to disclose how they got around the time/date issues to avoid mingling data. Dear Limited Run, please prove to us that you did not somehow merge datasets.

Limited Run also did not say how much they spent or the size of their sampling. That 80 percent could be 4 clicks or 400 clicks. So… how big was the sampling?

It is also funny to note that they apparently did not think this would be a big deal, even though they accused the largest social community on earth of not cleaning up fraud. The big winner here is Limited Run. They got all the PR.

Facebook Still Losing the PR War

Seasoned online marketers are aware of bot clicks on ads from Google, to the Display Network, to Bing, and now (not shockingly) Facebook. What we did find interesting was the paid Facebook traffic was cleaner than organic.

Also, Facebook has stated they expunge any spam or bot accounts when they find them. We believe that and believe it true from first hand observations.

“If we find fake accounts, we disable them immediately,” a Facebook spokesperson told SEW.

Facebook revealed, mere days after the Limited Run story, that they identified 83 million fake accounts. Hold the phone, marketers! Just because there are fake accounts does not necessarily translate to bots.

These were accounts Facebook identified as duplicate accounts (the majority at 4.8 percent), “user misclassified” (users who had a business & personal account at 2.4 percent), and (dun dunn dunnn) 1.5 percent of “undesirable accounts,” which breach fundamental terms of agreement.

Spammers? Bots? Maybe.

The revelation of these numbers only adds to the perceived crisis as headlines highlighted 83 million, not the real issue of the 1.5 percent “undesirable accounts,” a relatively small amount.

Bots and spammers exist. Why didn’t they only reveal this number? Not sure. We appreciated the transparency, but as a result, the frenzy surged on.

Let’s not forget that we’re still talking about 10 percent of just under a billion, so there are plenty of real people left to play with.

Should we let Facebook off the hook, now that we know their ad traffic is cleaner than our organic traffic and only a measly 1.5 percent of Facebook’s 955 million accounts may be spammers/bots? I don’t think so.

Facebook needs to bite the bullet and refund known bot clicks, just as Google does. They’ll clean it up. Facebook has said they have the technology.

We suggest that Facebook offer Facebook Ads customers greater transparency as to the process and a dedicated method to get refunded for verifiable bot analytics. Google has been down this road. Facebook will, too. Fortunately, Facebook has stated, there are already systems in place to identify users’ click patterns:

“Facebook has systems in place that attempt to detect and filter certain click activity, including repetitive clicks from a single user, clicks that appear to be from an automated program or bot, or clicks that are otherwise abusive, their spokesperson said.”

Since Facebook has the technology to identify these “undesirable users,” and has the click data, perhaps they should take them in to consideration when charging clicks and assessing quality scores.

Facebook Advertisers’ Real Gripe is Revealed

It’s no surprise the industry was abuzz over Limited Run’s claim, with GM ditching Facebook, the (laughable) BBC “investigation”, and plummeting-as-we-speak stock prices post their IPO. Some assume this may be just another nail in the social network’s coffin.

Facebook could have prevented this or at least curbed the backlash by supporting their advertising communities in real ways, not stagnant FAQs with elementary advertising suggestions, user-to-user help forum with no Facebook representative, or a support email that barely anyone knows about ([email protected]/* */).

It’s the smaller advertisers who need help. They’re the ones who get freaked out when it looks they’ve been ripped off by a billion-dollar company that doesn't care, not the giant multinationals who are spending millions per quarter.

Facebook will lose more and more advertisers if they continue to ignore the small and medium advertisers, and that affects Facebook’s bottom line more than a company no one has ever heard of, announcing their leave of the biggest social scene.

Become an Expert Digital Marketer at SES New York
March 25-28, 2013: With dozens of sessions on Search, Social, Local and Mobile, you'll leave SES with everything and everyone you need to know. Hurry, early bird rates expire February 21. Register today!