Behind The Scenes In Google’s Battle Against Bad Ads

When new employees join the Google ad quality team that manually reviews suspect ads, they start by studying internal documentation of policies that outline examples of ads that would be approved, and those that would be rejected. Then the employees’ skills are tested on ads that don’t run, they graduate to ads that get little traffic, and finally they move up to ads expected to get lots of impressions. These are some of the tidbits revealed by Google director of engineering for the advertising team, David W. Baker, in a wide-ranging conversation this week.

Despite many years of being tight-lipped about the AdWords ad review processes, the company has recently begun to reveal more about the efforts it’s making to prevent bad ads from slipping through. The company has been criticized for years, and recently paid a record $500 million judgment to the U.S. Justice Department, for incidents in which illegal, counterfeit or otherwise malicious ads have been displayed by Google. Baker today published a second blog post, following up on�an initial effort at becoming more transparent around the team’s work.

Toward More Transparency

“I have been talking with my peers in different departments for about a year now about how I think Google has been remiss about being transparent about what we do,” said Baker. “There has been a fear that if we talk about these bad things it’ll draw attention to the negative.” Baker said the new communication is aimed at reassuring end users and advertisers about the measures the company takes to deliver quality ads.

The company is in a difficult position in a couple of different ways. If it cracks down too hard and takes too long to review ads, or rejects good ads, Google gets grief from white-hat advertisers. When bad ads slip through, it’s accused of a conflict of interest — because it ostensibly profits when ads run, even if they’re bad. “It’s an interesting position to be in,” Baker told me.

Bad Ads = Lower Revenue?

“Ultimately this [ad quality] I believe is Google’s biggest opportunity for increasing our revenue,” Baker confided, saying he believes that many users distrust Google ads and therefore don’t click on them. “That makes me very sad,” he said. If the company can gain that trust, it will result in more clicks, and more revenue for Google and for its advertisers, Baker says.

In his blog post, Baker outlined the company’s three-pronged approach to catching bad ads.

Ad Review

Ads — along with their individual landing pages — go through two levels of automated/technological review. If an ad fails the first level, it’s disapproved. Some are flagged for further review and are tagged as “Under Review.” If the ad and landing page pass, the ad becomes “Eligible” and begins to serve on Google.com to users with safe search turned off.

Then, there’s a second level of review for ads in the last couple of categories. If the automated system is confident it’s judging the ad accurately, it will either approve the ad for distribution to everyone (“Approved”), approve the ad for certain users in certain locations (“Approved – Limited”), or reject it (“Disapproved”). If the technological system can’t come to a determination, it goes to a real person — who undergoes extensive and ongoing training — to make a final decision. All human decisions are fed back into the machine-learning system to make it “smarter.”

Site Review

The site review system looks at all sites across all ads from all advertisers and regularly crawls them. The company uses machine learning models and a rules engine to analyze the database of collected information, trying to determine whether a site abides, or doesn’t abide, by Google’s policies.

“The challenge is that we get these constant variations where the bad sites are trying to change how they look but still achieve their objective — to make it clear to the user about illicit goods they’re providing, or, in the case of counterfeit sites, make it look like they are authorized to offer the goods,” Baker said.

If a site is determined to be in violation, it’s disabled, and any ads leading to that site are disapproved. If the automated system can’t be confident about a decision, it goes to a real person.

Account Review

The account review system looks at patterns in the account as a whole to determine whether it should be investigated further and shut down. The system looks at things like ads and keywords, budget changes, the advertiser’s address and phone number, the advertiser’s IP address, disabled sites connected to the account, and disapproved ads.

This is continually ongoing and incorporates new data as it comes in. If the automated system is confident the advertiser is up to no good, the account will automatically be suspended. If not, a real person will decide.

A Never-Ending Battle

And, yet, bad ads continue to slip through, and Google is still accused of not working hard enough to catch the bad guys.

“This is a constant battle, and we are constantly revising the systems that we’ve built,” Baker said. “This is just a huge problem of scale. There is a tremendous cost to us for making any small error…. also, shutting down a good advertiser can have a cost, as well, in damaging a relationship.”