Google '(Not Provided)' Keywords: 10 Ways to Get Organic Search Data

SEO professionals have been dealing with the "(not provided)" issue in Google Analytics for a while. Despite initial claims by Google that it would only affect a small percentage of overall analytics data for organic search, the reality is some sites were seeing well over 50 percent "(not provided)", particularly in sectors like technology where there is a higher likelihood that a visitor would be logged into their Google account as well.

While not ideal, many websites still had a decent chunk of keyword data from Google organic search visitors that while it was annoying, it wasn't causing significant issues when it came to determining which keywords were the most important, converting the best, and had the highest ROI – as well as to determine where an SEO budget should be allocated.

All of this came to a crashing halt September 23. Google flipped a switch and made all Google searches secure, regardless of whether a searcher was logged into a Google account.

What does this mean? Suddenly, organic analytical data is nonexistent for Google traffic, and many webmasters are seeing 95+ percent of keywords being reported as "(not provided)", with that remaining 5 percent coming from non-Google searches.

What is Secure Search & Why Did Google Implement it?

What exactly does it mean when Google searches are secure? When a user goes to Google to search, they are automatically redirected to the https:// version of their Google domain of choice.

When someone conducts a search on Google, it is now always done as an SSL encrypted search, so all search queries are now encrypted. It routes the click to the website through a redirect so that the website a user lands on has no idea what actual keywords the searcher found the website under, and what keywords were used that brought the person there.

This is great from a privacy perspective, and is why Google is making the change. However, only natural search clicks are affected; Google AdWords advertisers still have full access to the keywords on every ad click made on their ads.

There has been plenty of speculation, running wildly from "the NSA told them to" to "they did it to force people to buy more AdWords ads!" Tin foil hats aside, neither of those options make a good case for why Google made the secure search change.

First, Google has been making major steps to protect everyone's privacy, to prevent eavesdropping from government agencies throughout the world. Google has filed an amended petition in the U.S. Foreign Intelligence Surveillance Court for greater transparency.

As for Google doing it for more ad revenue, when a business or website dominates natural search results for their chosen keywords, unless it's for branding or market saturation, they don't tend to dump a ton of money into AdWords as well. With the "(not provided)" change, I could see some businesses using AdWords to test new keyword market areas, but not as a huge budget for strictly keyword collection.

There is a possibility that after some time has elapsed, that Google might release a new "premium" version of Google Analytics where users would pay a monthly fee in order to get access to full keyword data from natural search referrals. As a public company always looking for new revenue streams, this would definitely be a way for Google to do it, and many SEO professionals would definitely pay the fee while others would likely refuse on principle for data that used to be free.

How Does Secure Search Affect SEO?

From an SEO perspective, this means whenever a user lands on your site, you have absolutely no idea what the person was looking for when they landed there or if they found it. And if your customers placing orders, you have no idea which organic keywords are converting into sales, because you can't track that from Google referrals anymore.

It also means that there is a high bounce rate for a page, aside from doing some landing page optimization, you can't make any adjustments since you not know precisely what the user was looking for when they got to your page in the first place and seemingly didn't find.

You can't look at referring keywords to try and discover what they were looking for, and how you can change things so that the next user can actually find it. So the change is actually likely to result in a worse user experience for searchers from that perspective, since in the past many webmasters and SEO professionals made that adjustment so that their visitors could find what they were expecting.

Ever since Google made the change to use secure searches for those logged into their Google accounts, it was only a matter of time before the switch to where all searches were secure. This wasn't a huge surprise, although granted it was an unwelcome one, as many had been planning for the day "(not provided)" became a ~100 percent reality.

Replacing '(Not Provided)' Data

There are some workarounds you can use in order to capture that traffic. Here are some valuable ways you can look at replacing the data now that Google isn't providing it.

1. Learn from Avinash Kaushik

In 2011, Avinash Kaushik, one of the world's top analytics experts and an evangelist for Google, wrote a blog post "Smarter Data Analysis of Google's https (not provided) change: 5 steps" detailing the things marketers can do within their Google Analytics account when Google first turned on the search encryption for logged in users. It is for high level Google Analytics users, but he goes over it step by step, and is well worth the time setting it up.

2. Look at Non-Google Keywords

Look at your non-Google search engine referral traffic. While Bing doesn't send a significant portion of traffic for a site will looking at overall search, you still can get an idea of what keywords are bringing traffic, particularly for sites that have significant amount traffic. However, for smaller sites, this isn't ideal.

If you're in a country where Google doesn't dominate the search market, you will have the advantage.

3. Use Google Webmaster Tools

If you've gone this long without using Google Webmaster Tools, you will definitely want to sign up for it now. You can still get keyword data in Google Webmaster Tools, by clicking on Search Traffic > Search Queries, and yes, it does include search data from encrypted searches.

There are some filters, so you can drill down things like web versus mobile. It only goes back for a limited period of time, currently only 90 days.

You should make a point of going into your account every 30 days to save your data, as it can be downloaded in both CSV and Google Docs, so you do have access to it forever.

It also gives you impression versus click data, which is data Google never gave webmasters in Google Analytics, so it can help you see what keywords people are using where your site got an impression in the search results, but the user didn't pick your site for whatever reason.

Using Google Webmaster Tools is currently the only reliable and accurate way to get keywords from Google encrypted search.

UPDATE: Search Query data has become unavailable, although Google's John Mueller has confirmed this is a bug.

4. Analyze On-Site Searches

You should capture all the on-site searches that visitors are doing on your site, so you can also get a better idea of exactly the keywords they are for. Crazy Egg has a detailed post on how to set up internal site search analytics.

5. Use Google AdWords

Using Google AdWords will still be useful, particularly if companies are trying to determine the value of either high traffic keywords, or keywords they suspect are the ones that are converting well and want to smart check it before they throw more budget into the organic SEO side for those keywords.

At this time advertisers still get full keyword referral data from Google, while there is speculation this could change sometime in the future, there is also the necessity for advertisers to be able to determine conversions from the traffic they are paying for.

Paying for AdWords doesn't get you any referral data for organic search keywords, even if a user clicks on an organic search result for your site when an ad for your site also appears.

6. Use Search Volume Tools

Particularly if you're looking to expand into new keyword areas, you can use the Google Ad Planner to determine search volume and estimated clicks (on ads) for particular keywords.

7. Use Filters to Segment '(Not Provided)' Data

Seeing that you have a 97 percent "(not provided)" rate isn't very useful for webmasters. But you can turn that data into something a bit more meaningful and helpful.

You can set up filters for all your "(not provided)" traffic so that it shows you the landing page for each of those "(not provided)" referrals. So even though you might not know the exact keyword that's bringing in the traffic, you can instead filter it so that you have a bit better idea of what the user was searching for when they landed on the page.

While not ideal, it does give more analytical data back to the webmaster. Econsultancy has a detailed post on how to set up "(not provided)" filters for landing pages in "How to steal some ‘not provided' data back from Google".

8. Look at Historical Data

You will still have all your historical data for keywords prior to the SSL switch to Google. While you won't have access to the new data, your old data is still going to be valuable, especially for so-called "evergreen" data that has been performing well consistently for a long period of time.

You can also use your historical data to look at seasonal content, such as seeing what keywords and content performed well over Christmas 2012 and apply that to the potential keywords and content for the same period this year.

9. Use Google Trends

Using real-time Google Analytics has been a great way to be able to spot content on the site that is trending with referrals from Google search. But without the keyword data, you'll be stuck kind of guessing at the potential keywords that are bringing in people to a page on your site.

Sometimes guestimating can work if you have a suspicion as to why people are landing on a specific page on your site. Other times, however, the page can cover enough keywords or be generic enough that you'll struggle to figure out which keywords are causing your traffic spike.

If you do notice a huge spike in traffic and you suspect it could be something newsworthy or trendy, but the majority is "(not provided)", checking Google Trends can possibly give you an idea on what exactly it is that is trending that is bring you traffic. While this isn't as easy as it used to be, it can help you if you're struggling to figure out what specifically is bringing in large amounts traffic and why.

10. Additional Ways to Capture Keyword Data

SEOORB has collected a list of 11 actionable posts on other ways you can try to determine some of the keyword data that is missing with the new SSL change.

The Future?

Whether we like it or not, Google's encrypted search is here to stay. Complaining to Google about how unfair it is won't make them change their stance because this was a security decision.

Now that Google has made the change to secure search, I wouldn't be all that surprised if other search engines, such as Bing, also make a switch to encrypted search as well, meaning that small percentage of keyword data you are still getting from search engines will be reduced even further.

The abilities and role of an SEO have been changed dramatically. It will never be as easy to get the organic search data as it has been up to this point, with the possible exception of Google making a paid version of Google Analytics that does contain that missing (not provided) data.

It will be the SEO professionals who are able to adjust and compensate for the missing data who will win the rankings game in the end.

 Learn More: Keyword Data, Secure Search & "(not provided)"Google Webmaster Tools Search Query Data is Accurate (and Valuable)5 Ways to Create Productive B2B SEO Benchmarks in a 'Not Provided' World5 Tips for Handling (Not Provided) DataHow to Understand Your Google “Not Provided” TrafficGoogle Encrypted Search: 9 Key Points B2B Marketers Need to Know

Image Credit: AT Internet Blog

The SES agenda focuses on aligning paid, owned and earned media to help you drive quality traffic and increase conversions.

SES Chicago is coming up!Book your pass to SES London
3 keynotes, plus over 60 sessions Super Saver rate ends Nov 1