Googlebot Doesn’t Crawl HTTP/2

Despite championing HTTP/2 and the speed advantages that come with it, googlebot doesn’t crawl pure HTTP/2 connections, or so it seems.

It’s a bit arcane and technical, but it’s also a bit surprising.

Read more in this extract from TWIS SEO News & Updates w/e 20th October 2017.

#SEO #SEONews #Googlebot #Crawling #HTTP2

Key Issues Summary:

• The Moz contributor “goralewicz” ran an innocuous, but important test to see if googlebot would crawl pages which were served only over HTTP/2.
• It didn’t.
• This means you cannot switch HTTP/1.1 off yet.
• It also calls into question Google’s commitment to HTTP/2 – based on SPDY which they created and are now in the process of dropping.

Key Actions To Take:

1. Do not switch HTTP/1.1 off, unless you want Googlebot to not crawl your site.
2. Continue to optimise HTTP/1.1, as well as HTTP/2.
3. Click here to contact me to discuss website speed issues and how to improve it.

Insights & Discussion:

We should all know by now that website speed is a critical factor in usability, and will be a significant factor in the upcoming Google Mobile First Index. If your website loads slowly, users will bounce back, or abandon their search. This annoys users, and ultimately, annoys Google, as they are not giving their search users a good experience.

That said, although the ins and and outs to HTTP standards are deeply arcane and technical, it is a still a tad surprising that googlebot will not deal with a connection which is made solely over HTTP/2. It should in theory be a vastly more speedy connection and should make everything move along much, much faster.

I haven’t been able to find the tweet. but I do recall John Mueller saying something along the lines of “googlebot doesn’t need to use HTTP/2” as it has alternative ways of speeding its downloads. Hmmmm.

More Information:

• Google: PageSpeed Is Difficult
• Google Test My Site Tool Updated
• Moz’s HTTP/2 Googlebot Test

Return to Top

We recently released the super-exciting The State of SEO in mid-2017. Read it now.

Return to Top

• No HTTP/2 for you Mr Googlebot!
• For lots of reasons, some no doubt valid, some spurious, googlebot doesn’t crawl as pure HTTP/2.
• That’s annoying if you only want to serve superfast connections.
• Read The State of SEO in mid-2017.
• Read about how Google’s Mobile First Index is not Mobile Friendly.
• Finally, get your content ranking well on Google by starting to understand Find Crawl Index.

Thanks for reading. If you would like to discuss what these changes mean for your web property, or would like to know how to implement them, please feel free to contact me.

Return to Top