What’s this about a Web Rendering Service?
This is an extract from TWIS SEO News & Updates 04 August 2017.
Read this guide on how Googlebot’s Web Rendering Service works, and tips on how to set up your website to be easily crawled and indexed by it.
Contact me to discuss how best to get your content into a state where it can be Found, Crawled and Indexed – that is critical for surfacing in Google’s rankings.
#SEO #Googlebot #WebCrawling #GoogleIndex
Google Reveals How Googlebot Web Rendering Works
Summary:
- In an update to Google’s developer documentation, they have revealed how Googlebot’s Web Rendering Service works, and importantly limitations.
- Googlebot’s WRS is based on Chrome 41. Meaning, if it works in that version of Chrome, it should work for Googlebot.
- WRS and Googlebot only supports HTTP/1.x and FTP. There is not WebSocket protocol supported yet.
- Googlebot and WRS are stateless across page loads – no cookie interaction or retention.
- Any permission requests are declined by WRS.
How to Test Against Google’s Web Rendering Service:
- Review the list provided by the Google Developer’s Guide. There are other limitations, especially around feature detection.
- Test your content in Chrome 41. If it works there, it should work for Googlebot’s WRS.
- Don’t try to serve Googlebot or the Web Rendering Service non-HTTP/1.x, or FTP content.
- Don’t try to force cookies onto Googlebot. It’ll take them, but will clear them. If pages require cookies for content, there will be nothing to read and index.
- Don’t think Googlebot will tick boxes for permission, or allow things to happen. It won’t.
- Contact me to discuss how best to get your content into a state where it can be Found, Crawled and Indexed – that is critical for surfacing in SERPs.
Web Rendering Service Insights & Discussion:
Many people get confused by Googlebot, and its extension the Web Rendering Service. The Web Rendering Service is essentially the headless browsers which turns HTML & JavaScript etc into something with can be displayed to a person, or interpreted by an algorithm.
It comes as no surprise that the Web Rendering Service is based on Chrome. Why else would you have a browser? Or possibly more accurately, if you have a browser, why wouldn’t you turn it into something which is used by Googlebot.
It is important to remember that Googlebot and the Web Rendering Service come to each page as a fresh user would, with no cookies, no settings, no permissions allowed, and it doesn’t do anything to change that. It’s likely that Googlebot eats the cookies and inspects them to see what they do, but it takes no action. It is also likely that Googlebot has an incognito crawler which goes out and crawls the web to see what happens if it does eat cookies, click permissions and allows for different settings. It’s vital to understand the web as a raw user, and as a returning user. This helps to defeat cloaking and other nasty SEO practices.
More info:
Mobile First is NOT Mobile Friendly
TL;DR
- Read The State of SEO in mid-2017.
- Read about how Google’s Mobile First Index is not Mobile Friendly.
Thanks for reading. If you would like to discuss what these changes mean for your web property, or would like to know how to implement them, please feel free to contact me.
