Understanding Web Crawling: Explore Insights into Googlebot

 

Explore insights into Googlebot and its many forms with Belmore Digital’s Expert Digital Marketing & SEO Consultancy.

AI Engine Crawlability: The GEO Problem Most Businesses Don’t Know They Have

2026-06-08T17:48:08+10:00Categories: GEO - Generative Engine Optimisation, SEO, Technology & Development, Virtual Chief Digital Officer|Tags: , , , , , |

If AI systems can't read your site, everything else in your GEO programme is built on a gap you haven't checked. Here's what blocks AI crawlers and how to find out if it's happening to you.

JB’s Digital Update 11th June 2024

2024-06-12T06:40:01+10:00Categories: Analytics & Performance, Email & Customer Relations, JB's Digital Update, SEO, Social Media Management, Technology & Development, Virtual Chief Marketing Officer|Tags: , , , , , , , , , , , , |

This week's update casts a gimlet eye over Google's latest AI Overview changes, customer loyalty motivations, mobile indexing issues, LinkedIn's new video ads and AI tools for B2B, and optimising your email programs. A mix of AI enhancements, marketing strategies, and practical tips to keep your digital efforts sharp and effective. Read on and let's get cracking.

JBs Digital Update 17th Oct 2023

2023-12-07T15:39:23+11:00Categories: Content Strategy & Creation, CRO and Conversions, Digital Marketing & Strategy, JB's Digital Update, PPC & Paid Social, SEO, Social Media Management, Technology & Development, Virtual Chief Marketing Officer|Tags: , , , , , , , , , , , , , |

This week's digital fun and frolics take us to Google's SGE using your content unless you block Googlebot entirely; Google Ads releasing Demand Gen ads; thoughts on the dullness of B2B social media, the importance of Customer Lifetime Value; and some reminders about how to make content *interesting*. All that wrapped up in an amusing bundle by yours truly. Read it now, without delay, and enjoy. 

JB’s Digital Update 26th September 2023

2026-04-21T20:52:54+10:00Categories: Analytics & Performance, Content Strategy & Creation, CRO and Conversions, Digital Marketing & Strategy, Email & Customer Relations, JB's Digital Update, PPC & Paid Social, SEO, Social Media Management, Technology & Development, Virtual Chief Digital Officer|Tags: , , , , , , , , , , , , , , , , , |

It’s all money, money, money this week with X and Meta going pay-to-play in different ways; Google upsetting some people with auto GA4 setup before bringing it home with a handy ecommerce report; and finally some useful insights on Landing Page Optimisation from the Think with Google team, just in time for the Spending Season. Good stuff to read / watch / listen to. Check it out now.

JB’s Digital Update 29th August 2023

2026-04-21T20:54:52+10:00Categories: Analytics & Performance, Content Strategy & Creation, Digital Marketing & Strategy, JB's Digital Update, PPC & Paid Social, SEO, Social Media Management, Technology & Development|Tags: , , , , , , , , , , , , , , , , , , , , |

What an exciting AI filled week it’s been. Just like many other weeks recently. Some brands are now blocking GPTBot (until offered cash, no doubt), Lyft is slapping ads into its app, Meta is being useful, Google is actually stopping UA processing (sniff), and there’s been another Google Core Update to pontificate over. Interested? Dive in for more. 

Google Dumping AJAX JavaScript Crawling Scheme

2023-12-05T17:54:09+11:00Categories: SEO News, Technology & Development|Tags: , , , , , , |

What’s this about Google, Ajax and JavaScript?  Hooray! At long last Google is dumping escaped fragment URLs. JavaScript developers the world over can breathe a sigh of relief. Unless they ignored Google’s warning two years ago that they were going to End of Life it. [...]

Google: Internal Link Anchor Text Matters

2023-12-07T15:23:06+11:00Categories: SEO News|Tags: , , , , |

In a stunning revelation, Google has said that internal link text matters. Or at least it's part of the mix. Or how about this for a wild and crazy idea? Labelling links correctly helps users and search engines contextualises the link and provides a better user experience for all concerned. Read all about it in this extract from TWIS SEO News & Updates 24th November 2017. #SEO #SEONews #InternalLinks #UX #Googlebot

Googlebot Doesn’t Crawl HTTP/2

2023-12-05T20:16:23+11:00Categories: SEO News|Tags: , , , , |

Despite championing HTTP/2 and the speed advantages that come with it, googlebot doesn't crawl pure HTTP/2 connections, or so it seems. It's a bit arcane and technical, but it's also a bit surprising. Read more in this extract from TWIS SEO News & Updates w/e 20th October 2017. #SEO #SEONews #Googlebot #Crawling #HTTP2

TWIS SEO News & Updates w/e 20th October 2017

2017-10-23T20:47:18+11:00Categories: SEO News, The Week In Search|Tags: , , , , , , , |

I think the SEO News and Updates for w/e 20th October make for interesting reading as it covers off these fine stories. Read it all now, and enjoy being ahead of the game: Google Posts Now in GMB API; Google Maps Adds And Removes Cupcakes Swiftly; Googlebot Doesn't Crawl HTTP/2; New SiteLinks Being Tested; Food Ordering Comes To Facebook; As well as some exceptionally interesting SEO Bits & Pieces. #SEO #SEONews #GooglePosts #GoogleMyBusiness #GoogleMaps #Googlebot #SERP #Facebook

SEO Bits & Pieces w/e 22 Sept 2017

2023-12-05T20:17:22+11:00Categories: SEO News|Tags: , , , , , , |

I like this week's SEO Bits & Pieces. It ranges across the gamut of small-scale SEO news and updates. There's a bit of Google update chatter, a touch of EU vs Google, some tidbits from Google engineers, and a couple of "teaching your grandmother to suck eggs moments". Follow the link to read the ins and outs of this week's SEO Bits & Pieces. #SEO #SEONews #GoogleUpdates #EUFine #Googlebot #MobileFirst #HTTP2

JavaScript Sites Lose Out on 35% of Traffic and Revenue

2017-09-06T17:06:16+10:00Categories: SEO News|Tags: , , , , |

Woah Nelly! I am shocked by this... You mean JavaScript is not the best thing for SEO since sliced bread? I can't believe I'm writing this: JavaScript is sometimes readable by Google, but the other search engines? It's like they haven't even heard of JavaScript. Read the full story from TWIS SEO News & Updates 01 Sept 2017. #SEO #JavaScript #Googlebot #WebCrawling #Indexation

SEO Bits & Pieces w/e 25 August 2017

2017-08-29T11:05:41+10:00Categories: SEO News|Tags: , , , , , , |

Click to read the latest SEO Bits and Pieces and find out what entertaining things Google and other have been up to. Contact me to discuss these Bits and Pieces and how implementing them might help your website to rank and perform better. #SEO #SEONews #Apple #Googlebot #GoogleSearchConsole #GoogleUpdates #MobileFirst

Web Rendering Service: Google Reveals How It Works

2018-10-19T14:24:36+11:00Categories: SEO, SEO News, SEO Tools, Technology & Development|Tags: , , , , , |

Read this guide on how Googlebot's Web Rendering Service works, and tips on how to set up your website to be easily crawled and indexed by it. Contact me to discuss how best to get your content into a state where it can be Found, Crawled and Indexed - that is critical for surfacing in SERPs. #SEO #Googlebot #WebCrawling #GoogleIndex

TWIS SEO News & Updates w/e 04 August 2017

2017-08-08T12:08:38+10:00Categories: SEO, The Week In Search|Tags: , , , , , |

Read the TWIS SEO News and Updates for w/e 4th August 2017. This week there are a bunch of interesting stories to read, including: - Google Reveals Two Secret Google Search Console Features; - Google Demonstrates How Googlebot Web Rendering Works; - Two Google Algorithm Flaws Exposed; - Only 20% of Clicks Now Go to No.1 - New CTR Study; - Are You Checking Your GMB Pages for Mystery Updates?; - As well as some fabulously fascinating Bits & Pieces. #SEO #GoogleSearchConsole #Googlebot #WebSpam #LocalSEO #ClickThroughRates

Go to Top