What’s this about a Python SEO URL Checker?

So, I got really bored and frustrated checking a bunch of domains for a client to see if their domains 301 redirected all variants from http to https and from non-www to www. There are only so many times you can type:

curl -I http://domain.com

Using my limited knowledge of Python and not-so-limited knowledge of SEO, I crafted this simple little script to do the leg work for me and report back its findings in a readable format which can be slapped straight into an email that even devs can understand (j/k).

#SEO #SEOTools #Python #CanonicalURLs #Redirects

Python SEO URL Checker for Canonical Domains & Redirects

As I say above, I got very bored checking domain redirects for a client with a whole bunch of domains and sub-domains. This script uses the requests package, does a head request on a list of domains for each variant of http(s)://(www.)domain.com, lists out the responses if they are 301, 302, or 200 and if it gets a connection error, it spits out the details.

It’s rough code. I don’t code for a living, but I do occasionally code to make my life easier. This little beauty has already saved me minutes & hours. It comes with no warranty, express or implied, and no support. That said, if you have any ideas, or improvements, I’m all ears.

Feel free to use it & redistribute, but please leave my details intact. as you can see. (It’s in image format as WordPress wraps without leaving indentation intact. The image links to the python file in text format.

Python Canonical Domain Redirects

A python script to check canonical domain redirects.

This is the output for google.com:

As you can see the naughty people at Google 302 redirect (!) to a different country domain (!!). The 302 redirect isn’t great, although Google is moving towards “all redirects are equal”, but redirecting based on IP address? No, no, no, no. Naughty Google! Let the user go where they want to go!

Tested: http://www.google.com
    http://www.google.com/ 
    302 
    http://www.google.com.au/?gfe_rd=cr&dcr=0&ei=G3gNWv3fNLHM8geRx4KwDQ
Tested: http://google.com 
    http://google.com/ 
    302 
    http://www.google.com.au/?gfe_rd=cr&dcr=0&ei=HHgNWpaHF8zr8Af7wYpg
Tested: https://www.google.com 
    https://www.google.com/ 
    302 
    https://www.google.com.au/?gfe_rd=cr&dcr=0&ei=HHgNWvWcKaHM8gextLT4DQ
Tested: https://google.com 
    https://google.com/ 
    302 
    https://www.google.com.au/?gfe_rd=cr&dcr=0&ei=HHgNWtG-M8zr8Af7wYpg
Done google.com

When testing my domain, all redirects go via 301 to point to a single, canonical variant of the domain, which is what you want.

Job done!

The State of SEO Mid-2017 Released

The State of SEO Mid 2017

We recently released the super-exciting The State of SEO in mid-2017. Read it now.

Return to Top

TL;DR

  • Got bored of repeatedly checking stuff, wrote a little Python script to check if http(s)://(www.) redirect to a canonical variant of a domain.
  • It’s GNU GPL, so feel free to use it, redistribute or modify, but I’d appreciate it if you left the contact details intact.
  • If you do make it better, please let me know!
  • Read The State of SEO in mid-2017.
  • Read about how Google’s Mobile First Index is not Mobile Friendly.
  • Finally, get your content ranking well on Google by starting to understand Find Crawl Index.

Thanks for reading. If you would like to discuss SEO, or the weather, please feel free to contact me.

Return to Top