What’s this about a Python SEO URL Checker?
So, I got really bored and frustrated checking a bunch of domains for a client to see if their domains 301 redirected all variants from http to https and from non-www to www. There are only so many times you can type:
curl -I http://domain.com
Using my limited knowledge of Python and not-so-limited knowledge of SEO, I crafted this simple little script to do the leg work for me and report back its findings in a readable format which can be slapped straight into an email that even devs can understand (j/k).
#SEO #SEOTools #Python #CanonicalURLs #Redirects
Python SEO URL Checker for Canonical Domains & Redirects
As I say above, I got very bored checking domain redirects for a client with a whole bunch of domains and sub-domains. This script uses the requests package, does a head request on a list of domains for each variant of http(s)://(www.)domain.com, lists out the responses if they are 301, 302, or 200 and if it gets a connection error, it spits out the details.
It’s rough code. I don’t code for a living, but I do occasionally code to make my life easier. This little beauty has already saved me minutes & hours. It comes with no warranty, express or implied, and no support. That said, if you have any ideas, or improvements, I’m all ears.
Feel free to use it & redistribute, but please leave my details intact. as you can see. (It’s in image format as WordPress wraps without leaving indentation intact. The image links to the python file in text format.
This is the output for google.com:
As you can see the naughty people at Google 302 redirect (!) to a different country domain (!!). The 302 redirect isn’t great, although Google is moving towards “all redirects are equal”, but redirecting based on IP address? No, no, no, no. Naughty Google! Let the user go where they want to go!
Tested: http://www.google.com
http://www.google.com/
302
http://www.google.com.au/?gfe_rd=cr&dcr=0&ei=G3gNWv3fNLHM8geRx4KwDQ
Tested: http://google.com
http://google.com/
302
http://www.google.com.au/?gfe_rd=cr&dcr=0&ei=HHgNWpaHF8zr8Af7wYpg
Tested: https://www.google.com
https://www.google.com/
302
https://www.google.com.au/?gfe_rd=cr&dcr=0&ei=HHgNWvWcKaHM8gextLT4DQ
Tested: https://google.com
https://google.com/
302
https://www.google.com.au/?gfe_rd=cr&dcr=0&ei=HHgNWtG-M8zr8Af7wYpg
Done google.com
When testing my domain, all redirects go via 301 to point to a single, canonical variant of the domain, which is what you want.
Job done!
The State of SEO Mid-2017 Released
TL;DR
- Got bored of repeatedly checking stuff, wrote a little Python script to check if http(s)://(www.) redirect to a canonical variant of a domain.
- It’s GNU GPL, so feel free to use it, redistribute or modify, but I’d appreciate it if you left the contact details intact.
- If you do make it better, please let me know!
- Read The State of SEO in mid-2017.
- Read about how Google’s Mobile First Index is not Mobile Friendly.
- Finally, get your content ranking well on Google by starting to understand Find Crawl Index.
Thanks for reading. If you would like to discuss SEO, or the weather, please feel free to contact me.