Got a new job recently and part of my responsibility will be to increase inbound inquiries by increasing site traffic – which in the immediate term means better organic positioning.
So I’ve embarked on a journey to optimize the website through different means. Some background info: I’ve been working on some SEO / SEM as a contractor for this website for a while, but now that I am actually part of the organization, I have to step up my SEO game and really dive deep into the site.
First thing I did was to identify all the strengths we have going for this website: 5,000+ product pages, monthly addition to page catalog, search engine friendly URL and META data, re-deployable, educational content…
So some of the things that is not so good: Very competitive keyword set, non e-commerce website (catalog only), low overall organic keywords positioning.
Day 1: What the competitors are doing
Perhaps the most telling piece of information I’ve gathered so far on their organic search competitors are their link count and authority. After a quick visit to the Open Site Explorer, here is what I’ve found:
My initial thought was – wait, this is not right, something is missing, how can me with so much more inbound links be ranked so much lower then my competitors? But then I saw my Domain and Page authority isn’t exactly the best either, so there clearly needs work…
I sampled several links from all competitors and realized that many of these links are from blogs, not just in the no-follow comment sections but mentioned directly in the articles. This brings me to strategy 1: Get some blog connection with relevant content for link-backs.
Next to took a trip down Google Webmasters tools. This is frankly a bit of a surprise for me as I realized the sitemap file I auto-generated before isn’t really doing a good job of telling Google what to crawl and what not to. Since I have Magento Community as a content engine, there’s a lot of directories related to the system that Google shouldn’t be crawling… So I had to update the robot.txt and disallow crawler access to the directories. Then I decided to scrap the auto-generated sitemap route and manually create one to see if I can improve my indexed page count and really get Google’s attention to those important landing / category listing pages. Two things I’ve learned form this exercise:
- Set up individual Update Frequency Correctly: Based on this blog post about Google Sitemap Update Frequency and Sitemap Stats, my product pages should be set to “Never” for Update Frequency, and “Monthly” for category listing pages when I plan to upload new products monthly. This way I can get Google to crawl more pages, new pages, and important pages then everything at “Daily” or “Monthly”.
- Set the Priority straight. I’ve noticed that I put almost all pages at Priority 1, but when I think about it, the scale of 0 to 1 is really relative, why would I want the never changing about pages to be priority 1, same as the category landing pages which I plan to use as major landing pages for keyword optimization? I switched that around and place a priority to close to 0 for these About Us pages and close to 0.9 and 1 for several key category pages.
I re-submitted the sitemap to Google and almost immediately the Google picked up all pages I have in my sitemap files. Then I demoted some URL Google currently has as sitelinks. I’ll see what effect this will have for Google sitelinks and overall result in organic search in about two weeks time.
Next is a big one. Google Webmaster Tools reports that I have 1,046 URLs that is not found by the crawler, 865 restricted by robots.txt and 46,112 unreachable… This is frankly worrying because it effectively telling Google that my site has a lot of broken links and crawl rate will likely decrease. Upon further examination all of these 1,046 URLs are from the old version of the site! So I have to first re-direct all these pages to the appropriate pages, and also have Google remove these URLs from its index.That’s going to take hours if not days, but nonetheless must be done because it’s hurting my ranking just having these around. Steps on how to remove them can be found here