Stories about Software


Logs for SEO

Editorial Note: I originally wrote this post for the LogEntries blog.  You can check out the original here, at their site.  While you’re there, check out the features of their offering.

I write blog posts on a number of different sites that are not my own, and that is an exercise in pure writing joy.  I compose the posts, I submit them, and viola!  They’re published on nice-looking sites, promoted by people with reach, and read by many (hopefully) interested readers.  Life is good.

By way of comparison, when it comes to my own blog, life is not quite so simple.  On my own blog, I have to write the posts and manage all of the details that are abstracted away when I write for other sites.  So many things distract from content.


There are the major ancillary concerns like the site’s look and feel and following up on any downtime or outages.  There are minor ancillary concerns, like checking for typos, promoting the posts, and making sure no one is inappropriate in the comments.  And then there are enigmatic ancillary concerns, like search engine optimization (SEO).  My primary concern is content generation, however, so, even with my own site, I seek to abstract as much of this away as possible.

The Weird World of SEO

Let me start off by saying, emphatically, that I am not an SEO expert.  Frankly, it’s not a topic that particularly interests me in and of itself… at least not until I had to be interested in it.

I have a blog as part of my website, but from that same website, I offer information about my consulting practice and about books and products that I offer.  More readers translates into more engagement, which, in turn, translates into a better living for me.  And it was against this backdrop that I became interested in SEO by default.

SEO actually reminds me of the credit score concept, in which a mysterious agency uses a mysterious algorithm to compute a score that has a serious effect on your life.  The mystery and complexity of the algorithm and proprietary nature of the score, in turn, create a cottage industry of advice and services aimed at helping you get just a little bit better.

As a blogger and entrepreneur, this is the SEO world for me.  Google (and nominally other search providers) have secret sauce algorithms that figure out how to rank content based on its likelihood of being valuable to people using the search engine.  I don’t care much about these for their own sake, but I do wind up having to learn enough about how the whole thing works in order to make (what I hope are) informed decisions on how to position myself.  Oh, and I have to do that while not wasting a whole lot of time on it and veering into the land of diminishing returns.

WordPress to the Rescue?

As I’ve already mentioned, SEO isn’t the only important and ancillary concern to my main goal.  It is all of this ancillary stuff that led me not to reinvent the wheel and to adopt WordPress, in spite of the fact that I’m a programmer capable of writing my own website.  Wordpress is a blogging (and, really, site-creating) engine that handles most of the ancillary concerns on my behalf, abstracting them away from me almost as much as if I were writing content for someone else.  Wordpress even includes some built-in functionality around SEO itself, offering basics like pairing strategic keywords with the content on any given page/post on your site.

That’s all fine and good until you read some buzzfeed article called “10 SEO-murdering Mistakes that are Ruining Your Life” and see that you’re making 8 of them.  Oh my goodness!  How much money am I leaving on the table?!  What can I do?!

Well, WordPress does more than just make skinning and content creation a snap — it also offers a rich plugin community that can help with concerns beyond just creating the site itself.  These concerns include SEO… in spades.  A search of the plugin index for “SEO” yields an impressive 1.5 million results.

Since that’s not especially helpful, you then head out and find a post like this one, that details the X number of essential SEO plugins that you totally need right now.  You install as many of them as you can before they start to actively conflict with one another, and then you sit back, satisfied that you are no longer making as many mistakes.  Armed with your broken link checker plugin, your SPAM bot blocker plugin, your query string remover plugin, and many more, you are ready to bask in the glow of superior SEO.

At least, you’re basking until one of your readers tells you that the time to load pages on your site has really spiked.  Not to worry, though, because surely there’s a WordPress plugin or 12 that can fix that too.  The only problem is, in googling around, you discover that too many plugins can start to conflict and generally cause performance degradation, so there’s a catch-22 to throwing more plugins at the problem.  Oh, and to add insult to injury, slow performance speed caused by all of your SEO plugins is hurting your SEO.

For Pete’s sake.

Web Server Access Log: A Low-Overhead Alternative

WordPress and other plugin-oriented CMS offer a wonderful value proposition in that website-creation novices can have both nice sites and lots of functionality.  But all of that value does not come without cost, and the cost is a tendency to bake in a lot of complexity.  Does anyone remember those applications like Front Page that would generate HTML for novices?  Remember what that generated HTML used to look like?  Yeah, I shudder too when I think about it.

So let’s stop and consider some of these plugins and their natures for a moment.  And then let’s think about the humble web server access log file that contains information such as what IP address made a request, when the request was made, what HTTP verb was used, what URL was requested, the response code, and the size of what was returned, among other things.  This is a simple text file that the web server spits out with no additional prompting, and it has a goldmine of information.

Against this backdrop, is a “broken link checker” plugin really necessary?  Can’t you just look for response codes other than 200 and investigate?  How about a SPAM bot checker?  Can’t you just look for suspicious crawling from an unknown, non-search-engine source?  And that query string remover?  Can’t you just search the log file for content being served that contains query strings and investigate that?

During the course of my reluctant education about SEO, I’ve learned that the main, overarching heuristic is that the search engines are trying to hook users up with content that they value.  Things that make your site worse chip away at that value — things like poor performance, broken links, server errors, etc.  And you don’t necessarily need plugins or even a CMS to discover problems along these lines.  You can start quite simply, with an analysis of your access log file.