Stories about Software


Logs for SEO

Editorial Note: I originally wrote this post for the LogEntries blog.  You can check out the original here, at their site.  While you’re there, check out the features of their offering.

I write blog posts on a number of different sites that are not my own, and that is an exercise in pure writing joy.  I compose the posts, I submit them, and viola!  They’re published on nice-looking sites, promoted by people with reach, and read by many (hopefully) interested readers.  Life is good.

By way of comparison, when it comes to my own blog, life is not quite so simple.  On my own blog, I have to write the posts and manage all of the details that are abstracted away when I write for other sites.  So many things distract from content.


There are the major ancillary concerns like the site’s look and feel and following up on any downtime or outages.  There are minor ancillary concerns, like checking for typos, promoting the posts, and making sure no one is inappropriate in the comments.  And then there are enigmatic ancillary concerns, like search engine optimization (SEO).  My primary concern is content generation, however, so, even with my own site, I seek to abstract as much of this away as possible.

The Weird World of SEO

Let me start off by saying, emphatically, that I am not an SEO expert.  Frankly, it’s not a topic that particularly interests me in and of itself… at least not until I had to be interested in it.

I have a blog as part of my website, but from that same website, I offer information about my consulting practice and about books and products that I offer.  More readers translates into more engagement, which, in turn, translates into a better living for me.  And it was against this backdrop that I became interested in SEO by default.

SEO actually reminds me of the credit score concept, in which a mysterious agency uses a mysterious algorithm to compute a score that has a serious effect on your life.  The mystery and complexity of the algorithm and proprietary nature of the score, in turn, create a cottage industry of advice and services aimed at helping you get just a little bit better.

As a blogger and entrepreneur, this is the SEO world for me.  Google (and nominally other search providers) have secret sauce algorithms that figure out how to rank content based on its likelihood of being valuable to people using the search engine.  I don’t care much about these for their own sake, but I do wind up having to learn enough about how the whole thing works in order to make (what I hope are) informed decisions on how to position myself.  Oh, and I have to do that while not wasting a whole lot of time on it and veering into the land of diminishing returns.

Read More


The Fragile Genius of Webforms

Turning Desktop Developers into Web Developers

I’ve been doing a lot of Webforms (ASP.NET) development over the last four months, as I’ve mentioned in some posts. Before that, I was doing a lot of WPF development for desktop applications and before that a hodgepodge of things that included web development with the Java stack. I had done very little with Webforms (though I had dabbled with a couple of internal corporate applications), so wrapping my head around it has been interesting.

For those of you not familiar with the way Webforms works, it could probably best be summarized as “helping you write web applications without realizing that’s what you’re doing.” Okay, that might be a bit snarky, so let’s go with “making windows desktop developers comfortable with writing web applications.” (in particular Winforms developers). It goes out of its way to create an abstraction layer that represents web concepts as their desktop equivalents which, I believe was done with the intention of easing the transition of a generation of Winforms developers to web application development. And, well done. I think it was wildly successful at that.

Indeed, it’s quite possible to write code for Webforms apps without understanding anything about HTTP, statelessness and the client-server nature of the web and understanding next to nothing about Javascript, HTML, and CSS. Instead of these concepts, Webforms developers are deal with concepts like “button click event” and “user control” ala Windows desktop development. It’s really quite a concept — “keep doing what you’re used to doing, and we’ll handle transforming the result from desktop apps to web apps.” It took me a while to stop wondering when I’d stop writing everything with wizards and datasources and gridviews and start outputting some templated markup with binding logic. It happened in trickles and drizzles but never really at all beyond that.

If your web experience is in any other technology, you’re certainly used to a less monolithic approach. Rather than “controls” that generate reams and gobs of HTML/CSS/Javascript behind the scenes, you’re no doubt used to the template approach where what you’re looking at is more or less HTML but with some notation for binding to a model and handling core concepts like iteration, validation or missing data. Perhaps there is also some notion of inline script execution. This is definitely how life works with technologies like JSP, PHP and even ASP.NET MVC. Developers using those technologies understand and embrace web as a delivery mechanism and architectural component and divide work up accordingly.

But not with Webforms. Webforms evens the playing field using wizards, WYSIWYG and extremely involved abstractions to make everyone working on a Webforms project a Webforms developer. Webforms actually abstracts understanding of underlying concepts away from the developers in this fashion, the way that Front Page and other such code generation tools have in the past. Karl Seguin gives an excellent Webforms vs ASP MVC summary that touches on these points and others.

A Brief Foray Into Architectural Philosophy

Persistence technologies, messaging/communication with externalities, web vs desktop are all non-core implementation details. Oh, don’t get me wrong, they’re important. But ebay doesn’t have an Oracle DB application or even a web application — it has a consumer resale application. The core of their business is enabling resale and that transcends the current technology stack. If Oracle went out of business tomorrow and a unified web became passe, ebay would no doubt try to soldier on by switching to MySQL and peer to peer app communication (or whatever). If people stopped wanting to resell goods, on the other hand, ebay would be kaput.

To staff a project that scales beyond tiny (i.e. multiple developers and up) a sensible way to divide up the work is to have some core business rules programmers and then others specializing in various satellite aspects of the application. So ebay might have people that develop server side application/business logic, people that specialize in Oracle stored procedures and people that specialize in web as a delivery mechanism and it would be expected that those people be domain experts, database experts and web experts, respectively. (Not to say that there couldn’t be some fluidity among these groups according to skill-set or availability and learning curve).

This separation of skills required in a team environment allows for the pairing of team members with tasks for which they are best suited. These pairings, in turn will naturally tend to foster productive collaboration, division of labor and generally good morale. People, especially knowledge workers like software developers, prefer autonomy and that tends to exist in greater abundance when each team member has a unique skill set that he or she is applying. A project with an architecture like the one I’m describing is more likely to be staffed by happy and productive developers.

Webforms and Generalist Mediocrity

Webforms does not lend itself well to the sort of architecture that I’ve just described. It invites developers to use controls that completely automate and obfuscate the generation of client side markup and scripts. But beyond that, it invites developers to describe the persistence layer in the specialized markup as well (the various “data sources” and their wizards), in effect flattening the application and rendering it monolithic. As such, Webforms developers can be (can be — not must be, please don’t misunderstand my point) relatively low-knowledge generalists whose only specialty is, well, Webforms. Webforms itself is the one with knowledge of different application layers and architectural patterns.

Image by “Scottius11” via wikimedia commons.

Whoah. Let’s think about that for a minute because it’s actually fairly profound. In a Webforms project, it is Webforms itself (or, more accurately, the people who designed ASP.NET) that understands the technologies involved. Webforms is your most knowledgeable ‘stakeholder’. Most successful frameworks and architectural technologies offer a cafeteria plan of sorts to let developers pick and choose what they need and want and to allow people to be paired up with particular facets of an application (a lot of IoC containers and various toolkits have this property). They empower developers. Webforms instead offers them a crutch — a way to perpetually develop applications without really understanding how they work.

Imagine that you owned dogs all your life but decide you want a cat. Now imagine a team of engineers coming along to ease the transition. They build a wooden “Trojan Dog” with a litter box in it so that you can “walk the cat” to get it to go to the bathroom. They build a sound altering muzzle so that when the cat meows, it sounds like a bark. They give you toys and pants loaded with catnip so the cat will ‘retrieve’ the toy and bring it back to you in game of ‘fetch’. In short, they create all manner of pretty elaborate and impressive adapter tricks to allow you to own cats and pretend that they’re dogs. They probably also do a bit of hand-waving when asked why this scheme makes any sense.

The Genius And Fatal Flaw

Like the hypothetical “dog to cat transition technology”, the Webforms abstraction of all things web is impressive. It’s impressive for a group of developers to be able to create a technology that allows tens of thousands of people to write applications without really understanding the underlying technology stack (again, not in all cases — just saying it’s possible). It’s impressive for those developers to create something that allows other developers to generate a radically different type of application without having to learn much in the way of new skills. I can’t overstate that — it’s amazing.

But the same thing that makes it amazingly effective at bringing old school Winforms developers into the 21st century also limits their growth and that of the applications that they produce. Webforms is a “helicopter parent” that doesn’t, on its own, know how to stop telling its 17 year old child when it’s time to brush his teeth and wash his face, and thus it runs the risk of perpetuating learned helplessness in the same. The internet is no longer new and alien, and our web development toolbox should reflect that.

ASP.NET MVC is slick, powerful and impressive as a framework. Developers should be comfortable with Javascript, HTML and CSS if they work on the client side. Developers should be familiar with IoC, communications protocols, decoupling and testing if they work on the server side. They should be using frameworks to enhance their knowledge and speed their work and not to handle everything for them so they don’t have to understand. It’s time to appreciate the genius of Webforms, acknowledge its importance, and wish it well in retirement.


A Tale of Two Web Stacks: Java vs .NET

For the last few years, I’ve focused largely on desktop development doing WPF and C#. I’ve dabbled a little here and there in web development, but the lion’s share of my web development up until the last few months occurred several years ago or earlier. Recently, I’ve been doing nothing but web development, in the form of webforms primarily, but also with Java and my home automation projects here at home. One weekend several weeks ago (it was “last weekend” when I started this post) I decided to upgrade my main machine at home from XP to Windows 7, and this required me to wipe everything and start fresh. Part of this meant that I’d have to port my IntelliJ/Spring/Maven/Java setup to a new machine.

I had ported my project from Eclipse to IntelliJ (which went very smoothly — compliments to IntelliJ), so it had been a long time since I’d actually set up a web development project in Java. Interestingly, it had also been a long time since I’d done the same in the ASP world since the work I’ve been doing the last several months had already been setup from a project structure perspective. However, given my situation with the home automation project and the fact that I’m starting on FeedPaper, I’m in a unique position to document my comparative experiences with both, being in the position of generally experienced developer and relatively familiar with the technologies, but not practiced at setting up these specific types of projects. I’ve done this documentation below.

Before you read on, please note that I’m not in the tank for anyone or a fanboy of any technology, company or platform. I’ve spent years developing in both Java and .NET and there are things that I like about both. I’m a happy, equal opportunist polyglot and hope to stay that way. But for me to do so (with Java and .NET at least) would require both technologies to succeed, and I see trouble on the horizon for Java. I don’t like this because I like Java. It was a nice alternative for web development when Microsoft wanted to charge me $500 for Visual Studio and who knows what for whatever else I would have needed to write web applications. I like it because it was real, big boy server side code, capable of expansion to enterprise sites and not sloppy (I’m looking at you PHP). I like it because of the vibrant and inventive community of developers committed to improving it. But, I still think dragons be coming and Java might have a fight on its hands not to become COBOL.

Setting up For Java Web Development in 158 Easy Steps

Here’s why I say there’s trouble. I wanted to set up a minimally functional site with both Java and .NET web technologies — “hello world” in concept. My steps to set up Hello World with Java are detailed here (if you get tired of reading, feel free to skip to the bottom of the list as it is incredibly long):

  1. I download and install IntelliJ Community Edition.
  2. I download and install JDK 7 and create a “JAVA_HOME” environment variable.
  3. I download Tomcat and unzip it in C:\program files\Tomcat.
  4. I set “CATALINA_HOME” environment variable and point it here.
  5. I go to Tomcat’s “webapps” directory and set its permissions to allow anyone to modify since that’s where I’ll be deploying my projects and I don’t want UAC messages each time I do.
  6. I run Tomcat’s “startup.bat” and fire up localhost:8080 to manage tomcat. Fail. HTTP Status 500 – java.lang.ClassNotFoundException: org.apache.jsp.index_jsp.
  7. I refresh and get a different error: HTTP Status 404 – /.
  8. I try going to the manager URL directly. Error: HTTP Status 500 – java.lang.IllegalStateException: No output folder
  9. I google, but don’t really know what to google, and don’t have much luck.
  10. I try running random bat files in the Tomcat directory with names like “setclasspath.bat”. No joy.
  11. I put quotes around my environment variables since Windows directory names and their spaces are kinda screwy.
  12. Oops, now startup.bat in the Tomcat folder does nothing. So much for that. I put the environment variables back.
  13. I open the file “RUNNING.txt” in Tomcat root directory and see if it has anything helpful to say. It does, but it only covers the stuff I already know (all this environment variable crap and that I should start the web server with startup.sh)
  14. I google again and find nothing useful.
  15. I start looking through the Tomcat output to the Console and see a bunch of exceptions about a log file not existing because access is denied. This doesn’t seem like it ought to be critical, but you never know.
  16. In a scorched Earth approach, I make the entire Tomcat directory writeable by any user and restart Tomcat where I see no more log errors.
  17. Joy! Half an hour in, I have the web server running.
  18. Time to make sure I can see what apps there are — I remember I can do that from the manager.
  19. Tomcat 7 helpfully tells me that I need to configure security for the web server (previous versions didn’t) so I do that by editing tomcat-users.xml.
  20. I uncomment the examples in the file to use and that doesn’t work, I get a 401. But, the 401 is pretty helpful (another improvement) and I follow the directions in it, which unfortunately don’t work.
  21. I try restarting the web server.
  22. Joy! That was my stupid mistake — assuming that the roles were processed at login time rather than startup. 45 minutes in I can manage the web server.
  23. Going back to IntelliJ, I try to create the project I need, but can’t because I don’t have a web template available.
  24. I google and discover I should use Maven.
  25. I read and try to understand what Maven was and where it came from. Don’t really understand (kinda like NuGet I think), but apparently IntelliJ has it so I don’t need to install it or anything.
  26. I try to create a Maven project, which goes fine except for the “Finish” screen, which tells me that the Maven Home Directory is not specified.
  27. I specify it as the directory with the user settings file, but apparently that isn’t right because the Maven Home Directory is “invalid” (no mention of why or where I might find one that isn’t).
  28. I google this error message, wondering why I have to bother with this and why it doesn’t just work. Epic fail. I’m apparently the only one that’s ever had this problem.
  29. I do a search on my hard drive for “maven” and find that there are a few hits in the folder C:\users\erik\.IdeaIC11, so I figure, what they hey, I’ll give that a try. Nope.
  30. I look at the “user settings file” and “local repositories” specified by default and notice they don’t exist, so I create them, figuring that this might make the directory a valid maven directory. It doesn’t.
  31. I google and find this stack overflow post that leads me to an example settings.xml file and I try adding the xml I find there into mine, hoping this might at least result in a new error message. It doesn’t.
  32. I go downstairs and get some soda and take a break.
  33. I come back, wondering whether I have to install maven separately.
  34. Apparently not, according to Jetbrains as it “ships with” IntelliJ. I testily wonder “if it ships with IntelliJ, why the %&$# is it asking me where the Maven directory is?!?”
  35. Back to google. I find this post telling me that “opening maven project is as easy as pie”. I chuckle.
  36. Back to google. Nothing helpful after another 5 or 10 minutes. At this point, I’m 1:15 into this effort.
  37. I decide that I’ll install Maven anyway, whether or not it “ships with” IntelliJ. What can it hurt at this point?
  38. I go to the maven download page and am a bit concerned that IntelliJ says that it integrates with Maven 2 and that a third of the roughly 17,400 options I have for downloading Maven are for a version 3. And which of the version 2’s on there does IntelliJ integrate with. I abandon this plan.
  39. I google again for a while and discover that most problems with the M2_Home/Maven Home seem to be for Linux and Mac users. Apparently there is some sort of known issue there. I wonder if that’s true on Windows at all.
  40. I go get another soda. I’m about 1:45 minutes in now.
  41. I download Apache Maven v2.2.1, reasoning that even messing things up badly at this point would be better than no change (and it takes less time to format my drive, re-install Windows 7 and get all my drivers going than setting up this hello world web project anyway, so how bad can it be?)
  42. I unzip Maven to C:\program files\maven.
  43. I look in the conf folder and see “settings.xml” so, having learned my lesson from Tomcat, I decide to move the whole operation to C:\users\erik\.m2
  44. I try again and it fails using M2_HOME, but when I “override” and type the directory in manually, it works. I guess “defaults to M2_HOME” is a bit of a fib, but that doesn’t matter at this point, since I’ve won.
  45. Joy! About 1:50 in, I have a web server installed and I’ve created my project!
  46. My joy is short-lived as I see that there is no Web-Inf or welcome JSP page or index or anything to indicate that this is a web app. Sigh.
  47. Whatever, let’s at least get what we have building.
  48. I do a “make project” and get an error: “Cannot find home directory C:\program files\java\jdk1.7.0. Update Project Configuration.” I’m actually pretty pumped about this error message since it offers some kind of actionable feedback. My standards, as you can tell, are now pretty low. I mean, you’d think this could be inferred from my “JAVA_HOME”, but whatever.
  49. I go into settings and look under “Compiler” and see nothing about the JDK to user, so back to google.
  50. I find nothing helpful there, so I start randomly looking at menus and context options.
  51. I right click on my module and see “open module settings”, which looks promising. I go to the “Project” under “Project Settings” and see that there is a SDK specified, but it just says “1.7” and it’s red. I click the dropdown and see there’s also 1.7 (1) and that’s red too. Apparently the default is not 1, but 2 copies of a nonexistent JDK. I delete both of those and manually browse to the actual JDK.
  52. Now, I build and nothing happens, which is an improvement. I’m just told “all files are up to date.”
  53. I’m not going to worry too much about that now because I don’t actually seem to have any files, so it stands to reason that there’s nothing to compile.
  54. I’m not sure what the Maven “web-app” goal or archtype or whatever actually did for me, but I did notice something interesting when clicking around called “Add Framework Support”, so I’ll try that.
  55. Bummer, my only option is “Groovy”. Back to the drawing board.
  56. At this point, 2:15 in, I knock off for the day. I’m doing this on nights and weekends and so clearly hello world web app is going to need to be a longer term project than just one day. I have chores and bills and occasionally a life, I’ll have to resume setting up the simplest, most basic web setup imaginable later when I can really devote a lot of time to it. I’ve managed to install the web server and create a project that doesn’t do anything, compile, or even have any files. Quite a productive day, I guess…
  57. I pick back up the next day and, having slept on it, I remember something about adding things to pom.xml. Perhaps if I add Spring stuff to it, the “Add Framework Support” thing will work.
  58. I google and find this stackoverflow post. It’s not particularly helpful, and I consider “add this bunch of random crap to this XML file” to be an enormous framework fail, but misery loves company, and I can see that most people that do this are also confused and that you can “learn the basics of Maven in a few days” if you read a book. I’m not really clear on why this Maven is better than downloading Jars on my own, which wouldn’t take me a few days, but I’m trying to do things the “right way”.
  59. I google some more and find this from Spring, and it looks promising. I still think it’s utterly preposterous that the way to get dependencies is by hunting down blobs of random XML from the internet to copy and paste into a file, but I seem to be making progress. I copy and paste.
  60. I then right click and click “sync Pom.xml” because I seem to remember doing that before and it seems to make sense for some reason.
  61. I try “Add Framework Support” again, but no joy. Still only groovy.
  62. I delete all that XML and flail around google some more.
  63. I find another post where I see some sample XML from another pom, and I see that all that crap I copy and pasted needs to go inside of a “dependencies” tag.
  64. The syntax highlighting looks more promising now, but I try synchronizing and importing and whatnot, and still no framework support. Now there’s a lot of angry red in the IDE about my pom.xml file and a squiggly under it in the project explorer. But, errors are different, and different is progress.
  65. Back to google, and I’m now 2:45 in.
  66. I find another blog post and learn that the “properties” tag needs to go outside of the “dependencies” one. Of course – I shoulda known (/sarcasm). This guy’s blog is pretty helpful — too bad he doesn’t use IntelliJ.
  67. I make the change and synchronize again, and nothing really happens.
  68. After a minute, the squigglies and angry red goes away and suddenly a bunch of stuff about spring appears under “External Libraries”. It’s like magic (but more like a kid putting on a magic show and needing a few mulligans than David Blaine).
  69. This doesn’t help setting up the directory structure to get hello world up and running.  I still have a blank module under a blank project.
  70. I google and find this on stack overflow.  It doesn’t help me, but it is interesting to note that someone experienced in all three technologies (Spring, Maven, IntelliJ) would likely take longer than 30 minutes to set this up.  Someone not experienced with Maven and IntelliJ… 3:00 and counting.
  71. I try creating a new module for the heck of it, to see if maybe this time the “Maven Web App” actually has some kind of directory structure or code in it.
  72. Nope.
  73. I go to delete this module that I don’t want and spend a few minutes getting annoyed until I discover that I have to delete all of the files in the module, then delete the directory, then do it again for reals (it’s separating delete from project and disk).  That took a pointless 5 minutes.
  74. Back to google.
  75. I find this page, which has exactly what I want.  It’d be awesome if it my IntelliJ actually worked like that.
  76. I see that that post was written 9 months ago, so perhaps setups were simpler back then.  Yep, those were the days.
  77. I try creating a new project anyway because I’m really pretty much out of ideas.  Not surprisingly, that doesn’t work.
  78. Back to google.  Nothing.
  79. I start to wonder if it’s even possible to have IntelliJ create a web project directory structure for you.  I could have sworn it worked before when I did this some weeks back.
  80. I find this on the IntelliJ site, which suggests creating a Java module from scratch instead of a web module.  I try that and it doesn’t work, but I try creating a Maven module again and now have the option of creating a Spring project.  I do that, and it actually gives me a “src” directory.  Sweet!  We’re getting somewhere.
  81. This module inside the project also has a pom.xml.  I copy all of the crap from the main pom to this one and try adding framework support.  Nope.
  82. I am seeing “fatal error, cannot find JRE1.7” in the “messages maven goal” window.  I clear the message and I can’t get it to come back.  I have no idea what’s causing it.
  83. At this point, I think it isn’t worth spending anymore effort trying to get the IDE to setup a basic project structure.  Clearly I’m asking the impossible of the incapable.
  84. Luckily, I have the source code for a previous Eclipse project that I converted to IntelliJ laying around, so I re-create that directory structure (I eventually want to be using a lot of the source from this project anyway, so I’ve kind of decided that skipping hello world might, ironically, be easier).
  85. I copy the entire WebContent directory and then copied the structure of src, at which point I copied a few controller java files.
  86. The files I copied have a little j with a red circle around them and a line through them.  That don’t look right, so I google and find this.  I have to specify a “source” directory.  Sigh.  Of course I do.
  87. I follow the instructions in the post and my src directory automatically converts into a package structure (which is admittedly pretty cool) and three of my controllers are error free.
  88. The problem was that they were referencing other files that I need to add, so I start adding them, but that’s where things get wacky.
  89. The package structure seems to work much differently than Eclipse and my package names are getting out of sync with the disk structure and each other, so I delete everything and started over.
  90. I try to google an example package structure for IntelliJ IDEA to mimic, with predictably useless results.
  91. For those keeping score, I’m now about 3:50 in.
  92. I find this on IntelliJ and it’s like a blast of fresh something.  It actually clearly explains something in this byzantine process for the first time in a long time, and I came to understand that IntelliJ’s way of handling packages is actually pretty slick — it lets you retain package names like “com.daedtech.daedalus.controller” without the silly requirement of having 4 nested folder on disk.
  93. I add my second most abstract set of classes to the new structure and do a build, which fails as I wanted it to (red before green, refactor).
  94. I add the most abstract types that these depend on those and then rework the package naming as necessary until compiling is successful (oddly, for one of the broken dependencies, alt-enter works and for the other it doesn’t).
  95. I’m calling it another night (I didn’t start working on this until a little after midnight, so it’s pretty late now).  I figure I’ll go to bed celebrating a win.  4:15 and 95 steps into setting up Java web development, I have compiling code.  Hopefully in the next few hours I can build a WAR file and actually shoot hello world to the screen, to say nothing of setting up the IOC.
  96. Picking back up the next day, I copy all of my code files over and start resolving the differences in package naming.
  97. I spend about 20 minutes resolving all of the various compiler errors and naming issues that resulted from copy.
  98. Now I copy over the ant Build that I had been using and try it out.  Fail.
  99. Start googling what on Earth “‘includeantruntime’ was not set” means.  This helped.  Apparently, I need to set some random attribute in the javac tag that I didn’t need to before for some mysterious reason I might have cared enough to investigate in the first few hours of this.
  100. New set of errors when I try to build.  Cool, different is good.
  101. The errors say that the spring framework package does not exist, which is weird considering the module compiles in the IDE.
  102. Looking at the source files, IntelliJ randomly borked my packages in the files, so I put them back to what they were.  I’m pushing the 4:45 mark and battling the IDE.
  103.  Now onto figuring out why the ant build says springframework packages don’t exist but the IDE compiler says they do.  Apparently, this is some sort of ‘feature’ where plugins compile code differently than the IDE or something.  Boo.
  104.  I do some fruitless googling but don’t really know what to search for.  “Why does Ant suck? ” amuses me but provides no answers.
  105. I see a squiggly under one of my java files and realize that I missed one of the borked package renames.  I have no idea why it would compile with an unresolved reference.  Maybe the Ant compiler is the good one and this mystery, non-functional one the IDE uses is the problem.  My apologies to Ant.
  106. This doesn’t fix the problem of the Ant build not recognizing the spring packages.  Back to flailing around google.
  107. After about 10 minutes, I stumble across this stackoverflow post.  This reminds me that usually there’s some goofiness about “classpath” in both Eclipse and IntelliJ, in my experience, so I poke around the project and module settings looking for something like that.
  108. Poking around, I see something in the Ant properties about “Additional Classpaths”.  I find this.  My suspicion is confirmed.  It’s apparently some kind of insurmountable technical challenge to have the IDE and the build plugin use the same library configuration for building and the task falls to the user.  What a mountain of fail.
  109.  Now I find where the libraries are actually located (that .m2 directory from about 70 steps ago) and add those as an “additional classpath”.
  110. Nope.  Same error.  Back to google to celebrate hour number 5 of hello world setup.
  111. I have no luck for a while and then decide to re-read that last link and see that after line 3 it says “if you want to add the contents of a whole directory, you can click the ‘Add All in Directory’ button”.  Silly me, I just would have assumed the classpath would already have meant this or else it would have been the classfile.  I delete my useless “directory with no children” entry and opt instead for the option whose existence makes sense.
  112. I try again and all errors but one disappear.
  113.  That seems like progress until I see that the single error says “Ant build completed with 86 errors and one warning” and provides me with a single stack trace and no information beyond “compile failed”.  Sweet.
  114. I run the IDE compiler to see if that works.  It does, but I don’t know how much that tells me since it seems to ‘work’ even when there are red squigglies.
  115. I spot inspect the files, but it doesn’t seem to have changed any of them this time, so I suppose the IDE compile is really working.
  116. I re-run Ant and the missing springframework exceptions re-appear.  Two steps forward, two steps back.
  117. I add one of the child directories containing an actual JAR, in case Ant/IntelliJ have yet to discover directory recursion, but that doesn’t seem to help.
  118. At this point, I decide to concede defeat for the night because it’s late and I’m tired.  The good news is that I’m probably about half done with my project — I imagine that actually writing all of the code for the application can’t be anywhere near as difficult as setting up the development environment.  Once you’ve configured Java, writing applications in it is probably a walk in the park. I’m reminded of Rational Clear Case in that perhaps it makes sense to have experts that specialize in falling on this setup grenade so that developers are freed up to actually develop code instead of doing what I’m doing. (As an aside, if I were doing this work for a client, it would have made economic sense to pay someone up to $1000 to set all of this up for me.)
  119. I’ve been somewhat busy, so two weeks have actually passed between steps 118 and now, but in terms of raw time I’m at 5 hours, 30 minutes and counting to get Hello World (or anything at all) going.
  120. First step coming back, I googled the error message Ant was reporting and found a stack overflow post asking for help.  No answers.  Gave a sympathy upvote and moved on.
  121. I expand the “compile” node in the output, even though it has no error indication and discover that it’s littered with errors.  I think I’m partially an idiot for not checking that sooner but think that the tool needs to meet me halfway by not collapsing dozens of errors and having their section header look like everything is fine.  Still, now I’m in business.
  122. All of the errors seem to be about packages and stereotypes not existing, so I suspect it’s something to do with Maven (which, for the life of me I can’t figure out why this isn’t worse than nothing).
  123. More googling and I find this post, which tells me that I maybe need to do something with the class path as I was flirting with back in step 111.  I’m not going to follow accepted answer though because it wants a lot of things and there has to be a less stupid way to do this than redundantly adding all of these definitions to the actual Ant file.
  124. I look at the settings and see there’s a bunch of checkboxes next to the Dependencies in Module->Daedalus-> Dependencies and they aren’t checked.  I check them.  It doesn’t make a difference, so I put them back.
  125. I go into the ant build properties and exhaustively add the actual jars, but that doesn’t matter.  Still errors.  I delete everything in there.
  126. I go under the Ant properties execution and tell it to use a different JDK.  Doesn’t matter, same result.
  127. Back to google, but I don’t really find anything after 5 or 10 minutes.
  128. I start poking around settings randomly.  I try adding the Maven repository root directory to my classpath.  Nope.  I celebrate the start of hours 6 of hello world with that unsurprising failure.
  129. I try adding the MongoDB driver explicitly to the classpath.  Doesn’t help.
  130. Deciding I have nothing to lose, I go scorched Earth and choose Build->Generate Ant Build.  After doing that, it adds a few new XML files to the project.  This doesn’t help my existing build run any better, so I remove it as an Ant build from the Ant window.
  131. I add the one that IntelliJ created and theirs only has two errors, which is some sort of rather sad progress, I suppose.  I also take solace in the fact that even IntelliJ doesn’t know how to generate an Ant build and they’re experts.
  132. Time for baby steps.  I run “clean” and that works.  Huzzah!  Init also runs without errors.  I run “build.modules”.  Doh!
  133. I get an error about IntelliJ not being able to find its own ant definition file.  Apparently I’m not alone.  In the world of setting up Java, just finding someone else that has the same error as me is a huge win – an actual proposed solution is like Shangri-La.  In this case, the problem is apparently that ${idea.home} is not defined.  Oh, of course — as the question asker points out after hearing the answer, he should have known to check a checkbox that was unchecked by default, as I should have too!  /sarcasm (As a brief aside, I think that this sort of thing epitomizes an Emperor’s New Clothes paradigm in programming where we’re afraid to point out that there might be benefits in simplification because we don’t want to appear as though we aren’t clever or knowledgeable.)
  134. Not knowing how to make this ‘obvious’ correction, I delete the whole build and regenerate it.  Success!  Woohoo!!!  There were no other defaults I needed to magically know to override.  Happy Day!  Finally, after 6 hours and 20 minutes, I have an application compiling both in the IDE and with the strangely separate build engine!
  135. Now time to put my build.xml targets into this new ant build and see if I can make the magic happen.  There are actually two ant build files, for some reason, and the targets seem to be spread out sort of randomly across them.
  136. I start adding all of my stuff from my old build into the new build and run it, and then I get all of the same errors.  Time to take baby steps again.  First, I go back to the known build that was running.
  137. First I define all of my custom properties, and that seems to work, so I add a path definition from that file, and that seems to work too.
  138. I then add the war target, and run it, and that works.  Woohoo!!!  I’m building a WAR file.  Almost there.
  139. Now, it’s building it in the wrong directory for some reason.  I hard code the directory name and give that a try.  That works.  I’ll figure out why my variable scheme didn’t port over later.  I want to keep going while I’m on a roll.
  140. I copy over the “start tomcat” and “run chrome” targets to see if I can run my web app.  Doh!  Doesn’t like the directory variables here either.  So, more hardcoding.
  141. That works and tomcat starts, but it can’t find chrome… forgot to hardcode that as well, so I do and then everything runs, and I get a 404!!  (I’m not being sarcastic — I’m delighted that everything at least compiles and deploys).
  142. When I look at the tomcat webapps directory, I see that the war is being generated and it’s being unpacked, but that it has actual Java files and no classpath/lib.  So, there’s something wrong with the actual packing of the WAR.  As I investigate, I see that the lib directory is empty and that so are some other key directories.  I realize that this is because the old Eclipse structure for ouptut was different than what I have now, so I reconfigure the war target, setting the classes directory appropriately.
  143. That works, and when I re-run, I see my default JSP page in the browser, which is definitely a win.  None of the controller links work, but I’m getting tantalizingly close.
  144. I see that the main problem I have now is my lib directory not being populated with my JAR dependencies.    I googled around for a while and found this page and decided to add an “artifact” with my dependencies for deployment rather than manully dropping a bunch of jars into my “lib” folder for deployment, which seems redundant and brittle.
  145. After doing this, the settings window pops back open once for each dependency I had, which seemed screwy.  I’m not optimistic about this working and I’m prescient because it doesn’t.
  146. After playing around in the settings some more, I see that I can move my libraries, so I figure I’ll move them to the lib folder.  I try that and the window starts showing me errors, so I cancel.  Unfortunately, the cancel seems only to have sort of canceled.  I now have an error left that I have to troubleshoot.  Back to google.
  147. I google around and can’t find anything, so I try  just deleting the offending library.  That seems to work and everything still builds.  Ah, the mysteries of life…
  148. After wasting another 15 minutes or so, I’m getting less picky about redundancy.  I decide to copy the jars manually to Web-Inf/lib, but apparently the IDE can only handle this one at a time, so I have to laboriously do it for each one with a right click rather than doing them all together.  /Sigh
  149. Upon further review, I don’t think that did what I thought it would.  It just created a blank module name in that same screen.  Nothing should ever be this convoluted and counter-intuitive.  I want to copy a file.  Only the java world could make this a task that requires a PhD.  Back to google.
  150. After a while, I just decide to give up and manually copy all of the dependencies in that stupid directory.  I was dissuaded all along from doing this because (1) it’s astronomically stupid that you should have to, (2) it’s redundant information, (3) it’s a huge PITA since all of the jars are in their own folders several levels deep and (4) did I mention how incredibly stupid it is that you should have to do it when the IDE already knows about both folders?
  151. I do find and try following the process here but the options in IntelliJ are different now, so this handy-dandy 10 step process for copying files is no longer accurate.  No doubt they’ve improved it to take 23 steps.
  152. Halfway through doing the manual copy, I just can’t bring myself to continue doing something this stupid, so I stop, celebrating the 7th hour of Java hello world with a soda break.
  153. Coming back and doing a lot of googling, I find  this site.  This is a three step process for copying the Jars where step 2 is “Copy the Jars”.  Of course!  /Facepalm.  But I do pick up the interesting tidbit here that this is not the proper, Maven way to do things.  That is, perhaps I shouldn’t have a Web-Inf/lib folder after all.  I tried following the link from that site to the Maven FAQ, but it might as well have been Greek for all the sense it made to me.  I am not “confident” with Maven, so apparently I can ‘conveniently’ use the command line to do God-knows-what.  If that massive parameter-list command line is the convenient option, I think the inconvenient option will probably break my spirit.
  154. I do some more googling for things like “IntelliJ Maven Web-Inf Lib” and lots of IntelliJ bug reports came up.  From using it, I like IntelliJ and the Jetbrains people are sharp.  The only possible conclusion I can formulate the further I get into this is that Maven is a complete train wreck of an implementation however good it might be in theory.
  155. More googling and I find this site.  It doesn’t help me in any way, but the post’s title, “Set up a maven web project in Intellij with Spring and JPA – Part Six” seems somehow perfect to me.  Notice the “part six”.  How many blog posts are required for hello world?  Perhaps his 155 and counting steps are simply distributed into 6 posts of a more manageable 25 step each.  Anyway, a bit of levity for the 7:30 mark.
  156. I found this wiki, which seems to indicate that this would all be much less comically difficult in the Ultimate edition.  Somehow, I’m not intersted in giving money to anyone that has anything to do with this.
  157. After another 20 minutes of fruitless googling, I bite the bullet and do the manual copy.  It only actually takes about 5 minutes, but I just can’t wrap my head around the fact that the IDE knows exactly where these JAR files are on the disk and exactly where my lib folder is, and yet the only way to get those files into that folder is for me to open up two explorer windows and copy/paste all the jars.  The mind boggles.
  158. It works!!!  Hello World (or really, just anything) up and running in just 8 hours with 158 easy steps!!!  It’s like

Setting up for Microsoft Web Development in Many, Many Fewer and Actually Easy Steps

Here are  the steps I followed for Microsoft’s ASP MVC:

  1. Downloaded and installed Visual Studio.
  2. Went to File->New Project and chose ASP MVC project (“Internet Application” template).
  3. Hit F5.

Yep. That’s it.

Threats To Validity

Is this a fair assessment? I’d like to address a few points that I can imagine people raising. First, while it’s true that I’ve been doing a lot of .NET development over the last few years and not much Java development, the overwhelming majority of that work has been desktop development in WPF and a bit of Webforms, so that gives me no experience setting up an ASP MVC project, which means I actually have more experience setting up Java web projects than ASP MVC projects (though a lot has no doubt changed in the last few years).

Another consideration is that I’m using the Community version of IntelliJ instead of the paid version of IntelliJ or using Eclipse. These are valid points, though with Eclipse, it’s been my experience that things are no less of a headache to configure. I also feel as though this is not a significant mitigating factor because of the sheer number of tutorials I encountered for both Eclipse and IntelliJ that had half a dozen or more different posts in sequence for “getting started with web development” or some such thing. As for the paid version of IntelliJ or anything else, there would be a high degree of irony to needing to pay for something in the Java world that you get for free from Microsoft.

Finally, and perhaps most up for debate is whether or not I’m just an idiot (or at least too much of a “noob” to figure things out). Perhaps I just don’t have the programmer chops to do something that an expert could have done quickly. I fully acknowledge that this could be argued (and I’d love anyone to give me time saving tips for next time), but I’d say it misses the point that I’m trying to make here. And that point is that an extremely complicated setup discourages adoption and use. I’m an experienced, polyglot developer that has previous web development experience (even if I am also an idiot), including J2EE experience, and it takes me 8 hours over several weekends to get this going. Imagine how it might go for someone new to web development or just development in general. They’ll try to set things up, get frustrated, look for web tutorials and see things like “Getting started with Java/Spring/etc part 22”. Their next step will be to download Visual Studio and start an ASP project or to download Apache start writing PHP.


Part of the beauty of Java and its stack of FOSS technologies is that it’s an alternative to Microsoft’s “golden coffin”. You have tooling options for everything you might want to do. Choose your web server, your IDE, your IoC container, your web framework, your runtime, your development kit version, etc. And not just that, but you can mix and match versions. Nobody tells you the ‘right’ way to do things — you can pick all of the options that work for you.

However, it’s important to recognize that this incredible flexibility is great for someone already immersed in these technologies, but daunting and discouraging to someone who isn’t. To this person, it’s pure, confusing fragmentation. Someone who wants to write a hello world app is going to say “what do you mean Ant v1.2.3 doesn’t work with Maven v4.5.6 and JUnit 7.8.9 on windows — I don’t even really know what those things are, I just want hello world!”

Something I’ve noticed that results from this too is that developers come to view understanding how to navigate this setup minefield as a badge of honor. Somewhere in the Java setup steps, I referred to this as “Emperor’s New Clothes” kind of tolerance for complexity. Everyone immersed in the world is afraid to call something out as byzantine and convoluted for fear of being labeled a newbie or lacking chops. But having battle scars from spending weeks configuring a development technology time and time again isn’t a badge of honor — it’s indicative of a problem in tooling.

Now that I have Java set up and finally working, I’m actually excited. I love developing in it. I like the language, I like the choices that I have with it, I love IntelliJ as a tool, and I like that I can work on any OS I choose. But I don’t think we need to settle for this horrendous “hello world” experience to have customizability. I would love to see work done on something that preserves all of the flexibility and power but allows developers to go to a site, download something, and then have a working java web setup. Something that has roughly the same number of steps and complexity of the Microsoft process (or the PHP process, which would be pretty close) would be ideal and perhaps even essential. In a world where the competition offers setup ease that allows developers to be up, productive and tweaking in minutes, a confusing process that spans hours or even days is a sure path to irrelevance and obsolescence. Maybe this already exists (and please someone correct me if I’m just not aware of it — seriously, I’d love it), but it seems doubtful given the number of entire series of posts dedicated to basic setup. If it doesn’t exist, it ought to, and not to stop me from grousing but for the future of the technology stack.

By the way, if you liked this post and you're new here, check out this page as a good place to start for more content that you might enjoy.


Speeding Up DaedTech

As I get back into doing web development more and more these days, I’ve started to pay attention to some of the finer points, such as not having a sluggish site that turns off visitors. To that end, my Trello Board for this site had a card/story sitting in the “To Do” bucket for speeding up DaedTech’s load performance.

The first thing that I did was visit GTMetrix and run a baseline to see if subsequent things that I did would improve performance. From there, I installed W3 Total Cache which is a WordPress plugin that provides a whole bunch of functionality for speeding your site, mostly revolving around setting cache settings. After a bit of poking around and research, I figured out what was going on, and I enabled settings that helped with some low hanging fruit.

This included “minification” of CSS and javascript, a process whereby the whitespace is compressed out of these things as they’re sent from the server, thereby reducing the total amount of data sent (and thus time to process that data on the client side before displaying it). It also included optimizing the caching settings that the site suggests to returning visitors so that pages, styles, media, etc are stored locally on your machine as much as possible, which prevents reloads. This also setup the further use of GZip for further compression.

For improvement in the future, I installed a plugin called WP-Smush.it that will use the Yahoo utility for image compression to any file I add through the media library. This seems convenient enough that I should probably start adding files through the media library in general rather than simply putting them on the server and linking to them at their full local URL to get this functionality.

While I’m at making resolutions to improve speed going forward, here are some other tips that I’ve picked up today:

  1. Serve scaled content. Meaning don’t put up some huge image that the browser downloads only to use CSS or HTML in to tell the client to shrink it down. Send over the smallest possible image.
  2. Favor using my own images instead of embedding them from other sites. This lets me control the cache expiration suggested to the browser and the size as well. With hyperlinked images, I don’t have this control.
  3. Specify the image dimensions rather than simply accepting the default.
  4. Consider using image “spriting” to combine images such as the gaggle of social buttons into a single “image” to reduce the amount of stuff getting sent over the wire.
  5. Consider using a content delivery network to store your resources in places closer to site readers.
  6. Try to limit the number of things that make HTTP requests (like social media buttons)
  7. Use a utility to defer javascript execution so that it doesn’t block page load

I’m no web performance guru by any stretch. This is just what I pieced together in a morning after saying “I want my site to load faster”. I’m hoping that people in a similar situation to me will see this and realize that there is some pretty low hanging fruit to be picked here and that it isn’t terribly complicated to do so.


Links As Buttons With CSS

I’ve recently started working on my home automation web server in earnest, and am trying to give it a nice look and feel. This is made more interesting by the fact that I’m developing a site intende specifically to be consumed by desktops, laptops, tablets and phones alike. With these constraints, it becomes important to have big, juicy click targets for users since there’s nothing more irritating than trying to “click” tiny hyperlinks on a phone.

To do this, I decided that what I wanted was a button. But, I’m using Spring MVC and I want to avoid form submissions for navigation and handle that through hyperlinking. So, after some experimentation, I came up with the following via a composite of things on other sites and my own trial and error tweaking:

/* This is for big content nav buttons */
a.button {
     padding: 5px 10px;
     background: -webkit-gradient(linear, left top, left bottom, from(#CC6633), to(#CC3300));
     background: -moz-linear-gradient(top,  #CC6633,  #CC3300);
     -moz-border-radius: 16px;
     -webkit-border-radius: 16px;
     text-shadow: 1px 1px #666;
     color: #fff;
     height: 50px;
     width: 120px;
     text-decoration: none;

table a.button {
    display: block;  
    margin-right: 20px;  
    width: 140px;  
    font-size: 20px;  
    line-height: 44px;  
    text-align: center;  
    text-decoration: none;  
    color: #bbb;  

a.button:hover {
     background: #CC3300;
a.button:active {
     background: -webkit-gradient(linear, left top, left bottom, from(#CC3300), to(#CC6633));
     background: -moz-linear-gradient(top,  #CC3300,  #CC6633);

/* These button styles are for navigation buttons - override the color scheme for normal buttons */
a.headerbutton {
    padding: 5px 10px;
    background: -webkit-gradient(linear, left top, left bottom, from(#1FA0E0), to(#072B8A));
    background: -moz-linear-gradient(top,  #1FA0E0,  #072B8A);
    -moz-border-radius: 16px;
    -webkit-border-radius: 16px;
    text-shadow: 1px 1px #666;
    color: #fff;
    height: 50px;
    width: 120px;
    text-decoration: none;

Nothing here is really rocket science, but it’s kind of handy to see it all in one place. The main things that I wanted to note were the gradients and border radii to make it look pretty, the webkit/mozilla stuff to make it compatible with versions of browsers not yet supporting HTML 5, shadowing and fixed height/width to complete the button effect.

In terms of client code, I’m invoking this from jsp pages:

Say Hello Sample

And, that’s about it. Probably old hat for many, but I haven’t done any web development for a while, so this was handy for me, and it’s nice to have as reference. Hopefully, it helps someone else as well.

Edit: Forgot to include the screenshot when I posted this earlier. Here’s the end result (and those are normal a tags):