DaedTech

Stories about Software

By

Cutting Down on Code Telepathy

Let’s say that you have some public facing method as part of an API:

public void Process(CustomerOrder order)
{
    //Whatever
}

CustomerOrder is something that you don’t control but that you do have to use. Life is good, but then let’s say that a requirement comes in saying that orders can now be post-dated, so you need to modify your API somewhat, to something like this:

public void Process(CustomerOrder order, DateTime orderDate)
{
    if(orderDate < DateTime.Now)
        throw new ArgumentException("orderDate");

    //Whatever
}

Great, but that was really painful because you learn that publishing changes to your public API is a real hassle for both yourself and for your users. After a lot of elbow grease and grumbling at the breaking change, though, things are stable once again. At least until a stakeholder with a lot of clout comes along and demands that it be possible to process orders through that method while noting that the order is actually a gift. You kick and scream, but to no avail. It has to go out and it has to hurt, and you're powerless to stop it. Grumbling, you write the following code, trying at least to sneak it in as a non-breaking change:

public void Process(CustomerOrder order, DateTime orderDate, bool isGift = false)
{
    if (orderDate < DateTime.Now)
        throw new ArgumentException("orderDate");
}

But then you start reading and realize that life isn't that simple and that you're probably going to break your clients anyway. Fed up, you decide that you're going to prevent yourself ever from being bothered by this again. You'll write the API that stands the test of time:

public void Process(CustomerOrder order, Dictionary options)
{
    if(((DateTime)options["orderDate"]) < DateTime.Now)
        throw new ArgumentException("options");
}

Now, this can never be wrong. CustomerOrder can't be touched, and the options dictionary can support any extensions that are requested of you from here forward. If changes need to be made, you can make them internally without publishing painful changes to the API. You have, fortunately, separated your concerns enough that you can simply deploy a new DLL that handles order processing, and any new values supplied by your clients can be handled. No more API changes -- just a quick update, some testing, and an explanatory Word document sent to your client explaining how to use the thing. Here's the first one:

public class ProcessingClient
{
    private OrderProcessor _orderProcessor = new OrderProcessor();
    public void SubmitAnOrder(CustomerOrder order)
    {
        var options = new Dictionary();
        options["orderDate"] = DateTime.Now;
        options["isGift"] = true;
        _orderProcessor.Process(order, options);
    }
}

There. A flexible API and the whole "is gift" thing neatly handled. If they specify that it's a gift, you handle that. If they specify that it isn't or just don't add that option at all, then you treat those equally as the default case. Important stakeholder satisfied, and you won't be bothered with nasty publications. So, all good, right?

Flexibility, but at what cost?

I'm guessing that, at a visceral level, your reaction to this sequence of events is probably to cringe a little, even if you're not sure why. Maybe it's the clunky use of a collection type instead of something slicker. Maybe it's the (original) passing of a Boolean to the method. Perhaps it's simply to demand to know why CustomerOrder is inviolate or why we couldn't work to an order interface or at least define an inheritor. Maybe "options" reminds you of ViewState.

But, whatever it is, doesn't defining a system boundary that doesn't need to change seem like a worthwhile goal? Doesn't it make sense to etch painful boundaries in stone so that all parties can rely on them without painful integration? And if you're going to go that route, doesn't it make sense to build in as much flexibility as possible so that all parties can continue to innovate?

Well, that brings me to the thing that makes me wince about this approach. I'm not a fan of shying away from the pain of "icky publish/integration" instead of going with "if it hurts, do it more and get better at it." That shying away doesn't make me wince in and of itself, but it does seem like the wrong turn at a fork in the road to what does make me wince, which is the irony of this 'flexible' approach. The idea in doing it this way is essentially to say, "okay, publishing sucks, so let's lock down the integration point so that all parties can work independently, but let's also make sure that we're future proof so we can add functionality later." Or, tl;dr, "minimize multi-party integration coordination with hyper-flexible API."

So where's the irony? Well, how about the fact that any new runtime-bound additions to "options" require an insane amount of coordination between the parties? You're now more coupled than ever! For instance, let's say that we want to add a "gift wrap" option. How does that go? Well, first I would have to implement the functionality in the code. Then, I'd have to test and deploy my changes to the server, but that's only the beginning. From there, I need to inform you what magic string to use, and probably to publish a Word document with an example, since it's easy to get this wrong. Then, once you have that document, I have to go through my logs and troubleshoot to discover that, "oh yeah, see that -- you're passing us 'shouldGiftwrap' when it should really be 'shouldGiftWrap' with a capital W." And if I ever change it, by accident or on purpose? You'll keep compiling and running, and everything will be normal except that, from your perspective, gift wrapping will just quietly stop working. How much pain have we saved in the end with this non-discoverable, counter-intuitive, but 'flexible' setup? Wouldn't it be better not to get cute and just make publishing a more routine, friction-free experience?

The take-away that I'd offer here is to consider something about your code and your software that you may not previously have considered. It's relatively easy to check your code for simple defects and even to write it in such a way to minimize things like duplication and code churn. We're good at figuring out how not to have to keep doing the same thing over and over as well and to simplify. Those are all good practices. But the new thing I'd ask you to consider is "how much out of band knowledge does this require between parties?"

It could be a simple scenario like this, with a public facing API. Or, maybe it's an internal integration point between your team and another team. But maybe it's even just the interaction surface between two modules, or even classes, within your code base. Do both parties need to understand something that's not part of the method signatures and general interaction between these entities? Are you passing around magic numbers? Are you relying on the same implicit assumptions in both places? Are there things you're communicating through a means other than the actual interactions or else just not communicating at all? If so, I suggest you do a mental exercise to ask yourself what would be required to eliminate that out of band communication. Otherwise, today's clever ideas become tomorrow's maintenance nightmares.

By

Avoiding the Perfect Design

One of the peculiar ironies that I’ve discovered by watching the way a lot of different software shops work is that the most intense moments of exuberance about software seem to occur in places where software development happens at glacial speeds. If you walk into an agile shop or a startup or some kind of dink-and-dunk place that bangs out little CRUD apps, you’ll hear things like, “hey, a user said she thought it’d be cool if she could search her order history by purchase type, so let’s throw that in and see how it goes.” If it goes insanely well, there may be celebrations and congratulations and even bonuses changing hands which, to be sure, makes people happy. But their happiness is Mercury next to the blazing Sun of an ivory tower architect describing what a system SHALL do.

“There will be an enterprise service bus. That almost goes without saying. The presentation tier and the business tier will be entirely independent of one another, and literally any sort of pluggable module that you dream up as a client can communicate with any sort of rules engine embedded within the business tier. Neither one will EVER know about the other’s existence. The presentation layer collaborators are like Schrodinger and the decision engines are like the cat!

And the clients. Oh yes, there will be clients. The main presentation tier client is a mobile staging environment that will be consumed by Android, iOS, Windows Phone, Blackberry, and even some modified Motorolla walkie-talkies. On top of the mobile staging environment will be a service adapter that makes it so that clients don’t need to worry about whether they’re using SOAP or REST or whatever comes next. All of those implementations will hide behind the interface. And that’s just the mobile space. There are more layers and subtleties in the browser and desktop spaces, since both of those types of clients may be SPAs, other thick clients, thin clients, or just leaf nodes.

Wait, wait, wait, I’m not finished. I haven’t even told you about the persistence factories yet and my method for getting around the CAP theorem. The performance will be sublime. We’re talking picoseconds. We’re going to be using dynamically generated linear programming algorithms to load balance the vertical requests among the tiers, and we’re going to take a page out of the quantum computing book to introduce a new kind of three state boolean… oh, sorry, you had a question?”

“Uh, yeah. Why? I mean, who is going to use this thing and what do they want with it?”

“Everyone. For everything. Forever.”

You back out slowly as the gleam in his eye turns slightly worrisome and he starts talking about the five year plan that involves this thing, let’s call it HAL, achieving sentience, bringing humankind to the cusp of the singularity, and uploading the consciousnesses of all network and enterprise architects.

Singularity

Like I said, the Sun to your Mercury. Has your puny startup ever passed the Turing Test? Well, his system has… as spec’ed out in a document management system with 8,430 pages of design documents and a Visio diagram that’s rumored to have similar effects to the Ark of the Covenant. And that, my friends, is why I think that a failing ATDD scenario should be the absolute first thing anyone who says, “I want to get into programming” learns to do.

Now to justify that whiplash-inducing segue. I wrote a book about unit testing in which I counseled complete initiates to automated testing to forgo TDD and settle for understanding the mechanics of automated tests and test runners first before making the leap. I stand by that advice, but I do so because I think that there is a subtle flaw to the way that most people currently get started down the programming path.

I was watching a Pluralsight course about NUnit to brush up on their latest and greatest assertion semantics, and the examples were really well done. In particular, there was a series of assertions oriented around a rudimentary concept of a role playing game with enumerations of weapons, randomization of events, and hit points. This theme exercised concepts like ranges, variance, collection cardinality, etc and it did so in a way that lent itself to an easy mental model. The approach was very much what mine would have been as well (I wouldn’t have come at this with TDD because there’d have been a lot of ‘downtime’ writing the production code as opposed to just showing the assert methods).

Nevertheless, it’s been a while since I’ve watched someone write tests against a pre-baked system when they weren’t characterization tests in a legacy rescue, and the experience was sort of jarring. I couldn’t help but think, “I wouldn’t want to write these tests if I were in his position — why bother when the code is already done?” Weird as it sounds from a big advocate of testing, writing tests after you’ve completed your implementation feels like a serious case of “going through the motions” in the same way that developers fill out random “SDLC” artifacts for no other purpose than to get PMPs to leave them alone.

And that’s where the connection to the singularity architect comes in. One of the really nice, but subtle perks of the TDD (especially ATDD) approach is that it forces you to define exit criteria before you start doing things. For instance, “I know I’ll be done with this development effort when my user can search her order history by purchase type.” Awesome — you’re well on your way because you’ve (presumably) agreed with stakeholders ahead of time when you can stop coding and declare victory. The next thing is to prove it, and you can approach this in the same way that you might approach fixing a leaking pipe under your sink. Turn the water on, observe the leak, turn the water off, fix the leak, turn the water back on, observe that there is no leak. Done.

In the case of the search, you write a client call to your web service that supplies a “purchase type” parameter and you say that you’re done when you get a known result set back, instead of the current error message: “I do not understand this ‘purchase type’ nonsense you’ve sent — 400 for you!” Then you scurry off to code, and you just keep going until that test that you’ve written turns green and all of the other ones stay green. There. Done, and you can prove it. Ship it.

Our poor architect never knows when he’s done (and we know he’ll never be done). The origin of this Sisyphean struggle started with hobby programming or a CS degree or something. It started with unbounded goals and the kinds of open-ended tasks that allow hobbyists to grow and students to excel. Alright, you’ve got the A, but try to play with it. See if you can make it faster. Try adding features to it. Extra credit! Sky’s the limit! At some point, a cultural norm emerges that says it’s more about the journey than the destination. And then you don’t rise through the ranks by automating for the sake of solving people’s problems but rather by building ever-more impressive juggernauts, leveraging the latest frameworks, instrumented with the most analytics, and optimized to run in O(Planck Time).

I really would like to see initiates to the industry learn to set achievable (but slightly uncomfortable) goals with a notion of value and then reach them. Set a beneficial goal, reach it, rinse, repeat. The goal could be “I want to learn Ruby and I’ll consider a utility that sorts picture files to be a good first step.” You’re adding to your skill set as a developer and you have an exit criteria It could be something for a personal project, a pro-bono client, or for pay. But tie it back to an outcome and assess whether that outcome is worthwhile. This approach will prevent you from shaving microseconds off of an app that runs overnight on a headless server and it will prevent you from introducing random complexity and dependency to an app because you wanted to learn SnazzyButPointless.js. True, the approach will stop you from ever delighting in design documents that promise the birth of true artificial intelligence, but it will also prevent you from experiencing the dejection when you realize it ain’t ever gonna happen.

By

Encapsulation vs Inversion of Control

This is a post that I’ve had in my drafts folder for nearly two years. Well, I should say I’ve had the title and a few haphazard notes in my drafts folder for nearly two years; I’m writing the actual post right now. The reason I’ve had it sitting around for so long is twofold: (1) it’s sort of a subjective, tricky topic and (2) I’ve struggled to find a concrete stand to take. But, I think it’s relatively important, so I’ve always vowed to circle back to it and, well, here we are.

The draft was first conceived when I’d given a presentation about testability and inversion of control — specifically, using dependency injection via constructors and setters to achieve these ends. I talked, among other things about the Open/Closed Principle, and how this allows modifications to system behavior that favor adding over editing code. The idea here is that we can achieve new functionality with a minimum of violence to the code base and in a way for which it is easy to write unit tests. Everyone wins, right?

Well, not everyone was drinking the Kool-Aid. I fielded a question about encapsulation that, at the time, I hadn’t prepared to answer. “Doesn’t this completely violate encapsulation?” I was a little poleaxed, and sputtered out an answer off the cuff, saying basically, “well, not completely….” I mean, if you’ve written a class that takes ILogger in its constructor and uses it for logging, I control the logger implementation but you control when and how it is used. So, you encapsulate the logger’s usage but not its implementation and this stands in contrast to what would happen if you instantiated your own logger or implemented it yourself — you would encapsulate everything and nothing would be up to me as a client of your code. Certainly, you have more encapsulation. So, I finished: “…not completely…. but who cares?!?” And that was the end of the discussion since we were out of time anyway.

koolaid

I was never really satisfied with that answer but, as Creedence Clearwater says, “time and tears went by, and I collected dust.” When I thought back to that conversation, I would think to myself that encapsulation was passe in the same way that deep inheritance hierarchies were passe. I mean, sure, I learned that encapsulation was one of the four cornerstone principles of OOP, but so is inheritance, and that’s kind of going away with “favor composition over inheritance.” So, hey, “favor dependency injection over encapsulation.” Right? Still, I didn’t find this entirely satisfying — just good enough not to really occupy much of a place in my mind.

But then I remember a bit of a brouhaha last year over a Stack Overflow question. The question itself wasn’t especially remarkable (and was relatively quickly closed), but compiler author and programming legend Eric Lippert dropped by to say “DI is basically a bad idea.” To elaborate, he said:

There is no killer argument for DI because DI is basically a bad idea. The idea of DI is that you take what ought to be implementation details of a class and then allow the user of the class to determine those implementation details. This means that the author of the class no longer has control over the correctness or performance or reliability of the class; that control is put into the hands of the caller, who does not know enough about the internal implementation details of the class to make a good choice.

I was floored. Here we have one of the key authors of the C# compiler saying that the “D” in the SOLID principles was a “bad idea.” I would have dismissed it as blasphemy if (1) I were the sort to adopt approaches based on dogma and (2) he hadn’t helped author at least 4 more compilers than I have. And, while I didn’t suddenly rip the IoC containers out of my projects and instantiate everything inside constructors, I did revisit this topic in terms of my thoughts.

Maybe encapsulation, in the information hiding sense, isn’t so passe. And maybe DI isn’t a magic bullet. But why not? What’s wrong with the author of a class ceding control over some aspects of its behavior by allowing collaboration? And, isn’t any method parameter technically a form of DI, if we’re going to be pedantic about it?

The more I thought about it, the more I started to see competing and interesting use cases. Or, I should say, the more I started to think what class authors are telling their collaborators by using each of these techniques:

Encapsulation: “Don’t worry — I got this.”
Dependency Injection: “Don’t worry — if this doesn’t work, you can always change it.”

So, let’s say that you’re writing a Mars Rover or maybe a compiler or something. The attitude that you’re going to bring to that project is one in which correctness, performance and reliability are all incredibly important because you have to get it right and there’s little room for error. As such, you’re likely going to adopt implementation preference of “I’m going to make absolutely sure that nothing can go wrong with my code.”

But let’s say you’re writing a line of business app for Initrode Inc and the main project stakeholder is fickle, scatterbrained, and indecisive. Then you’re going to have an attitude in which ease and rapidity of system changes is incredibly important because you have to change it fast. As such, you’re likely to adopt an implementation preference of “I’m going to make absolutely sure that changing this without blowing everything up is easy.”

There’s bound to be somewhat of an inverse relationship between flexibility and correctness. As a classic example, a common criticism of Apple’s “walled garden” approach was that it was so rigid, while a common praise of the same was how well it worked. So I guess my take-away from this is that Dependency Injection and, more broadly, Inversion of Control, is not automatically desirable, but I also don’t think I can get to Eric’s take that it’s “basically a bad idea,” either. It’s simply an exchange of “more likely to be correct now” for “more likely to be correct later.” And in the generally agile world in which I live and with the kind of applications that I write, “later” tends to give more value.

Uncle Bob Martin once said, I believe in his Clean Coders video series, that (paraphrased) the second most important characteristic of good software is that it meet the customer requirements. The most important characteristic is that it be easy to change. Reason being, if a system is correct today but rigid, it will be wrong tomorrow when the customer wants changes. If the system is wrong today but flexible, it’s easy to make it right tomorrow. It may not be perfect, but I like DI because I need to be right tomorrow.

By

Notes on Writing Discoverable Framework Code

A while back, I found myself plugged into working on a rather large .NET project. The project had been going on for a while, and I was helping out only in terms of implementation manpower; architectural decisions had long since been made. And, in fact, some of the architectural decisions had pre-dated the project altogether. There was a “framework” that these guys were using for the purpose of rapid implementations. The idea was that developers could sort of slap this framework into place and make a few tweaks to have new applications rather than having to take more time to write the code. This wasn’t especially effective, but that was the goal.

As I used (and kind of battled with) this framework, something bothered me about it that I couldn’t put my finger on at first. But as I worked with it, I started to figure it out. The framework wasn’t discoverable in the slightest and thus fought against its own goal of rapid ramp-up and development. I actually put a placeholder in my drafts folder all of those moons ago wherein I made notes about how to make a framework discoverable. I never wound up bothering to make suggestions for improvements to that framework since, luckily, it was the last time I had to use it. But I’m trying to reduce the number of unpublished drafts in my folder, so I figured I’d offer up the suggestions to the world at large so that maybe some of you reading can get some value out of them.

What Do You Mean, Discoverable?

In the last paragraph I used a word without defining it, so I’ll correct that here. To understand what I mean, ask yourself the question, “how do developers discover how to use code?” 10 years ago, the most common answer to that question would have been “read the documentation.” Ah, the bad old days. I remember doing some Linux kernel programming using a library called RTAI, and I had these binders full of documentation on the API. They described in detail what each of the hundreds of parameters for dozens of functions were and what they meant. You certainly couldn’t tell from the names of these things — they were abbreviated, cryptic, and unclear.

The API itself lent few obvious clues about how to use it, and documentation was necessary for understanding. This is an API that is not discoverable. Discoverability is the degree to which you can figure out how to use something simply by playing with and examining it. For instance, if some API had a method called “Connect_To_The_SQL_Server_Database_Using_The_Following_Connection_String(string connection_string)”, you’d be dealing with something highly discoverable; you know how to use this without doing anything but looking at the code.

Developers have a variety of strategies for discovering how to use code. Reading documentation is probably the most traditional and established method, and one with which most experienced developers are familiar. However, let’s be frank — it’s also kind of a bummer. When you’re coding, you just want to write code via experimentation, trial and error. Having to lug out a big binder and pore through it, parsing jargon-heavy text is definitely a fly in the ointment. And while I’ve seen a lot of people express the sentiment, “you have to fight through the burn or you’re not a real programmer,” the fact of the matter is that when something is drudgery, attention spans wander and productivity decreases. So while reading documentation is the most established way of discovering code, it’s also the least effective, typically.

By contrast, the most effective manner of code discovery is use. Developers start using an API in whatever way seems most intuitive to them, typically using tools like Intellisense or word completion. They have some class they’ve declared an instance of in their IDE and they type the variable name and then a period and then they see what the auto-completion tool gives them by way of options for methods. There are other ways for developers to discover APIs as well, such as reading the code when it’s available, pairing with the code’s author, discussing the code with others, etc. But by and large, discovery through experimentation and clarity of purpose is most effective.

So the important question for architects, API authors, and framework providers thus becomes “how do I make my stuff discoverable for the developers that will use it?” Here are some suggestions.

Favor composition over inheritance

Inheritance is one of the main facets of polymorphism and boy oh boy was it popular in the late 1990’s and early 2000’s, especially in Java-land. I recall working on and with inheritance hierarchies that were 10 deep before any useful functionality emerged. Inheritance was one of the best ways at the time to avoid code duplication, but we, as an industry, learned the ugly downside of this approach: confusing non-discoverability.

There is nothing quite like getting a bug report that your code isn’t working and then taking a look to see what went wrong. You look at the source control history and see no record of any change that would explain the bug. None of the classes that your class is using have changed. Is it a bug in the GUI technology or database driver or something? Nope. Turns out, someone checked in a change 3 levels above you in the inheritance hierarchy and suddenly your class behaves differently. /Sigh.

When you use composition (the practice where your class uses other classes or implements interfaces), this sort of mysterious behavior in your classes is eliminated. If your class’s behavior has changed and you haven’t changed it, you know that one of the things that you’re using must have been changed. Eliminating or minimizing inheritance gets rid of a source of confusing possible changes.

Favor few parameters over many

This is relatively straightforward, but avoid using a lot of parameters with your methods. The fewer the parameters, the fewer the amount of things that callers of your methods have to understand and remember. This is a “Golden Rule” kind of situation. When you find the method you want to use by name, but it takes 12 arguments, do you think to yourself, “perfect — just what I want!” or do you think to yourself, “ugh, why me?” Well, users of your API/framework are no different. Consolidate the parameters you require into objects if need be — this will still be easier to understand for clients of your work.

Be judicious with overload methods

Method overloads are a powerful tool, but use them sparingly. It’s nice to offer your clients options, but it’s easy to offer them too many options. Applying the “golden rule” concept from the previous section, imagine if your code autocomplete tool shows you the method that you want, but there are 20 different parameter lists. Ugh. Which one to use? This isn’t discoverable anymore because you’re going to start googling or reading documentation or something. If there’s only one option, you’re simply going to use it and move on.

Favor early binding over late binding

One of the nastiest facets of the framework that I mentioned in the introductory section of the post was rampant late binding. Late binding is a scheme where relationships are resolved at run time rather than compile time. A common example of late binding would be a scheme that uses reflection to resolve names of fields in code to strings and to sync them with values on a form. So, if you have a Customer object with a “Name” property, you use reflection to get the string “Name” and look for a matching string on the form to which to write that value. Contrast this with a scheme where you simply have a strongly typed form object and you assign a value to its “Name” property. In the former scheme, mis-typing the property name results in a confusing runtime exception whereas in the latter scenario, the code won’t compile.

Programmers understand non-compiling. It’s one of the first feedback schemes you learn with a new language and you’ll very quickly understand why the compiler complains at you. So when something doesn’t compile, it’s quick and easy to figure this out and fix it. But if something fails weirdly at runtime as with a late binding scheme, it becomes hard to figure out what’s wrong. And, more importantly, it’s very frustrating. Part of discoverability is that it’s easy to understand why the right things work and the wrong things don’t. Late binding makes this understanding far more difficult for people working with the code.

Fail as early as possible

Very closely related is the concept of failing fast and early. An important concept for programmers is a tight feedback loop — a minimum of time between when you take an action and see the result of that action. A framework that creates negligible lapses between action and reaction is a discoverable one because users can quickly understand what works and what doesn’t. If there’s a lag between action and reaction, things are harder to figure out. This is a big problem with late binding, but it would also apply to a framework that’s slow running or slow to build as well.

Make screwing up impossible

Perhaps most important to discoverability is to make screwing up as hard as possible. Have clients of your code “fall into the pit of success.” This means taking whatever steps are possible to disallow doing the wrong thing. If a class Car in your framework requires an Engine in order to work, force people using your framework to pass Engine to Car’s constructor so that if they don’t, the code won’t compile. If they pass null, throw an exception. Contrast this with a behavior where the code makes passing in an engine optional and simply limps along and maybe throws a null reference exception somewhere if it’s missing. The latter approach is confusing — you’re communicating through your API that the Engine isn’t needed, and yet you’re failing everywhere it’s not present. You’re creating a minefield for your users rather than a pit of success, and consequently making your stuff really non-discoverable.

By the way, if you liked this post and you're new here, check out this page as a good place to start for more content that you might enjoy.

By

Uber-Architects: The Building Metaphor Is Dead

Building Bad Software is Like Building a Tower

A little over a year ago, I wrote a post explaining how you can use the metaphor, “building software is like building a tower” to help you write bad software. The idea was that building something like a skyscraper requires insane amounts of planning because the actual act of building is laborious, time-consuming, and expensive, and also pretty much impossible to change once you get going. Furthermore, the real brains of the operation is required for the up-front planning, which is then done in such detail that the actual construction is a pretty straight-forward task that doesn’t require a whole lot of thinking — just following of detailed instructions and a practiced knack for skills like assembling plumbing, taping drywall joints, etc.

Software really only follows this pattern if it’s awful. If you were to describe a project you were working on by saying, “it’s important that we get everything right from the get-go because once we get started, this system is going to be so set and rigid that change is impossible,” wouldn’t you be a little alarmed and/or defeated? And what about the idea that the developer is so wildly different from the architect that they have two entirely separate vocational training paths (in the case of buildings, architectural studies versus carpentry trade school or apprenticeship). Is planning how to write software so very different than writing software?

I believe you’d be pretty hard pressed to continue to like this metaphor when you give it a lot of thought. But that hasn’t stopped the metaphor from being iconic in our industry, to the extent that it still vestigially governs roles, titles, career paths, and team behavior. Even though building software is nothing like structural construction, we continue to have a very real role/title called “Architect” that is responsible for coming up with documentation that looks suspiciously like a set of blueprints so that the lower paygrade laborers can toil away while he supervises.

Is this the best role and set of duties for the person called “architect” — the person on the team who has probably gotten to the position by being a good developer for a long time (or, more cynically, by being a mediocre and tenured Expert Beginner)? Should the result of spending years getting good at writing software be that you get to be in charge of the software by writing less of it? Are the architects of our buildings the people who are really, really good at pouring concrete and hanging drywall? Obviously not. And while we’re at it and this frayed metaphor is truly breaking down, should we even call them architects any more?

A Bit of Philosophy

I think that those of us who occupy the role or aspire to it should perhaps start striving to become post-architects or Uber-Architects. I’m borrowing this latter term from Friedrich Nietzsche, a philosopher and sporter of an awesome mustache that wrote about what he called the “Ubermensch,” which I believe translates from German to something along the lines of “over-man.” Nietzsche’s concepts in “Thus Spake Zathustra” are extremely nuanced and layered, but I’ll summarize what I took out of it when I read it and how I think it relates.

Nietzsche

You might know Nietzsche as the philosopher who famously said, “God is dead,” and he said that in this book. I believe that this is largely interpreted as a profoundly and stridently atheist sentiment, but that interpretation is one that ignores the context of the work and his other works. He was looking at a world (he lived in Europe during the 1800s) where morality had long been a matter simply dictated by the tenets of Christianity, but also in a world where the rejection of religion was becoming increasingly common. Nietzsche wasn’t gloating over the corpse of God; he was expressing worry that a growing atheist/agnostic segment of the population would opt for nihilism in the absence of any religion and that society would become dominated by what he called something like “last man,” a rather wretched creature interested only in its own comfort and with no direction or broader purpose. He was saying, “I don’t care for your religion, but if you take it away, I’m worried that things will get a lot worse.”

From Nietzsche’s perspective, 19th century Europe was in trouble and the path forward was for mankind to become the “Ubermensch,” a version of man that was capable of supplying himself with all of the things for which religion had previously been responsible. Basically, he should do good in life because it’s good to do good rather than because he’ll be in trouble if he doesn’t. He should define his own purpose and leave his mark on the world because it’s the right thing to do and the highest calling for an individual would be to leave a mark on history for the better. In the absence of the previous and declining moral order, a new, sustainable one had to be defined, so his argument went (or at least my recollection of my reading and understanding of it).

Forget the religious angle here in a discussion of software. I’m not interested in discussing the merits of Nietzsche’s religious beliefs or lack thereof here. But I am interested in relating his perception of the world to our situation. Throughout the history of software development, our roles have been defined by this now flagging and failing metaphor of “software is like building a tower.” We’ve mimicked construction in our historical approach with lengthy and detailed planning along with the division of labor. We’ve gone so far as to borrow the titles for the roles in that line of work and appropriate them for ourselves. Your software group has to have an “architect” that will make the “blueprints” for the software. But that’s absurd and people are starting to realize it (see the growth of agile methodologies that have no equivalent at all in the construction world).

The danger then becomes what happens in the absence of that metaphor. Do we adopt improvident and cavalier approaches to software architecture, swinging the other way on the “lots of planning versus no planning” pendulum? Do we abolish the role of any kind of technical leader and make all software development groups pure democracy? Do the former architects or developers in general become “last architects,” just nihilistically banging out whatever code seems interesting or gets them out the door at 5 without worrying over the future or the needs of the business?

Emergence of the Uber-Architect

This is where the Uber-Architect comes in. The Uber-Architect deals not in blueprints and orders from on high but from leadership by example in the trenches. Uber-Architecture isn’t about web services, database technologies, N-Tiers or enterprises, but about teaching and demonstrating important fundamental concepts of the craft: abstractions, design trade-offs, and implementation patterns. Uber-Architects don’t create a bunch of rules and enforce them across large organizations for consistency’s sake, like a foreman with a clipboard overseeing hundreds of fungible laborers. They pick up the hammers and nails and work along side those workers, showing them how it’s done, building the thing together, and making those around them better until it is no longer a fungible collection of workers, but a humming, autonomous machine that’s more than the sum of its parts. They leave their mark on the group not because they’re “architect” but because it’s the right thing to do, and it’s something of which they can be proud.

So what do teams look like when all of this comes to pass? I don’t know, exactly. But I think we’re finding out. I think that we’re going to see more and more teams with flatter structures, less worried about seniority, and more buying into the agile concept of self-organizing teams. And on those teams, there is no architect because people doing a good job of building software won’t assemble into organizational structures that are ill suited to building software. On these teams, there will only be Uber-Architects, who don’t hold a position afforded to them by 15 years in with the company, but who hold a place of respect among their peers due to ability and vision, and who create design concepts that make the lives of those around them easier and the skills of those around them sharper.

If this sounds overly idealistic, perhaps it is, but that’s because I view it as a goal and something to start reaching toward. And besides, with all of the cynical posts I make about Expert Beginners and overrated people and whatnot, some starry-eyed optimism probably balances out the cosmic scales a bit.