How Software Groups Rot: Legacy of the Expert Beginner

Expert Beginner Recap

In my last post I introduced the term “Expert Beginner” to describe someone who has capped out in their learning at some sort of local maximum, convinced that the local is global. Expert Beginners are developers who do not understand enough of the big picture to understand that they aren’t actually experts. What I mean by this is that they have a narrow enough perspective to think that whatever they have been exposed to is the best and only way to do things; examples would include a C# developer who pooh-poohs Java without ever having tried it, or a MySQL DBA who dismisses the NoSQL movement as a passing fad. This isn’t to say that not liking a technology or not having worked with one makes someone an Expert Beginner, but rather the vaguely solipsistic mindset that “if it’s not in my toolchest or something I’ve had experience with, it’s not worth doing.”

Another characteristic of the Expert Beginner, however, is that they have some position of relative authority or influence within a software group. In my last post, I proposed the term Expert Beginner without explaining the rationale, planning to discuss that here. The Advanced Beginner is someone who is in the advanced stage of being a beginner, whereas the Expert Beginner descriptor is both literal and intentionally ironic; it’s literal in that someone with that much practice at being a beginner could be said to be an expert and ironic in that the “expert” title is generally self-applied in earnest or else applied by managers or peers who don’t know any better.

The iconic example might be the “tech guy” at a small, non-technical firm. He “knows computers” so as the company grew and evolved to have some mild IT needs, he became a programming dilettante out of necessity. In going from being a power user to being a developer, his skills exceeded his expectations, so he became confident in his limited, untrained abilities. Absent any other peers in the field, the only people there to evaluate his skills are himself and non-technical users who offer such lofty praises as “it seems to work, kind of, I think.” He is the one-eyed man in the valley of the blind and, in a very real and unfortunate sense, a local expert. This is the iconic example because it has the fewest barriers to Expert Beginnerism–success is simple, standards are low, actual experts are absent, competition is non-existent, and external interaction is not a given.

A Single Point of Rot…

So far I’ve talked a lot about the Expert Beginner–his emergence, his makeup, his mindset, and a relatively sympathetic explanation of the pseudo-rationality, or at least understandability, of his outlook. But how does this translate into support of my original thesis that Expert Beginners cause professional toxicity and degeneration of software groups? To explain that, I’m going to return to my bowling analogy, and please bear with me if the metaphor is a touch strained. For those of you who didn’t read the first post, you might want to give the second section of it a read because this is a lot to rehash here.

Let’s say that bowling alleys make their money by how well their bowlers bowl and that I live in a small town with a little, startup bowling alley. Not being able to find any work as a software developer, I try my hand bowling at the local alley. I don’t really know what I’m doing, and neither do they, but we both see me improving rapidly as I start bowling there, in spite of my goofy style. My average goes up, the bowling alley makes money, and life is good–there’s no arguing with profit and success!

Around about the time my score was topping 150 and the sky seemed the limit, the bowling alley decided that it was time to expand and to hire a couple of entry-level bowlers to work under my tutelage. On the day they arrived, I showed them how to hold the ball just like I hold it and how to walk just like I do. When they ask what the thumb and finger holes are for, I respond by saying, “don’t worry about those–we don’t use them here.” Eager to please, they listen to me and see their averages increase the way mine had, even as I’m starting to top out at around a 160 average.

As time goes by, most of them are content to do things my way. But a couple are ambitious and start to practice during their spare time. They read books and watch shows on bowling technique. One day, these ambitious bowlers come in and say, “the guys on TV put their fingers in the ball, and they get some really high averages–over 200!” They expect me to be as interested as they are in the prospect of improvement and are crestfallen when I respond with, “No, that’s just not how we do things here. I’ve been bowling for longer than you’ve been alive and I know what I’m doing… besides, you can’t believe everything you see on TV.”

And thus, quickly and decisively, I squash innovation for the group by reminding them that I’m in charge by virtue of having been at the bowling alley for longer than they have. This is a broadly accepted yet totally inadequate non sequitur that stops discussion without satisfying. At this point, half of the ambitious developers abandon their “fingers in the ball” approach while the other half meet after bowling at another alley and practice it together in semi-secret. After a while, their averages reach and surpass mine, and they assume that this development–this objective demonstration of the superiority of their approach–will result in a change in the way things are done. When it instead results in anger and lectures and claims that the scores were a fluke and I, too, once bowled a 205 that one time, they evaporate and leave the residue behind. They leave my dead-end, backward bowling alley for a place where people don’t favor demonstrably inferior approaches out of stubbornness.

The bowling alley loses its highest average bowlers not to another alley, but to an Expert Beginner.

…That Poisons the Whole

The bowlers who don’t leave learn two interesting lessons from this. The first lesson they learn is that if they wait their turn, they can wield unquestioned authority regardless of merit. The second lesson they learn is that it’s okay and even preferred to be mediocre at this alley. So when new bowlers are hired, in the interests of toeing the company line and waiting their turn, they participate in the inculcation of bad practices to the newbies the same way as was done to them. The Expect Beginner has, through his actions and example, created additional Expert Beginners and, in fact, created a culture of Expert Beginnerism.

The other interesting development that results comes in the acquisition process. As the Expert-Beginner-in-Chief, I’ve learned a pointed lesson. Since I don’t like being shown up by ambitious young upstarts, I begin to alter my recruitment process to look for mediocre “team players” that won’t threaten my position with their pie-in-the-sky “fingers in the ball” ideas. Now, I know what you’re thinking–doesn’t this level of awareness belie the premise of the Expert Beginner being unaware of the big picture? The answer is no. This hiring decision is more subconscious and rationalized than overt. It isn’t, “I won’t hire people that are better than me,” but, “those people just aren’t a good fit here with my ‘outside the box’ and ‘expert’ way of doing things.” And it may even be that I’m so ensconced in Expert Beginnerism that I confuse Competent/Expert level work with incompetent work because I don’t know any better. (The bowling analogy breaks down a bit here, but it might be on par with a “bowling interview” where I just watched the form of the interviewee’s throw and not the results, and concluded that the form of a 220 bowler was bad because it was different than my own.) And, in doing all this, I’m reinforcing the culture for everyone including my new Expert Beginner lieutenants.

And now the bowling alley is losing all of its potentially high average bowlers to a cabal of Expert Beginners. Also notice that Bruce Webster’s “Dead Sea Effect” is fully mature and realized at this point.

Back in the Real World

That’s all well and good for bowling and bowling alleys, but how is this comparable to real software development practices? Well, it’s relatively simple. Perhaps it’s a lack of automated testing. Giant methods/classes. Lots of copy and paste coding. Use of outdated or poor tooling. Process. It can be any number of things, but the common thread is that you have a person or people in positions of authority that have the culturally lethal combination of not knowing much; not knowing what they don’t know; and assuming that, due to their own expertise, anything they don’t know isn’t worth knowing. This is a toxic professional culture in that it will force talented or ambitious people either to leave or to conform to mediocrity.

You may think that this is largely a function of individual personalities, that departments become this way by having arrogant or pushy incompetents in charge, but I think it’s more subtle than that. These Expert Beginners may not have such personality defects at all. I think it’s a natural conclusion of insular environments, low expectations, and ongoing rewards for mediocre and/or unquantifiable performances. And think about the nature of our industry. How many outfits have you worked at where there is some sort of release party, even (or especially) when the release is over budget, buggy and behind schedule? How many outfits have you worked at that gave up on maintaining some unruly beast of an application in favor of a complete rewrite, only to repeat that cycle later? And the people involved in this receive accolades and promotions, which would be like promoting rocket makers for making rockets that looked functional but simply stopped and fell back to Earth after a few hundred feet. “Well, that didn’t work, Jones, but you learned a lot from it, so we’re promoting you to Principal Rocket Builder and having you lead version two, you rock star, you!” Is it any wonder that Jones starts to think of himself as King Midas?

As an industry, we get away with this because people have a substantially lower expectation of software than they do of rockets. I’m not saying this to complain or to suggest sweeping change but rather to explain how it’s easy for us to think that we’re further along in our skills acquisition than we actually are, based on both our own perception and external feedback.

Create a Culture of Acquisition instead of Stagnation

Having identified a group-(de)forming attitude that could most effectively be described as a form of hubris, I would like to propose some relatively simple steps to limit or prevent this sort of blight.

First of all, to prevent yourself from falling into the Expect Beginner trap, the most important thing to do is not to believe your own hype. Take pride in your own accomplishments as appropriate, but never consider your education complete or your opinion above questioning regardless of your title, your years of experience, your awards and accomplishments, or anything else that isn’t rational argumentation or evidence. Retaining a healthy degree of humility, constantly striving for improvement, and valuing objective metrics above subjective considerations will go a long way to preventing yourself from becoming an Expert Beginner.

In terms of preventing this phenomenon from corrupting a software group, here is a list of things that can help:

1. Give team members as much creative freedom as possible to let them showcase their approaches (and remember that you learn more from failures than successes).
2. Provide incentives or rewards for learning a new language, approach, framework, pattern, style, etc.
3. Avoid ever using number of years in the field or with the company as a justification for favoring or accepting anyone’s argument as superior.
4. Put policies in place that force external perspectives into the company (lunch-and-learns, monthly training, independent audits, etc).
5. Whenever possible, resolve disputes/disagreements with objective measures rather than subjective considerations like seniority or democratic vote.
6. Create a “culture of proof”–opinions don’t matter unless they’re supported with independent accounts, statistics, facts, etc.
7. Do a periodic poll of employees, junior and senior, and ask them to list a few of their strengths and an equal number of things they know nothing about or would like to know more about. This is to deflate ahead of time an air of “know-it-all-ism” around anyone–especially tenured team members.

This list is more aimed at managers and leaders of teams, but it’s also possible to affect these changes as a simple team member. The only difference is that you may have to solicit help from management or persuade rather than enforce. Lead by example if possible. If none of that is possible, and it seems like a lost cause, I’d say head off for greener pastures. In general, it’s important to create or to have a culture in which “I don’t know” is an acceptable answer, even for the most senior, longest-tenured leader in the group, if you want to avoid Expert Beginner fueled group rot. After all, an earnest “I don’t know” is something Expert Beginners never say, and it is the fundamental difference between a person who is acquiring skill and a person who has decided that they already know enough. If your group isn’t improving, it’s rotting.

Series is continued here: “How Stagnation is Justified: Language of the Expert Beginner”