DaedTech

Stories about Software

Sep
13

By

The Perils of Free Time at Work

Category: The Life of a Programmer Tags: , | 7 Comments

Profitable Free Time

If you’ve ever been invited to interview at Google or you simply keep up with the way it does things, you’re probably familiar with Google’s “20 percent time”. According to HowStuffWorks (of all places):

Another famous benefit of working at Google is the 20 percent time program. Google allows its employees to use up to 20 percent of their work week at Google to pursue special projects. That means for every standard work week, employees can take a full day to work on a project unrelated to their normal workload. Google claims that many of their products in Google Labs started out as pet projects in the 20 percent time program.

In other words, you can spend 4 days per week helping the company’s bottom line and one day a week chasing rainbows and implementing unicorns. The big picture philosophy is that the unbridled freedom to pursue one’s interests will actually result in profitable things for the company in the long run. This is a good example of something that will tend to keep programmers around by promoting mastery, autonomy, and purpose.

We tend to think of this perk as characteristic of progressive employer startups where you imagine beer in the fridge, air hockey tables and Playstations in the break room, and various other perks designed to make it comfortable to work 70+ hour weeks in the “work hard, play-hard” culture. But interestingly, this idea went all the way back to 3M in 1948:

3M launched the 15 percent program in 1948. If it seems radical now, think of how it played as post-war America was suiting up and going to the office, with rigid hierarchies and increasingly defined work and home roles. But it was also a logical next step. All those early years in the red taught 3M a key lesson: Innovate or die, an ethos the company has carried dutifully into the 21st century.

So, for over half a century, successful companies (or at least a narrow subset of them) have been successful by allowing their employees a portion of their paid time to do whatever they please, within reason. That seems to make a good case for this practice and for a developer who finds himself in this position to be happy.

We Interrupt this Post to Bring You A Lesson From Donald Rumsfeld

Every so often politicians or other public figures say things that actually make sense, but go down in sound byte lore as debacles. John Kerry’s “I voted for the bill before I voted against it” comes to mind, but a very unfortunately misunderstood one, in my opinion, is this one from Donald Rumsfeld:

[T]here are known knowns; there are things we know that we know. There are known unknowns; that is to say there are things that, we now know we don’t know. But there are also unknown unknowns – there are things we do not know, we don’t know.

Admittedly, this is somewhat of a rhetorical mind-twister, but when you think about what he’s saying, not only does it make sense, but it is fairly profound. For example, consider your commute to work. When you go to your car, you know what kind of car you drive. That is a “known-known”. As you get in your car to go to work, it may take you 30-50 minutes depending on traffic. The traffic is a “known-unknown” in that you know it will exist but you don’t know what it will be like and you are able to plan accordingly (i.e. you allow 50 minutes even though it may take you less). But what if, while getting in your car, someone told you that you wouldn’t be at work for about 4 hours? Maybe a fender-bender? Maybe a family member calls you with an emergency? Who knows… these are the “unknown-unknowns” — the random occurrence in life for which one simply can’t plan.

The reason that I mention this is that I’d like to introduce a similar taxonomy for free as it relates to our professional lives, and I’ll ask you to bear with me the way I think you should bear with Rummy.

Structured-Unstructured Time or Unstructured-Unstructured Time

Let’s adopt these designations for time at the office. The simplest way to talk about time is what I’ll call “structured-structured time”. This is time where your boss has told you to work on X and you are dutifully working X. Google/3M’s “20/15 percent time”, respectively, is an example of what I’ll call “structured-unstructured time.” This is time that the organization has specifically set aside to let employees pursue whims and fancies while not accountable to normal scheduling consideration. It is unstructured time with a purpose. The final type(**) of time is what I’ll call “unstructured-unstructured” time, and this is time that you spend doing nothing directly productive for the company but without the company having planned on you doing that. The most obvious example I can think of is if your boss tells you to go make 10,000 copies, the copy machine is broken, and you have no other assignments. At this point you have unstructured-unstructured time where you might do anything from taking it upon yourself to fix the copy machine to taking a nap in the break room.

Making this distinction may seem like semantic nitpicking, but I believe that it’s important. I further believe that unstructured-unstructured time chips away at morale even as the right amount of structured-unstructured makes it soar. With structured-unstructured time, the company is essentially saying “we believe in you and, in fact, we believe so much in you that we’re willing to take a 20 percent hit to your productivity on the gamble that you doing what interests you will prove to be profitable.” Having someone place that kind of confidence in you is both flattering and highly motivating. The allure of doing things that interest you combined with living up to the expectations will make you work just as hard during structured-unstructured time as you would at any other time, if not harder. I’ve never had this perk, but I can say without a doubt that this is the day I’d be most likely to put in 12 or 13 hours at the office.

Contrast this with unstructured-unstructured time and the message conveyed to you by the organization. Here the company is basically saying the opposite: “we value your work so little that we’d rather pay you to do nothing than distract someone much more important than you with figuring out what you should do.” Have you ever been a new hire and twiddled your thumbs for a week or even a month, perusing stuff on the local intranet while harried employees shuffled busily around you? Ever needed input or approval from a more senior team member and spent the whole day clicking through reddit or slashdot? How about telling a manager that you have nothing you can work on and hearing, “that’s okay — don’t worry about it — you deserve some downtime.”

The difference in how this time is perceived is plain: “we’re going to carve out time because we are excited to see just how much you can do” versus “we don’t really give a crap what you do.”

But Don’t People Like Free Time and Freedom From Pressure?

You would think that everyone would appreciate the unstructured-unstructured time when it came their way, but I believe that this largely isn’t the case. Most people start to get nervous that they’re being judged if they have a lot of this time. Many (myself included) start to invent side-projects or things that they think will help the company as ways to fill the vacuum and hopefully prove themselves, but this plan often backfires as organizations that can’t keep employees busy and challenged are unlikely to be the kind of organizations that value this “self-starter” type behavior and the projects it produces. So the natural tendency to flourish during unstructured time becomes somewhat corrupted as it is accompanied by subtle feelings of purposelessness and paranoia. About the only people immune to this effect are Dead-Sea, checked out types who are generally aware consciously or subconsciously that being productive or not doesn’t affect promotions or status anyway so they might as well catch up on their reading.

I believe there is a natural progression from starting off trying to be industrious and opportunistically create structured-unstructured time during unstructured-unstructured time to giving up and embarking on self improvement/training missions during that time to sending out resumes during that time. So I would caution you that if you’re a manager or lead, make sure that you’re not letting your team flail around without structure this way. If you think they’re out of tasks, assign them some. Make time for them. Or, if nothing else, at least tell them what google/3M tell them — we value your ingenuity, so this is actually planned (not that I’m advocating lying, but come on – you can figure out a good way to spin this). Just don’t think that you’re doing team members a favor or taking it easy on them by not giving them anything to do.

If you’re a developer with this kind of time on your hands, talk to your manager and tell him or her how you feel. Formulate a plan for how to minimize or productively use this time early on, before it gets out of hand and makes you restless for Career Builder. Other people do get busy and understandably so, and so it may require gentle prodding to get yourself out of this position.

But regardless of who you are, I advocate that you do your best to provide some structure to unstructured time, both for yourself and for those around you. Working with a purpose is something that will make just about anyone happier than the alternative.

** As an aside, both Rummy and I left out a fourth option, not closing the set. In his case, it would be the “unknown-known” and in my case, it would be “unstructured-structured time”. In both cases, this assumes utter incompetence on the part of the first person in the story – an “unknown known” would mean some obvious fact that was being missed or ignored. In the case of an organization, “unstructured-structure” would mean that team members had effectively mutinied under incompetent management and created some sort of ad-hoc structure to get work done in spite of no official direction. This is a potentially fascinating topic that I may later come back to, although it’s a little far afield for the usual subject matter of this blog.

Aug
12

By

How To Keep Your Best Programmers

Category: The Life of a Programmer Tags: , , | 79 Comments

Getting Philosophical

The thinking manGiven that I’ve just changed jobs, it isn’t entirely surprising that I’ve had a lot of conversations recently about why I decided to do so. Generally when someone leaves a job, coworkers, managers, HR personnel, friends, and family are all interested in knowing why. Personally, I tend to give unsatisfying answers to this question, such as, “I wanted a better opportunity for career advancement,” or, “I just thought it was time for a change.” This is the corporate equivalent of “it’s not you–it’s me.” When I give this sort of answer, I’m not being diplomatic or evasive. I give the answer because I don’t really know, exactly.

Don’t get me wrong. There are always organizational gripes or annoyances anywhere you go (or depart from), and it’s always possible that someone will come along and say, “How would you like to make twice as much money doing the coolest work imaginable while working from home in your pajamas?” or that your current employer will say, “We’re going to halve your pay, force you to do horrible grunt work, and send you to Antarctica to do it.” It is certainly possible that I could have a specific reason for leaving, but that seems more the exception than the rule.

As a general practice, I like to examine my own motivations for things that I do. I think this is a good check to make sure that I’m being rational rather than impulsive or childish. So I applied this practice to my decision to move on and the result is the following post. Please note that this is a foreword explaining what got me thinking along these lines, and I generalized my opinion on my situation to the larger pool of software developers. That is, I’m not intending to say, “I’m the best and here’s how someone can keep me.” I consider my own programming talent level irrelevant to the post and prefer to think of myself as a competent and productive developer, distinguished by enthusiasm for learning and pride in my work. I don’t view myself as a “rock star,” and I generally view such prima donna self-evaluation to be counterproductive and silly.

What Others Think

Some of my favorite blog posts that I’ve read in the last several years focus on the subject of developer turnover, and I think that these provide an excellent backdrop for this subject. The oldest one that I’ll list, by Bruce Webster, is called “The Wetware Crisis: the Dead Sea Effect,” and it coins an excellent term for a phenomenon with which we’re all probably vaguely aware on either a conscious or subconscious level. The “Dead Sea Effect” is a description of some organizations’ tendency to be so focused on retention that they inadvertently retain mediocre talent while driving better talent away:

…what happens is that the more talented and effective IT engineers are the ones most likely to leave — to evaporate, if you will. They are the ones least likely to put up with the frequent stupidities and workplace problems that plague large organizations; they are also the ones most likely to have other opportunities that they can readily move to.

What tends to remain behind is the ‘residue’ — the least talented and effective IT engineers. They tend to be grateful they have a job and make fewer demands on management; even if they find the workplace unpleasant, they are the least likely to be able to find a job elsewhere. They tend to entrench themselves, becoming maintenance experts on critical systems, assuming responsibilities that no one else wants so that the organization can’t afford to let them go.

Bruce describes a paradigm in which the reason for talented people leaving will frequently be that they are tired of less talented people in positions of relative (and by default) authority telling them to do things–things that are “frequent stupidities.” There is an actual inversion of the pecking order found in meritocracies, and this leads to a dysfunctional situation that the talented either avoid or else look to escape as quickly as possible.

Bruce’s post was largely an organizational perspective; he talked about why a lot of organizations wind up with an entrenched group of mediocre senior developers, principals, and managers without touching much on the motivation for the talented to leave beyond the “frequent stupidities” comment. Alex Papadimoulis from the Daily WTF elaborates on the motivation of the talented to leave:

In virtually every job, there is a peak in the overall value (the ratio of productivity to cost) that an employee brings to his company. I call this the Value Apex.

On the first minute of the first day, an employee’s value is effectively zero. As that employee becomes acquainted with his new environment and begins to apply his skills and past experiences, his value quickly grows. This growth continues exponentially while the employee masters the business domain and shares his ideas with coworkers and management.

However, once an employee shares all of his external knowledge, learns all that there is to know about the business, and applies all of his past experiences, the growth stops. That employee, in that particular job, has become all that he can be. He has reached the value apex.

If that employee continues to work in the same job, his value will start to decline. What was once “fresh new ideas that we can’t implement today” become “the same old boring suggestions that we’re never going to do”. Prior solutions to similar problems are greeted with “yeah, we worked on that project, too” or simply dismissed as “that was five years ago, and we’ve all heard the story.” This leads towards a loss of self actualization which ends up chipping away at motivation.

Skilled developers understand this. Crossing the value apex often triggers an innate “probably time for me to move on” feeling and, after a while, leads towards inevitable resentment and an overall dislike of the job. Nothing – not even a team of on-site masseuses – can assuage this loss.

On the other hand, the unskilled tend to have a slightly different curve: Value Convergence. They eventually settle into a position of mediocrity and stay there indefinitely. The only reason their value does not decrease is because the vast amount of institutional knowledge they hoard and create.

This is a little more nuanced and interesting than the simple meritocracy inversion causing the departure of skilled developers. Alex’s explanation suggests that top programmers are only happy in jobs that provide value to them and jobs to which they provide increasing value. The best and brightest not only want to grow but also to feel that they are increasingly useful and valuable–indicative, I believe, of pride in one’s work.

In an article written a few years later titled “Bored People Quit,” Michael Lopp argues that boredom is the precursor to developers leaving:

As I’ve reflected on the regrettable departures of folks I’ve managed, hindsight allows me to point to the moment the person changed. Whether it was a detected subtle change or an outright declaration of their boredom, there was a clear sign that the work sitting in front of them was no longer interesting. And I ignored my observation. I assumed it was insignificant. He’s having a bad day. I assumed things would just get better. In reality, the boredom was a seed. What was “I’m bored” grew roots and became “I’m bored and why isn’t anyone doing anything about it?” and sprouted “I’m bored, I told my boss, and he… did nothing,” and finally bloomed into “I don’t want to work at a place where they don’t care if I’m bored.”

I think of boredom as a clock. Every second that someone on my team is bored, a second passes on this clock. After some aggregated amount of seconds that varies for every person, they look at the time, throw up their arms, and quit.

This theme of motivation focuses more on Alex’s “value provided to the employee” than “value that employee provides,” but it could certainly be argued that it includes both. Boredom implies that the developer gets little out of the task and that the perceived value that he or she is providing is low. But, beyond “value apex” considerations, bored developers have the more mundane problem of not being engaged or enjoying their work on a day to day basis.

What’s the Common Thread?

I’m going to discount obvious reasons for leaving, such as hostile work environment, below-market pay, reduction of benefits/salary, etc., as no-brainers and focus on things that drive talented developers away. So far, we’ve seen some very compelling words from a handful of people that roughly outline three motivations for departure:

  • Frustration with the inversion of meritocracy (“organization stupidities”)
  • Diminishing returns in mutual value of the work between programmer and organization
  • Simple boredom

To this list I’m going to add a few more things that were either implied in the articles above or that I’ve experienced myself or heard from coworkers:

  • Perception that current project is futile/destined for failure accompanied by organizational powerlessness to stop it
  • Lack of a mentor or anyone from whom much learning was possible
  • Promotions a matter of time rather than merit
  • No obvious path to advancement
  • Fear of being pigeon-holed into unmarketable technology
  • Red-tape organizational bureaucracy mutes positive impact that anyone can have
  • Lack of creative freedom and creative control (aka “micromanaging”)
  • Basic philosophical differences with majority of coworkers

Looking at this list, a number of these are specific instances of the points made by Bruce, Alex and Michael, so they aren’t necessarily advancements of the topic per se, though you might nod along with them and want to add some of your own to the list (and if you have some you want to add, feel free to comment). But where things get a little more interesting is that pretty much all of them, including the ones from the linked articles, fall into a desire for autonomy, mastery, or purpose. For some background, check out this video from RSA Animate. The video is great watching, but if you haven’t the time, the gist of it is that humans are not motivated economically toward self-actualization (as widely believed) but are instead driven by these three motivating factors: the desire to control one’s own work, the desire to get better at things, and the desire to work toward some goal beyond showing up for 40 hours per week and collecting a paycheck.

Frustration with organizational stupidity is usually the result of a lack of autonomy and the perception of no discernible purpose. Alex’s value apex is reached when mastery and purpose wane as motivations, and boredom with a job can quite certainly result from a lack of any of the three RSA needs being met. But rather than sum up the symptoms with these three motivating factors, I’m going to roll it all into one. You can keep your good developers by making sure they have a compelling narrative as employees.

Guaranteeing the Narrative

Bad or mediocre developers are those who are generally resigned or checked out. They often have no desire for mastery, no sense of purpose, and no interest in autonomy because they’ve given up on those things as real possibilities and have essentially struck a bad economic bargain with the organization, pay amount notwithstanding. That is, they give up on self-actualization in exchange for a company paying a mortgage, a few car payments, and a set of utilities for them. I’ve heard a friend of mine call this “golden handcuffs.” They have a pre-defined narrative at work: “I work for this company because repo-men will eventually show up if I don’t.” These aren’t necessarily bad or unproductive employees, but they’re pretty unlikely to be your best and brightest, and you can be assured that they will tend to put forth the minimum amount of effort necessary to hold up their end of the bad bargain.

These workers are easy to keep because that is their default state of affairs. Going out and finding another job is not the minimum effort required to pay the bills, so they won’t do it. They are Bruce’s “residue” and they will tend to stick around and earn obligatory promotions and pay increases by default, and, unchecked, they will eventually sabotage the RSA needs of other, newer developers on the team and thus either convert them or drive them off. The narrative that you offer them is, “Stick around, and every five years we’ll give you a promotion and a silver-plated watch.” They take it, considering the promotion and the watch to be gravy.

But when you offer that same narrative to ambitious, passionate, and talented developers, they leave. They grow bored, and bored people quit. They refuse to tolerate that organizational stupidity, and they evaporate. They look for “up or out,” and, realizing that “out” is much quicker and more appealing, they change their narrative on their own to “So long, suckers!”

You need to offer your talented developers a more appealing narrative if you want them to stay. Make sure that you take them aside and reaffirm that narrative to them frequently. And make sure the narrative is deterministic in that their own actions allow them to move toward one of the goals. Here are some narratives that might keep developers around:

  • “If you implement feature X on or ahead of schedule, we will promote you.”
  • “With the work that we’re giving you over the next few months, you’re going to become the foremost NoSQL expert in our organization.”
  • “We recognize that you have a lot of respect for Bob’s Ruby work, so we’re putting you on a project with him to serve as your mentor so that you can learn from him and get to his level.”
  • “We’re building an accounting package that’s critical to our business, and you are going to be solely responsible for the security and logging portions of it.”
  • “If your work on project Y keeps going well, we’re going to allow you to choose your next assignment based on which language you’re most interested in using/learning.”

Notice that these narratives all appeal to autonomy/mastery/purpose in various ways. Rather than dangling financial or power incentives in front of the developers, the incentives are all things like career advancement/recognition, increased autonomy, opportunities to learn and practice new things, the feeling of satisfaction you get from knowing that your work matters, etc.

And once you’ve given them some narratives, ask them what they want their own to be. In other words, “we’ll give you more responsibility for doing a good job” is a good narrative, but it may not be the one that the developer in question envisions. It may not always be possible to give the person exactly what he or she wants, but at least knowing what it is may lead to attractive compromises or alternate ideas. A new team member who says, “I want to be the department’s principal architect” may have his head in the clouds a bit, but you might be able to find a small, one-man project and say, “start by architecting this and we’ll take it from there.”

At any point, both you and the developers on your team should know their narratives. This ensures that they aren’t just periodic, feel-good measures–Michael’s “diving saves”–but constant points of job satisfaction and purpose. The developers’ employment is a constant journey that’s going somewhere, rather than a Sisyphean situation where they’re running out the clock until retirement. With this approach, you might even find that you can coax a narrative out of some “residue” employees and reignite some interest and productivity. Or perhaps defining a narrative will lead you both to realize that they are “residue” because they’ve been miscast in the first place and there are more suitable things than programming they could be doing.

Conclusion

The narratives that you define may not be perfect, but they’ll at least be a start. Don’t omit them, don’t let them atrophy and, whatever you do, don’t let an inverted meritocracy–the “residue”–interfere with the narrative of a rising star or top performer. That will catapult your group into a vicious feedback loop. Work on the narratives with the developers and refine them over the course of time. Get feedback on how the narratives are progressing and update them as needed.

Alex thinks that departure from organizations is inevitable, and that may be true, but I don’t know that I fully agree. I think that as long as talented employees have a narrative and some aspirations, their value apex need not level off. This is especially true at, say, consulting firms where new domains and ad-hoc organization models are the norm rather than the exception. But what I would take from Alex’s post is the perhaps radical idea that it is okay if the talented developer narrative doesn’t necessarily involve the company in five or ten years. That’s fine. It allows for replacement planning and general, mutual growth. Whatever the narrative may be, mark progress toward it, refine it, and make sure that your developers are working with and toward autonomy, mastery, and purpose.

No Fields Found.

Jul
11

By

Developer Process Gerrymandering

Category: The Life of a Programmer Tags: , | 4 Comments

Gaming the System

As projects get a little behind schedule or perhaps a little contentious in terms of scope, I’d say it’s common for people to start taking steps to indemnify themselves against blame. I’d also say that this is fairly natural. There’s nothing more likely to attract wagging management fingers than a project behind schedule and/or over budget, so it makes sense to do a little rehearsing as to what one is going to tell them. Furthermore, it’s somewhat natural to try to showcase that one’s own efforts were a general positive even against a negative backdrop.

This is reminiscent of the “plus/minus” tracking in sports such as basketball, where it’s possible for a team to outscore its opponent when player X is on the floor, but still lose. The most common reason for this is that player X is a star, but the rest of the team is so bad that even his or her stellar play cannot overcome the shortcomings of the rest of the team. Long story short is that every developer wants to be that player X on a winning team if possible. But, absent that, being player X on a losing team is fine, so long as management and peers notice.

In such a situation (potentially “losing effort”), I’ve recently observed the following developer behaviors:

  1. Obvious (to me, if not management) estimate sandbagging.
  2. Wheedling, begging, berating, manipulating and generally badgering defect reporters into retracting reported defects.
  3. Refusal to fix obvious and embarrassing problems with the software because said solutions weren’t “in the requirements statement”.

These behaviors aren’t particularly difficult to understand in the context of seeking to create the impression of a better “plus/minus” score for a developer. Consider the following explanations:

  1. Estimate sandbagging allows a developer to finish way ahead of schedule, creating a heroic sort of aura for a time
  2. Improving code quality is one way to reduce the number of defects reported against one’s code. Browbeating people into not reporting the defects is another way to do this and more expedient in the short term, in a sense.
  3. Refusing to address something on the grounds that “I never had a requirement for this” is a nice two-fer; it creates the impression that not addressing the shortcoming was a conscious decision rather than an oversight and it also identifies a different blame target (whoever is responsible for requirements).

This frank categorization may induce a chuckle or two, but I doubt I’m breaking any drastically new ground for people who work in programming shops where this kind of thing probably occurs with frequency somewhere between “here and there” and “constant”. I do think there is value in coining some terms around it though so that we can examine the issues it causes, its own root cause, and some potential solutions.

Elbridge Gerry, An Inspiration

In the early 1800’s Massachusetts had a governor by the name of Elbridge Gerry. A Democratic-Republican by party affiliation, Gerry saw trouble in the tea leaves for his own party in the state senate and decided to do something about it. You might think that he decided to take his case to the people, touting the benefits of his party and convincing them to vote for Democratic-Republicans. You’d be wrong though.

Convincing the public to like you is hard and time consuming and there’s no guarantee of success. Gerry had a more expedient idea. He simply signed a law that radically altered the voting districts in his state in such a way that a majority would result in the state senate without actually having the voters to support it. One of the newly created districts was so contorted and weirdly shaped that political opponents said it looked like salamander. Gerry’s Salamander became the term “Gerrymander” and the phrase stuck around. Today it retains its definition as a sleazy but legal practice in which politicians stack the deck in favor of their own party as they’re dealing out political cards to the voters.

Getting back to the topic of software development, the behaviors I mentioned in the first section and their attendant explanations are all examples of a behavior similar in intentions to gerrymandering. In our world, the ideal for a software group is obviously to write good code, provide good estimates, and deliver quality software on time for a good value. As an individual the ideal is to have a good “plus/minus” within the group toward that same end — to write better than average code, delivered more quickly than average for less money than average. That’s all noble, but it’s also hard. And so some developers prefer to emulate Gerry and change the rules of the game (distort time, change definitions, employ technicalities) rather than playing it well. In the “plus/minus” world, this is like lobbying to play against the third string of the other team to pad one’s statistics.

Eliminate the “Plus/Minus”, Eliminate the Gerrymandering

So, if you’re managing a development team, how do you eliminate the counter-productive gerrymandering behavior? Eliminate the individual “plus/minus” ratio mentality. Imagine that you have a software project and you tell everyone working on it that their performance and pay/bonus were going to be dictated by the ranking on a scale of 1-10 that users gave the software in a survey (encompassing usability, quantity/quality of features, etc). Let’s think about what happens to the three behaviors at the start of the post:

  1. Estimate sandbagging is pointless because coming in ahead of a schedule that you’ve defined has no bearing on your bonus and performance review.
  2. Badgering defect reporters makes no sense because the goal is fewer defects in the user’s hands rather than fewer defects reported against the individual.
  3. Refusal to fix problems (finger pointing) is also silly because whining that someone else is at fault is cold comfort in the face of a bad review and no bonus

Now, I don’t necessarily believe that there will be entirely smooth sailing in this or any other construct. But, eliminating the focus on individual incentives and performance has some nice side effects including the one on which we’re focused: removing the tendency to try to game the system without adding value. (Another nice side effect is that you tend to downplay the unfortunate “rockstar” designation that panders to megalomaniacal personality types who tend drastically to overestimate their own irreplaceability and scare others into the same).

But How to Measure Individual Performance

Still, the question of individual performance evaluation remains. It’s all well and good to reward or punish the whole team as a unit, but there are HR org charts, career maps and other individual concerns to think of. So how do you address this while still avoiding the gerrymandering?

It’s all about the code and source control. There are a lot of metrics out there for evaluating developer productivity from the obtuse (counting lines of code generated or altered) to the anecdotal (does this guy seem to do a good job), but I’m not really aware of one that captures the truth as well as looking at someone’s changesets/commits in the context of a code base. So, what if you had some impartial and extremely knowledgeable party read through a code base, looking at its architecture and tendencies, and then read through the change sets to give each developer some kind of rating or at least partial rating.

It seems almost as if you could create a cottage industry out of this and offer it as a consulting service. This way, the consultant is truly removed from any office politics and can focus entirely on the code. If one did this as a craft, I’d imagine that the addition of data points to the general corpus would cause the process to become quite refined over the course of time and probably generate some relatively objective, interesting and meaningful metrics.

Of course, this wouldn’t be perfect. It’s only as good as the evaluator is sharp, and it has the shortcoming of not capturing “intangibles” on a project — maybe someone wrote little code but spent a lot of time helping other people with source control and other technical issues, for instance. Code and changesets are mute witnesses to the story of the project, but they don’t witness everything.

Still, I think the notion is an interesting enough one that it bears exploring. Developer (or any team member) posturing and gerrymandering are counter-productive activities that put the developer’s interests above the projects but, more problematically, misalign those two sets of interests. The combination of team effort and metric individual evaluation could help prevent that sort of thing.

Jun
28

By

Abstractions are Important 4 – Personal Interactions

Category: Abstractions Tags: , , |

An Announcement

I’ve enlisted the help of freelance editor Amanda Muledy to copy edit this blog. The content will not be altered in terms of substance; rather, she’ll be correcting grammar and spelling mistakes and the like. She’ll even know whether or not the semi-colon I just used was some kind of faux pas.

She has started on my oldest posts and will be working her way toward the newest ones and we’ll eventually probably work to a state where I send them to her prior to publication. So, if you see mild edits to content here and there, that’s the reason for it. If you’re interested in her services or more information along these lines, she has contact info on her blog.

And now back to your regularly schedule post…

Beyond the Code

In the second post in this series, I described in detail an abstraction that had nothing to do with programming: the pizza parlor with delivery service. Today I’d like to revisit abstractions in the real world, but not focus on static institutions. Instead, I’d like to talk about the ad-hoc abstractions that we form in one another’s lives as we work collaboratively.

Huh?

When we work with other people, what we generally do is offer abstractions to one another for the purpose of getting things done more efficiently. Have you ever said something like “you prep the chicken, and I’ll start the grill”? This is an ad-hoc, process-based abstraction. What does prepping the chicken entail? Maybe a dry drub, chopping it into chunks, etc. But, the interesting thing is that the person starting the grill doesn’t care at all, as long as it’s taken care of. Likewise with the starting of the grill. Is it a charcoal grill? What kind of charcoal to use? None of that matters to the person prepping the chicken.

This is an example of a good process abstraction, and these types of abstractions are the basis for the concept of effective teamwork. They allow people to work in parallel and to be more efficient than they would be working on their own, assuming that the tasks are not temporally dependent. Sure, one person could prep the chicken and also start the grill, but two people doing it simultaneously will go faster. It would be a different story if the two tasks were interdependent or mutually exclusive. “Hey, you clean the bottom of the grill while I start it” wouldn’t work very well.

So efficient teamwork requires tasks that are relatively orthogonal and participants providing abstractions to one another. (It is certainly possible to have abstraction in sequential tasks, but that’s not nearly as interesting). Without orthogonality, the team members would step on one another’s toes while completing their tasks, and without abstractions there wouldn’t be any productivity boost.

For Example

Last week, I wanted to order a server and confirm that our MSDN licensing covered the server software that I wanted to install. Management paired me with someone whose job it was to worry about these things so that I could focus on my job of being a programmer. Things started to go a bit off the rail at this point, though. I got an email about computer wholesalers and the names of salespeople at these wholesalers, distrust of Microsoft and its licensing and any number of other things that weren’t of interest to me — things that should have been abstracted away from me. My goal was simple — get a computer and install server software on that computer.

I responded to this email by saying that I appreciated all of the information, but I didn’t really want to worry about all that stuff. I just wanted the computer. I did ask if it made sense for me to call one of these wholesaler employees directly, which I would have been happy to do, since this would have provided me with an actual abstraction. The response to this email was another torrent of information that was tangentially related at best (overseas latency issues, server hard drive configuration snags, etc) , but no servers ordered and no contact information provided. This went on for several emails more emails in the same fashion (the situation is now on hold for other reasons, so there is no resolution, either).

What I had was a consistent failure to provide an abstraction, and thus I had no actual help with anything. If I have to stop working at my assignment (being a programmer) to deal with an IT assignment (ordering a server) than the person whose responsibility is the latter is providing no abstraction and thus, functionally useless to me. In this example, the time I spend reading and responding to emails could better have been spent calling Microsoft and/or the wholesaler directly. Either way, I couldn’t focus on my job, so at least something productive could have come out of it.

Why Does This Happen?

As you read through this, you’re probably both asking yourself why anyone would do this and also thinking of people who have provided a bad abstraction to you in the past. That is to say, it’s often incomprehensible to a problem solver not really mired in office politics, but it’s also a common behavior. I posit that there are three reasons someone will provide a bad ad-hoc personal abstraction:

  1. Passive-aggressive desire to sabotage an initiative
  2. Earnest over-exuberance or over-inquisitiveness
  3. Simple incompetence

The last of these is the simplest and least interesting. If someone is simply incompetent at a task, they will serve poorly as an abstraction. If you were to ask me to ask someone for the time of day in Mandarin, I would respond with either “how do you ask someone the time of day in Mandarin?” or “will you do it for me?”. I am simply ill-suited for this task since I don’t speak that language.

The second of these is what results from people who are nervous about responsibility or perhaps overly excited about it, but are willing to put in a good faith effort. Perhaps you ask me to cook you an omelette, and I respond by saying, “how much pepper would you like in it; do you care if the eggs are whisked for 20 seconds or do you want 30; these are those brown eggs – is that alright; do you want me to use PAM or oil; etc?” I mean well, but I’m defeating the purpose of what you asked of me. By bogging you down in these details, I’m forcing you to spend more time helping me than was your intent and I might well be annoying you. This sort of behavior might result from me really wanting you to enjoy the omelette or me being afraid that you’ll yell at me if I mess something up, but there’s no ill intent.

The first item in the list is probably the most interesting from a realpolitik perspective in a collaborative setting, though it isn’t because it’s a sophisticated technique. Children learn to do this at an early age and employ it without any subtlety. Ask a child to clean his room and, after some initial whining and the gears start turning:

Child: Where does this go?
Parent: In the drawer over there.
Child: But I can’t reach!
Parent: Get a stool.
Child: Where are the stools?
Parent: There’s one on the other side of your bed.
Child: It’s heavy!

The goal here is to feign simple incompetence to get out of the task, which is a rather natural thing for a child to do since parent-child relationship often doubles as teacher-student. The child banks on the parent becoming so exasperated that he or she finally exclaims “Oh, for the love of… I’ll just do it!” This gets interesting in adult interaction dynamics since it necessarily involves a sacrifice of perceived status. Whoever does this must be willing to accept a role of subordinate status and often to move down the path toward learned helplessness. If you work with me, and I respond to all of your requests by saying that I don’t know how to do things, it’s only a matter of time before you and others begin to question my competence and value in my role.

As a result, the best way to get out of work (or sabotage an initiative in general if the motivation is something other than laziness) is not to offer oneself sacrificially as a buffoon, but instead to exaggerate the difficulty of the task with which one is presented:

Oh, sure, I’ll make you an omelette, but there’s really a lot to think about here. You have to consider what kind of pan you want, what size and color of egg, the quanity of pepper involved… I mean, there are a lot of variables here. Don’t worry, though — I’ll make some inquiries as to what the best kind of ingredients are and get back to you tomorrow.

Notice that it’s very similar to the over-exuberant explanation, but with a bit more of an air of authority. It also subtly implies that the earliest possible completion of the task renders it not worth doing — who wants to wait 24 hours for their breakfast? This shift from excited questioning to authoritative is often a giveaway, but sometimes these motivations may even come off as identical. They’ll certainly be similar and it would be a shame to accuse someone of some sort of malfeasance when the explanation is benign. It’s genuinely hard to distinguish items (1) and (2) in someone else in a vacuum without track records and understanding of possible ulterior motives.

Avoiding All Of It

And, most diplomatic people will avoid this sort of accusation because it is rude and assuming. They’ll act on the surface as though you’re simply excitable. But, here’s the catch — they’ll assume that you’re passive-aggressive if it happens too often and if it’s a regular source of impediment. So really, this type of behavior, even when benign has at best a short term benefit with a long term cost. People will probably assume that you deliberately make yourself an obstacle, but even if you’re fortunate enough to avoid that, they will assume that you are incompetent and/or annoying.

Do you really want any of those labels? I don’t, myself. So, I’d stress that you make sure to be a good abstraction for others when asked. Make sure that your work makes things simpler for other people and removes obstacles from their path, rather than adding them. And, if you really are ill-suited for a task, explain why and try to think of something you can offer instead to be helpful. It really does parallel code in a lot of ways — you should hide complexity from others and present them with simple but meaningful choices.

Good abstractions in code, good abstractions in life — a good way to live.

Mar
23

By

Are Unit Tests Worth It?

Category: The Life of a Programmer Tags: , , | 5 Comments

The Unit Test Value Proposition

I gave a presentation yesterday on integrating unit tests into a build. (If anyone is interested in seeing it, feel free to leave a comment, and I’ll post relevant slides to slideshare or perhaps make the power point available for download). This covered the nuts and bolts of how I had added test running to the build machine as well as how to verify that a delivery wouldn’t cause unit test failures and thus break the build. For background, I presented some statistics about unit testing and the motivations for a test-guarded integration scheme.

One of the questions that came up during Q&A was from a bright woman in the audience who asked what percentage of development time was spent writing unit tests for an experienced test writer and for a novice writer. My response to this was that it would be somewhat slower going at first, but that an experienced TDD developer was just as fast doing both as a non-testing developer in the short term and faster in the long term (less debugging and defect fixing). From my own personal experience, this is the case.

She then asked a follow up question about what kind of reduction in defects it brought, and I saw exactly what she was driving at. This is why I mentioned that she is an intelligent woman. She was looking for a snap-calculation as to whether or not this was a good proposition and worth adopting. She wanted to know exactly how many defects would be avoided by x “extra” days of effort. If 5 days of testing saved 6 days of fixing defects, this would be worth her time. Otherwise, it wouldn’t.

An Understandable but Misguided Assessment

In the flow of my presentation (which wasn’t primarily about the merits of unit testing, but rather how not to break the build), I missed an opportunity to make a valuable point. I wasn’t pressing and trying to convince people to test if they didn’t want to. I was just trying to show people how to run the tests that did exist so as not to break the build.

Let’s consider what’s really being asked here. She’s driving at an underlying narrative roughly as follows (picking arbitrary percentages):

My normal process is to develop software that is 80% correct and 20% incorrect and declare it to be done. The 80% of satisfied requirements are my development, and the 20% of missed requirements/regressions/problems is part of a QA phase. Let’s say that I spend a month getting to this 80/20 split and then 2 weeks getting the rest up to snuff, for a total of 6 weeks of effort. If I can add unit testing and deliver a 100/0 split, but only after 7 weeks then the unit testing isn’t worthwhile, but if I can get the 100/0 split in under 6 weeks, then this is something that I should do.

Perfectly logical, right?

Well, almost. The part not factored in here is that declaring software to be done when it’s 80% right is not accurate. It isn’t done. It’s 80% done and 20% defective. But, it’s being represented as 100% done to external stakeholders, and then tossed over the fence to QA with the rider that “this is ‘done’, but it’s not done-done. And now, it’s your job to help me finish my work.”

So, there’s a hidden cost here. It isn’t the straightforward value proposition that can be so easily calculated. It isn’t just our time as developers — we’re now roping external stakeholders into helping us finish by telling them that we’ve completed our work, and that they should use the product as if it were reliable when it isn’t. This isn’t like submitting a book to an editor and having them perform quality assurance on it. In that scenario, the editor’s job is to find typos and your job is to nail down the content. In the development/QA work, your job is to ensure that your classes (units) do what you think they should, and it’s QA’s job to find integration problems, instances of misunderstood requirements, and other user-test type things. It’s not QA’s job to discover an unhandled exception where you didn’t check a method parameter for null — that’s your job. And, if you have problems like that in 20% of your code, you’re wasting at least two people’s time for the price of one.

Efficiency: Making More Mistakes in Less Time

Putting a number to this in terms of “if x is greater than y, then unit testing is a good idea” is murkier than it seems because of the waste of others’ time. It gets murkier still when concepts like technical debt and stakeholder trust of developers are factored in. Tested code tends to be a source of less technical debt given that it’s usually more modular, maintainable, flexible, etc. Tested code tends to inspire more confidence in collaborators as, you may run a little behind schedule here and there, but when things are delivered, they work.

On the flipside of that, you get into the proverbial software death march, particularly in less agile shops. Some drop-dead date is imposed for feature complete, and you frantically write duct-tape software up until that date, and then chuck whatever code grenade you’re holding over the QA wall and hope the shrapnel doesn’t blow back too hard on you. The actual quality of the software is a complete mystery and it may not be remotely close to shippable. It almost certainly won’t be something you’re proud to be associated with.

One of my favorite lines in one of my favorite shows, The Simpsons, comes from the Homer character. In an episode, he randomly decides to change his name to Max Power and assume a more go-getter kind of identity. At one point, he tells his children, “there are three ways of doing things: the right way, the wrong way, and the Max Power way.” Bart responds by saying, “Isn’t that just the wrong way?” to which Homer (Max Power) replies, “yes, but faster!”

That’s a much better description of the “value” proposition here. It’s akin to being a student and saying “It’s much more efficient to get grades of C and D because I can put in 10 hours per week of effort to do that, versus 40 hours per week to get As.” In a narrow sense that’s true, but in the broader sense of efficiency at being a good student, it’s a very unfortunate perspective.  The same kind of nuanced perspective holds in software development.  Sacrificing an objective, early-feedback quality mechanism such as unit tests in the interests of being more “efficient” just means that you’re making mistakes more efficiently.  And, getting more things wrong in the same amount of time is a process bug — not a feature.

So, for my money, the idea of making a calculation as to whether or not verifying your work is worthwhile misses the point.  Getting the software right is going to take you some amount of time X.  You have two options here.  The first option is to spend some fraction of X working and then claim to be finished when you’re not, at which point you’ll spend the other portion of the fraction “fixing” the fact that you didn’t finish.  The second option is to spend the full time X getting it right.

If you set a standard for yourself that you’re only going to deliver correct software, the timelines work themselves out.  If you have a development iteration that will take you 6 weeks to get right, and the business tells you that you only get 4, you can either deliver them “all” of what they want in 4 weeks with the caveat that it’s 33% defective, or you can say “well, I can’t do that for you, but if you pick this subset of features, I’ll deliver them flawlessly.”  Any management that would rather have the “complete” software with defect landmines littering 33% of the codebase than 2/3rds of the features done right needs to do some serious soul-searching.  It’s easy to sell excellent software with the most important 2/3rds of the features and the remaining third two weeks out.  It’s hard to sell crap at any point in time.

So, the real value proposition here boils down only to “do I want to be adept at writing unreliable software or do I want to be adept at writing software that inspires trust?”

« Older Posts
Newer Posts »