DaedTech

Stories about Software

By

The Whiteboard Interview: Adulthood Deferred

I haven’t traveled this week (at least, not for work).  As a result of that, I’ve sat at home, where I tend to have somewhat higher social media consumption.  I therefore couldn’t help but see this post about “confessing coding sins.”

Twitter has, apparently, overflowed with established software developers ‘confessing’ that they would fail Gigantech Inc’s whiteboard/trivia interviews.  I’d like to go on record to point out that I ranted about the foolishness of this practice long before DHH made doing so cool with this tweet.

Here we have legendary techie David Heinemeier Hansson confessing that the Silicon Valley Gigantechs of the world would fail him out of their phone screens.  His tweet offers a compelling symmetry.  After all, when a cranky Thomas Edison invented the ineffective fad known as the “job interview” (that we haven’t bothered to revisit in the last 100 years), his interview would have failed Albert Einstein.

So, when it comes to the humble job interview, we at least know that it’s consistent.  It fails at its only job just as miserably today as it did in the beginning.  All of the MegaTechs out there in The Valley (and emulators around the world) would have passed on hiring meteoric value-creator DHH, thus calling into question the ubiquitous and vacuous claim of every company out there that “we only hire the best and brightest.”

But let’s come back to DHH a little later.  First, to celebrate the coming spring, I want to talk about baseball.

Wins Above Replacement (WAR)

Even if you don’t enjoy the sport of baseball, you should at least appreciate it for its data.  Unlike many sports out there, baseball happens transactionally.  The pitcher throws a pitch, and then a bunch of easily recorded stuff happens before play stops and this all starts over.  Oh, and we’ve kept logs of this going back 150 years or so.  This property has given rise to an entire discipline of statistics called sabermetrics.  So even if you don’t like home runs and hot dogs, you can at least appreciate the Big Data.

Baseball has a fascinating stat known by industry nerds as “Wins above Replacement (WAR).”  I’ll quote them directly on the meaning.

WAR offers an estimate to answer the question, “If this player got injured and their team had to replace them with a freely available minor leaguer or a AAAA player from their bench, how much value would the team be losing?”

Let me parse out the baseball jargon and simplify.  It asks, “how much value (in wins) does this player provide compared to an unremarkable replacement?”  Modern baseball clubs wager hundreds of millions of dollars answering this question.  A player with WAR above 5 commands that kind of money whereas one with a negative WAR gets a pat on the butt and an imminently unremarkable minor league contract.

WAR ain’t perfect.  But it pretty reasonably approximates player financial value.

What does any of this have to do with the job interview or whiteboard coding algorithms?  Well, the job interview represents the business world’s ludicrous attempt to calculate VAR (value above replacement) of prospective hires.

The Quixotic Pursuit of Corporate VAR

In baseball, if you want to evaluate the relative value of Hank Aaron and Joe Average, you have quite a corpus of data at your disposal.  You can base assumptions about future performance on thorough, objective data about past performance of them doing their job.  Baseball teams evaluate whether or not to pay them, and how much, on this basis.

The corporate world, on the other hand, lacks access to much, if any past data.  So, it attempts to compute VAR by handing them a marker and telling them to go to town on bubble sort for an hour.  Of course, I understand that professional baseball teams risk far more money, thus calling for more of a due diligence budget.  Nevertheless, though, the methodology differences are stark.

But going even beyond that, how much VAR does JuggerTech Inc really risk when hiring employee number 128,332?  Let’s call it 10X.  After all, we frequently hear assertions that “ninjas” offer 10 times the “productivity,” probably at times from the same companies that claim to hire these 10X-ers for 1.05X the pay.  So, we’ll just go with that.

This means that Alice and Bob wait in the corporate lobby, ready to pseudo-code bubble sort on a whiteboard.  If they get it right and hire Alice, she’ll bring 10 times the awesome.  If they whiff and hire Bob, he’ll bring just the standard amount of awesome.  But, let’s say they hire them both and put them both on the same project.

Alice will make the company 10 times as much money as Bob.  Er, wait, probably not.

The Fuzzy Value of Programmer N+1

Here’s the thing.  If HumongoTech sold “programmer awesomeness” at “X awesomes per dollar,” then Alice makes them 10 times the money.  But, for argument’s sake, let’s assume that HumongoTech sells things that actually exist in the real world.

Alice and Bob hire on and get their badges on day one.  Then they get down to business, each working on a feature for some HumongoTech product.  Except Alice brings 10 times the awesome.

Now, they both work on a feature that makes it into this iteration of the module if product manager Connie gives it the go ahead.  Coin flip says she does, so Alice now has a 50% chance of offering 10 times the awesome and a 50% chance of offering no marginal awesome.  So, Alice is down to a still-impressive 5X.

Except, ruh roh.  VP Daniel will determine whether or not the module makes it into the next release.  Flip another coin and Alice reduces to a respectable 2.5X.  Now only the chance of not shipping that release stands in the way.  But, that’s an actual thing, so let’s shave Alice down to 1.25X.

And all of that comes before we figure out if that release will make or lose the company money.  In the latter case, we get to throw a negative sign in front of marginally more productive Alice, who has now used her awesome to help the company lose money with 1.25 times the efficiency of Bob.

Are you seeing the trouble here?  And are you starting to wrap your head around the preposterousness of HumongoTech figuring out and evaluating hiring VAR?  Baseball teams have 9 starters whose every action gets measured.  HumongoTech has more people than a small city with utterly unknowable contributions.

Making It Up as We Go

So against this backdrop, what do we do in the corporate world?  How do we hire?  Truthfully speaking, we ask people to write about themselves on a piece of paper, pick a handful based on what they write, and then pick randomly.  But we tell ourselves that we totally have a process.  We don’t pick randomly.  Instead, we channel Edison and say things like, “take this marker and draw me a doubly linked list with a blah, blah, blah” or “tell me why they make manhole covers round.”  Yep, that’ll tell us who will help make MassiveTech more profitable.

We have no idea what we’re doing with this stuff, and no idea how to measure the results of our decisions.  So, in the end, we just do what people have been doing since a guy made up the process out of thin air.

But we techies, unlike most other industries, focus almost exclusively on the “technical” (read: VAR-theater) part of the interview at the expense of the part where you have a conversation about mutual fit like two human beings should.  We obsess over the meritocracy delusion that is this style of interview.  And, I think I have a hypothesis as to why we do so.

Adult Education as Never-Ending Juniority

We dwell on meritocracy and have no idea how to solve the probably-chaos-math-hard problem of measuring individual value contributions in non-trivial organizations.  How to dispel this cognitive dissonance?  Re-create an easy, familiar meritocracy.  I can think of none better than school.

When you hit the entry level (assuming a CS degree), you’ve spent roughly 22 years doing “interviews.”  You get homework, take quizzes, and make presentations.  Then, one of your betters evaluates your work and gives you a grade.  Some answers allow for objective evaluation while others rest purely in the subjective eye of the teacher/professor.  This happens endlessly and the same thing happens as you progress from one institution to the next: quizzes (standardized tests), applications (resumes), essays (interview questions) and the like.

So what do you do, lacking any reasonable criteria whatsoever for evaluating candidate VAR, but with a driving need to impose the illusion of meritocracy?  You cast your senior team members as professors and keep the college dream alive right on through everyone’s career and to the edge of retirement.

The Office Political Ramifications of the Trivia Interview

Last fall, I borrowed a term from my upcoming book and wrote about it on my blog: journeyman idealist.  That subject covers a ton of ground, but I can summarize relatively quickly.  To move up in a company the “traditional” route, programmers have to run quite the gauntlet.  First they have to ace enough trivial interviews and peer posturing to earn titular distinction baubles like “tech lead,” “architect,” and “senior” and only then can they pierce the next veil and get into management.

With just about every other position in pyramid-shaped companies, you become a thing and then, a few years later, you become a thing-manager (e.g. sales, account management, finance, etc).  Only those of us in industries with intense penchants for stack ranking create a whole other obstacle course at the lowest possible rung of the company.  But I digress a bit. (If you’re interested, you can find a lot more of that in my book.)

Politically speaking, when you enter a company, you enter below both idealists (usually management) and journeyman idealists.  You can tell the journeyman idealist from his gleeful administration of what DHH calls “whiteboard algorithm hazing.”  They become professor and you student as they grade you.

And, once you enter the company, it’s not like you suddenly become another professor.  You remain the student whenever that professor is present.  And everyone knows it.

Sure, if you hang around long enough, they gradually extend an invite to the fraternity.  But this takes years.  And, if you leave, you start all over with new professors grading your entrance exam.

Trivia interviews don’t just undermine your dignity while they happen. They establish you as a perpetual student.

What’s the Alternative?

I have a simple solution.  Throw out the bathwater and don’t sweat the baby.  When talking to recruiters or whomever, state up front that you have a policy against trivia/algorithm/whiteboard style interviews.  You understand that BureaucraTech does all of its interviews that way, but you hope they’ll give you a call if they ever rethink their process.

But, I get it.  Easy for me to say.  I earn my living as a free agent and a content one at that.  Your desires and ambitions differ from mine, and you really want to head to The Valley and sign on with BureaucraTech.  They have great perks.  People you respect work there.  You’ll work with the cutting-edgiest stuff.

That’s fair.

But you still don’t have to agree to trivia interviews.  I’ve previously offered the advice to start some kind of cool open source project or startup and wait for them to buy you out.  Then you’ll enter the mix as an organizational player, rather than a perpetual student (pragmatist grunt).  But I recognize that’s a long play.

So here’s a thought instead.  Identify what we free agents think of as “a buyer” at the company that presides over software developers. Buyers have budget and can make money decisions… like hiring.  Follow that buyer and try to get to know her.  Understand her group, her department, her challenges, and her gaps.  Then conceive of a way that she could fill one of those gaps with a technical solution and approach her to talk about it.

Remember all of the talk earlier about VAR?  Remember how BureaucraTech’s VAR is completely unknowable for a one in 100K software developer hire?  BureaucraTech’s VAR is unknowable.  Your buyer’s is not.  You can figure that out and make yourself valuable.

VAR, Revisited for Your Next Interview

Once you do that, you will have an entirely different interview experience.  You’ll have a sales conversation, exchanging realistic discussions of mutual fit and opportunity.  You’ll slip past the mundane hiring machinery and enter the organization mattering.

Every year I see discussions about how entirely broken the hiring process is, and every year I see companies more or less just keep doing the same, nonsensical things.  This will continue happening as long as the candidate and company both punt on a meaningful value assessment and start playing professor-student games.  VAR-theater drives these stupid interviews — not a general agreement that they work.  If you go back and read my post, you’ll see that a BureaucraTech employee even admits he doesn’t think the process works.

We won’t break this stalemate with undirected outrage or complaints about the process.  But we can break it if we refuse to participate.  And I can think of no better way to help your career and to refuse than by putting on your sales and delivery hat and figuring out how to deliver something to someone that matters.

As DHH himself said, he can’t do a bubble sort.  He isn’t DHH because he won some programming mastery competition over 8 million other competitors and was awarded an internship at BureaucraTech for his trouble.  He’s DHH because he built something that made countless people more productive.  He’s DHH because he wears a really high VAR on his sleeve.

So leave the junior varsity games to people who like to amuse each other with riddles.  Ace your next interview by refusing to have it in a context with penny ante table stakes.

Want more content like this?

Sign up for my mailing list, and get about 10 posts' worth of content, excerpted from my latest book as a PDF. (Don't worry about signing up again -- the list is smart enough to keep each email only once.)

newest oldest most voted
Notify of
Craig
Guest
Craig

I like it, but most engineers are not like DHH.
if I want to work for Apple, Yahoo, MSFT, Google, Amazon or any company really, how do I get that technical solution? I would need to invest a lot of time, and that’s only if I do realize what they need.

Erik Dietrich
Guest

You’re absolutely right that there’d be a significant time investment to do something like that. But I’ll offer up a few caveats/thoughts on the relative value of that effort. First of all, preparing for the algorithm/trivia interviews also requires a significant degree of effort. Even if you have a CS degree, it’ll mean spending a lot of time brushing up on your Algorithms coursework, “practicing” for the interview, and cramming as if for a midterm. If you never earned one, then you basically have to do a self taught, college level CS course. As an industry, we’ve just sort of… Read more »

Lliamander
Guest
Lliamander

You might be interested in this SE Radio interview with James Whittaker[0]. When he went to Google, he convinced the manager who hired him to let him not do any work for three months while he figured out where he would best be able to help the company (he eventually settled on security testing for Chrome). He goes into more depth about it in the interview, but basically to get into the position where you can negotiate more the terms of your employment, you need to: * build up a few specialties (that people need) * have a story for… Read more »

Lliamander
Guest
Lliamander

Interesting. I was going to comment on how my company has better ways of measuring technical ability, but upon rereading this article I think that would have been missing the point. As I understand it, your point is not just that the whiteboard bubble sort (WB-BS) test is a poor measure of developer ability (which many people argue) but that even if it were a good measure of technical chops, it’s not the same thing as measuring value to the company. Now, I would say that general technical ability isn’t a terrible indicator of value, even if it isn’t a… Read more »

Erik Dietrich
Guest

I think you’ve expressed this very well, because your comment, to my read, lines up exactly with what I’m trying to say. Any process wherein you spend a couple of hours with a stranger and make a hiring decision will have little correlation with value-added. Some things (like asking them to simulate doing the job) will be a little less bad than other things (peppering them with arcane trivia). But both will still be pretty bad (Google did some research and, if I recall correctly, found that the least bad stranger interview style was something like 30% effective at predicting… Read more »

Lliamander
Guest
Lliamander

> I think you’ve expressed this very well, because your comment, to my read, lines up exactly with what I’m trying to say. Thanks! I’m glad to hear we are on the same page. > Google did some research and, if I recall correctly, found that the least bad stranger interview style was something like 30% effective at predicting job performance. I’m not surprised, but I wonder how the methods they tested compare with how Matasano (a security consultancy) conducts their hiring process[0]. It seems like they have a pretty good track record. My team does something similar and (anecdotally)… Read more »

Andy Bailey
Guest
Andy Bailey

I recently answered a question in an interview by saying, “The wheel is out there on the Internet and all I have to do is use Google to find it.”. Saved 10 minutes in the interview and kept my dignity in the process. I refused the hire on the grounds that I am only going to work for a company that is interested in how I solve problems instead of being a parrot. I treat the interview process as a two way street and have no problems rejecting companies on the basis of that. The position I recently accepted involved… Read more »

Erik Dietrich
Guest

My hat’s off to you — love the approach and sticking to your guns.

craig
Guest
craig

I like the two way street. I also see it this way
I do hate when they leave 5 minutes for you to ask questions. I need more time. I want to interview the company for more than 5 minutes.
The problem is that a lot of companies don’t see it this way.
also, this is a valid point for senior developers I believe. the more options you have (and as a junior you have less options out there) , the more power the candidate have when the market is great.

Andy Bailey
Guest
Andy Bailey

I agree with you on having more time to ask questions but at the level I am being recruited at, for senior positions, I expect the interview(s) to be more of a conversation than a Q&A session. This means that I can show off my wares as part of a flowing narrative and they can do the same. It is on me to make sure I am ready for this and that I do my due diligence on the company that is courting me. Also everything has to be a lot more professional, I think that recruiting for “junior” positions… Read more »

craig
Guest
craig

senior positions = senior developer?

Andy Bailey
Guest
Andy Bailey

On paper yes but my duties will cover more than the title suggests. Architect for starters as well as other, non development related responsibilities

Dave McCraw
Guest
Dave McCraw

I love these articles, but I wish you would write one about what I should be doing to make good hires, instead of reinforcing how bad all the possible options I can think of would be 😉

Erik Dietrich
Guest

Added to the reader question backlog — stay tuned!

trackback

[…] means you have no buyer.  Your ‘buyer’ is a system.  And that system gives you whiteboard interviews and purchases by process and committee.  That’s no good.  You need to speak directly to […]

trackback

[…] means you have no buyer. Your 'buyer' is a system. And that system gives you whiteboard interviews and purchases by process and committee. That's no good. You need to speak directly to your […]