The “Synthesize the Experts” Anti-Pattern
I have a relatively uncomplicated thought for this Wednesday and, as such, it should be a reasonably sized post. I’d like to address what I consider to be perhaps the most relatable and understandable anti-pattern that I’ve encountered in this field, and I call it the “Synthesize the Experts” anti-pattern. I’m able to empathize so intensely with its commission because (1) I’ve done it and (2) the mentality that drives it is the polar opposite of the mentality that results in the Expert Beginner. It arises from seeking to better yourself in a a vacuum of technical mentorship.
Let’s say that you hire on somewhere to write software for a small business and you’re pretty much a one-person show. From a skill development perspective, the worst thing you could possibly do is make it up as you go and assume that what you’re doing makes sense. In other words, the fact that you’re the least bad at software in your company doesn’t mean that you’re good at it or that you’re really delivering value to the business; it only means that it so happens that no one around can do a better job. Regardless of your skill level, developing software in a vacuum is going to hurt you because you’re not getting input and feedback and you’re learning only at the pace of evolutionary trial and error. If you assume that reaching some sort of equilibrium in this sense is mastery, you’re only compounding the problem.
But what if you’re aware of this limitation and you throw open the floodgates for external input? In other words, you’re alone and you know that better ways to do it exist, so you seek them out to the best of your ability. You read blog posts, follow prominent developers, watch videos from conferences, take online learning courses, etc. Absent a localized mentor or even partner, you seek unidirectional wisdom from outside sources. Certainly this is better than concluding that you’ve nailed it on your own, but there’s another problem brewing here.
Specifically, the risk you’re running is that you may be trying to synthesize from too many disparate and perhaps contradictory sources. Most of us go through secondary and primary education with a constant stream of textbooks that are more or less cut and dry matters of fact and cannon. The more Algebra and Geometry textbooks you read and complete exercises for, the better you’re going to get at Algebra and Geometry. But when you apply this same approach to lessons gleaned from the movers and shakers in the technosphere, things start to get weird, quickly.
To put a specific hypothetical to it, imagine you’ve just read an impassioned treatise from one person on why it’s a complete violation of OOP to have “property bag” classes (classes that have only accessors and mutators for fields and no substantive methods). That seems to make sense, but you’re having trouble reconciling it with a staunch proponent of layered architecture who says that you should only communicate between layers using interfaces and/or “DTO” classes, which are property bags. And, hey, don’t web services do most of their stuff with property bags… or something? Ugh…
But why ugh? If you’re trying to synthesize these opinions, you’re probably saying, “ugh” because you’ve figured out that your feeble brain can’t reconcile these concepts. They seem contradictory to you, but the experts must have them figured out and you’re just failing to incorporate them all elegantly. So what do you do? Well, perhaps you define some elaborate way to get all of your checked in classes to have data and behavior and, at runtime to communicate, they use an elaborate reflection scheme to build and emit, on the fly, property bag DTO classes for communication between layers and to satisfy any web service consumers. Maybe they do this by writing a text file to disk, invoking a compiler on it and calling into the resulting compiled product at runtime. Proud of this solution, you take to a Q&A forum to show how you’ve reconciled all of this in the most elegant way imaginable, and you’re met with virtual heckling and shaming from resident experts. Ugh… You failed again to make everyone happy, and in the process you’ve written code that seems to violate all common sense.
The problem here is the underlying assumption that you can treat any industry software developer’s opinion on architecture, even those of widely respected developers, as canonical. Statements like, “you shouldn’t have property bags” or “application layers should only communicate with property bags” aren’t the Pythagorean Theorem; they’re more comparable to accomplished artists saying, “right triangles suck and you shouldn’t use them when painting still lifes.” Imagine trying to learn to be a great artist by rounding up some of the most opinionated, passionate, and disruptive artists in the industry and trying to mash all of their “how to make great, provocative art” advice into one giant chimera of an approach. It’d be a disaster.
I’m not attempting a fatalistic/relativistic “no approach is better than any other” argument. Rather I’m saying that there is healthy disagreement among those considered experts in the industry about the finer points of how best to approach creating software. Trying to synthesize their opinionated guidance on how to approach software into one, single approach will tie you in knots because they are not operating from a place of universal agreement. So when you read what they’re saying and ask their advice, don’t read for Gospel truth. Read instead to cherry pick ideas that make sense to you, resonate with you, and advance your understanding.
So don’t try to synthesize the experts and extract a meeting that is the sum of all of the parts. You’re not a vacuum cleaner trying to suck every last particle off of the carpet; you’re a shopper at a bazaar, collecting ideas that will complement one another and look good, put together and arranged, when you get home. But whatever you do, don’t stop shopping — everything goes to Expert-Beginner land when you say, “nah, I’m good with what’s in my house.”
Great post.
Most of us are likely guilty of this at some point in our career 🙂
Trust your instincts, be pragmatic and remember one size does not fit all!
Glad you liked, and a nice, succinct summary there in the last line of the comment.
Learnt this very much the hard way when starting my career in software development. Only then there was only a fledging Internet, no world wide web, and Knut was your bible.
Nice to see someone sharing this very important nugget of wisdom!
Yes, I too definitely remember when most knowledge for self-learning programmers came from man pages, a handful of programming books or giant programmer user manuals. Fewer sources of potentially conflicting information, but also fewer avenues for vetting and corroboration.
It’s amazing how your “one-man-army” metaphor describes my current situation perfectly. What I believe today is this:
1) Software Engineering has some more or less well established first principles (modularity, hierarchical decomposition, etc);
2) Patterns and such are ways to accomplish these first principles;
3) There are different ways to accomplish any given first principle, and this can lead to contradiction;
4) You can choose the solution you want.
5) You can bend a solution. You should not bend the first principle.
I think this is certainly an approach that gets you away from the relativism I alluded to in the post. Rightly or wrongly (in terms of what you pick), you will need to accept certain core principles as axiomatic or you’d always be starting from square one. It sounds like this is what you mean with (1), and I agree with the approach. It’s an interesting problem in software engineering to get good working ‘axioms’ in order to make progress toward solving problems but without getting too specific and dogmatic.
Yeah, one could say the axioms are “requirements” for a good process, while methodologies are “products” we devise to meet those. You can build different products to solve the save problems, and they’re fine as long as they meet the requirements. A more concise formulation would be “[right] axioms are rigid, methodologies are flexible”. Be purist with axioms, and pragmatic with methodologies.
Well, perhaps you define some elaborate way to get all of your checked in classes to have data and behavior and, at runtime to communicate, they use an elaborate reflection scheme to build and emit, on the fly, property bag DTO classes for communication between layers and to satisfy any web service consumers. Interestingly enough you just described one of the best practices in software development. This is holy grail-SOA in a nut shell. You refuse to pass mutable state even outside of a single class. Well defined SOA in many ways is Functional Programmming (FP) as opposed to OOP.… Read more »
Yeah, mainly what I was driving at is the “you can’t win” frustration. I’ve seen people who are making a good faith effort to learn as much as possible go off and perform good faith implementations of what they thought was being recommended only to have the recommenders say “no, you’ve got it all wrong.” I was having a little fun with it by suggesting a standard OOP language app that would write source code to disk, compile it, and then dial into the result for functionality. What was in my head was Console.WriteLine(“public class SomeDTO : BaseDTO”); Console.WriteLine(“{“); etc… Read more »
Great Post.
Thought you might like it 🙂
I find an important antidote to the problem is finding the motivation behind the advice. That way you can better infer when and how to combine conflicting advices.
I think that’s sound advice across the board. I’ve been dealing with a parallel line of thought lately in trying to steer a group away from “requirements” and toward business value. “We need a component that takes XML and turns it into…. etc” And my response is, “let’s back up — what is everyone actually trying to do here?”
Ah yes. It’s the ‘such that’ from scrum stories. I find lately that this is more valuable when you don’t own the product (ie. no startup, but external customers). If you don’t know why a customer wants a certain feature it can be very hard to implement.
I’ve seen a lot of instances in the wild where leaving off “such that” leads to large scale implementations of crap no one actually wants or uses.
This fits fine in the “axioms vs methodologies” speculation in a comment I posted. I believe once this “clicks” into a developer’s mind (knowing that the real value of any methodology is in the underlying motivations), a lot of angst is eliminated.
A bit off topic, but what is your opinion on property bag objects vs function only objects?
A recent foray into entity framework code first just yesterday led me to think about that question. I guess when I’m dealing with generated bags of objects I’m content to leave them alone. However when I’m doing code first I’m highly tempted to write my business logic class and hit the big ol persist button. Then I feel moderately uneasy about it (with new things I often do) and start questioning myself.
I might write a post about this, but the short answer to the question is that I don’t have much of an opinion one way or the other. I’ve always been leery of the “that’s not true OOP” argument because the easy response is, “who cares?” I’m very much not prescriptive when it comes to how to tackle a specific problem. I mean, I can’t see your code base and I don’t know what decisions you’re facing, so it’s hard to pass too much judgement. In my experience, a combination of good abstraction, dependency management and separation of concerns tends… Read more »