I remember something pretty random about my first job. In my cubicle, I had a large set of metal shelves that held my various and sundry programming texts. And, featured prominently on that shelf, I had an enormous blue binder.
Back then, I spent my days writing drivers and custom Linux kernel modules. I had to because we made use of a real time interface to write very precisely timed machine control code. As you might imagine, a custom Linux kernel in 2005 didn’t exactly come with a high production quality video walking users through the finer points. In fact, it came with only its version of the iconic Linux “man pages” for guidance. These I printed out and put into the aforementioned blue binder.
I cannot begin to tell you how much I studied this blue binder. I pored through it for wisdom and clues, feeling a sense of great satisfaction when I deciphered some cryptic function example. This sort of satisfaction defined a culture, in fact. You wore mastery of a difficult API as a badge of honor. And, on the flip side, failure to master an API represented a failure of yours.
Death of “Manual Culture”
What a difference a decade makes. No longer do we have battleship gray windows applications with dozens of menus and sub-menus, with hundreds of settings and thousands of “advanced settings”. No longer do we consider reading a gigantic blue documentation binder to be a use of time. And, more generally, no longer do we put the onus of navigating a learning curve on the user. Instead, we look to lure users by making things as easy as possible.
Ten years ago, a coarse expression described people’s take on this responsibility. I’ll offer the safe-for-work version: “RTM” or “Read the Manual.” Ten years later, we have seen the death of RTM culture. This applies to APIs and to user experiences in general.
The term “API” seems to present something of a Rorschach Test for software developers. Web developers think of APIs as REST endpoints and WSDLs. In contrast, desktop developers may think of APIs as “library code” written by developers from other organizations. Folks writing low level code like drivers? API provides the hooks into OS system calls (emphasis mine).
I understand this mild myopia that can happen, but the bigger picture matters. API offers a much more generic way of thinking. I personally like the definition from “tech terms.”
An API is a set of commands, functions, protocols, and objects that programmers can use to create software or interact with an external system. It provides developers with standard commands for performing common operations so they do not have to write the code from scratch.
An API defines how other programmers interact with your software. Thus each of the personas I mentioned have the right answer, but a small slice of it. As you write code, ask yourself about the user of that code. If that user interacts with your code via code of their own, you’re building an API. And before you ask, yes, that even applies to other developers in your company or even your team that use your code.
Consequently, understanding properties of good APIs carries vital importance for our industry. It makes the difference between others building on your work or avoiding it. And, let’s be honest — there’s a certain amount of professional pride at stake. So what makes APIs good? Let’s consider some characteristics of good APIs.
I’m aware that recently there’s been some brouhaha over criticism/trashing of open-source code and the Software Craftsmanship movement, and I’d like to continue my longstanding tradition of learning what I can from things without involving myself in any drama. Toward that end, I’d like to critique without criticizing, or, to put it another way, I’d like to couch some issues I have with an API as things that could be better rather than going on some kind of rant about what I don’t like. I’m hoping that I can turn the mild frustration I’m having as I use an API into being a better API writer myself. Perhaps you can pick up a few tidbits from this as well.
Recently I’ve been writing some code that consumes the Microsoft CRM API using its SDK. To start off with the good, I’d say that API is fairly powerful and flexible–it gives you access to the guts of what’s in the CRM system without making you write SQL or do any other specific persistence modeling. Another impressive facet is that it can handle the addition of arbitrary schema to the database without any recompiling of client code. This is done, in essence, by using generalized table-like objects in memory to model actual tables. Properties are handled as a collection of key-value string pairs, effectively implementing a dynamic-typing-like scheme. A final piece of niceness is that the API revolves around a service interface which makes it a lot nicer than many things for mocking when you test.
(If you so choose, you can use a code generator to hit your CRM install and spit out strongly-typed objects, but this is an extra feature about which it’s not trivial to find examples or information. It also spits out a file that’s a whopping 100,000+ lines, so caveat emptor.)
But that flexibility comes with a price, and that price is a rather cumbersome API. Here are a couple of places where I believe the API fails to some degree or another and where I think improvement would be possible, even as the flexible scheme and other good points are preserved.
The API that I’m working with resides around a series of calls with the following basic paradigm:
There is a service that accepts very generic requests as arguments and returns very generic responses as return values. The first component of this is nice (although it makes argument capture during mocking suck, but that’s hardly a serious gripe). You can execute any number of queries and presumably even create your own, provided they supply what the base wants and provided there isn’t some giant switch statement on the other side downcasting these queries (and I think there might be, based on the design).
The second component of the paradigm is, frankly, just awful. The “VeryAbstractBaseResponse” provides absolutely no useful functionality, so you have to downcast it to something actually useful in order to evaluate the response. This is not big-boy polymorphism, and it’s not even adequate basic design. In order to get polymorphism right, the base type would need to be loaded up with methods so that you never need to downcast and can still get all of the child functionality out of it. In order not to foist a bad design on clients, it has to dispense with needing to base your response type on the request type. Here’s what I mean:
varmeaningfulResponse=(GetSomeCookiesResponse)response;//Succeeds only because I have inappropriate knowledge of an implied coupling
varsecondmeaningfulResponse=(GetSomeBrowniesResponse)response;//Bombs out because apparently brownies and cookies aren't fungible
varthirdMeaningfulResponse=(GetSomeBrownieBytesResponse)response;//Succeeds oddly because apparently BrownieBytes are children of Coookies (or a conversion operator exists)
Think about what this means. In order to get a meaningful response, I need to keep track of the type of request that I created. Then I have to have a knowledge of what responses match what requests on their side, which is perhaps available in some horrible document somewhere or perhaps is only knowable through trial/error/guess. This system is a missed opportunity for polymorphism even as it pretends otherwise. Polymorphism means that I can ask for something and not care about exactly what kind of something I get back. This scheme perverts that–I ask for something, and not only must I care what kind of something it is, but I also have to guess wildly or fail with runtime exceptions.
Do consumers of your API a favor. Be as general as possible when accepting arguments and as specific as possible when returning them. Think of it this way to help remember. If you have a method that takes type “object” and can actually do something meaningful with it, that method is going to be incredibly useful to anyone and everyone. If you have a method that returns type object, it’s going to be pretty much useless to anyone who doesn’t know how the method works internally. Why? Because they’re going to have to have an inappropriate knowledge of your workings in order to know what this thing is they’re getting back, cast it appropriately, and use it.
Hierarchical Data Transfer Objects
With this interface, the return value polymorphism issues might be livable if not for the fact that these request/response objects are also reminiscent of the worst in OOP, reminiscent of late ’90s, nesting-happy Java. What does a response consist of? Well, a response has a collection of entities and a collection of results, so it’s not immediately clear which path to travel, but if you guess right and query for entities, you’re just getting started. You see, entities isn’t an enumeration–it’s something called “EntityCollection” which brings just about nothing to the table except to provide the name of the entities. Ha ha–you probably wanted the actual entities, you fool. Well, you have to call response.EntityCollection.Entities to get that. And that’s an enumeration, but buried under some custom type called DataCollection whose purpose isn’t immediately clear, but it does implemented IEnumerable so now you can get at your entity with Linq. So, this gets you an entity:
But you’re quite naieve if you thought that would get you anything useful. If you want to know about a property on the entity, you need to look in its Attributes collection which, in a bit of deja vu, is an AttributeCollection. Luckily, this one doesn’t make you pick through it because it inherits from this DataCollection thing, meaning that you can actually get a value by doing this:
D’oh! It’s an object. So after getting your value from the end of that train wreck, you still have to cast it.
Don’t force Law of Demeter violations on your clients. Provide us with a way of getting what we want without picking our way through some throwback, soul-crushing object hierarchy. Would you mail someone a package containing a series of Russian nesting dolls that, when you got to the middle, had only a key to a locker somewhere that the person would have to call and ask you about? The answer is, “only if you hated that person.” So please, don’t do this to consumers of your API. We’re human beings!
Unit Tests and Writing APIs
Not to bang the drum incessantly, but I can’t help but speculate that the authors of this API do not write tests, much less practice TDD. A lot of the pain that clients will endure is brought out immediately when you start trying to test against this API. The tests are quickly littered with casts, as operators, and setup ceremony for building objects with a nesting structure four or five references deep. If I’m writing code, that sort of pain in tests is a signal to me that my design is starting to suck and that I should think of my clients.
I’ll close by saying that it’s all well and good to remember the lessons of this post and probably dozens of others besides, but there’s no better way to evaluate your API than to become a client, which is best done with unit tests. On a micro-level, you’re dog-fooding when you do that. It then becomes easy to figure out if you’re inflicting pain on others, since you’ll be the first to feel it.
I am Erik Dietrich, founder of DaedTech LLC, programmer, architect, IT management consultant, author, and technologist.