My first job at Microsoft was providing developer support for the early Windows SDKs. To do my job well, I spent hours studying the Windows SDK documentation, the Windows source code, and writing sample applications. I then spent hours poring over customers’ (such as Lotus, WordPerfect, and Computer Associates) code helping them figure out what was not working.
This gave me a deep appreciation for API design early in my career. I saw clear examples of elegant APIs as well as horrific monstrosities. I once created a sample for selecting a font in GDI that I called “ReallyCreateFont” because both of the built-in Windows APIs (CreateFont() and CreateFontEx()) were basically broken.
The CreateFont case taught me first hand the pain resulting from exposing an API publicly, having customers use it in ways not expected, and then having to do unnatural acts to avoid breaking backwards compatibility.
I was supporting WordPerfect’s development of WordPerfect for Windows. This was sometime in early 1991 if I remember correctly. I had the WordPerfect engineer on the phone. He had given me access to the WordPerfect source code, but I couldn’t give him access to the Windows code. So we were each stepping through WordPerfect in the debugger, him in Utah and me in Bellevue. I could see what his code was doing and what Windows was doing, but he could only see disassembly for Windows.
The problem he was seeing had to do with font rendering. In the debugger we stepped into CreateFontEx(), which calls into CreateFont(), which calls into some internal GDI function, which calls into the device driver’s Escape() function (I can’t believe I actually remember all this crap like it was yesterday). Somewhere in this call stack I came across a block of code in Windows with comments that read something like
// This is a hack to deal with Adobe’s Type Manager. // ATM *injects code* into GDI replacing our code with their own. // In order to retain backwards compatability we detect // ATM’s attempt to do this, let it do it, and then trick it // into thinking it actually hacked us this way.
I am not making this up (although that comment is paraphrased from my memory).
It turns out that the way WordPerfect was using CreateFontEx() was reasonable, but pushing the envelope of what anyone had done before, effectively exposing a bug caused not by the original API design or implementation, but something the API provider had to do to ensure app compatibility because of something another 3rd party had done!
Holy-shit! Let me get this straight:
- A 3rd party app (Adobe Type Manager) used
CreateFontEx()in a way the API designer failed to anticipate. - The API designer needed to retain backwards compatibility so it had to resort to putting 3rd party app specific code in the API implementation.
- Another 3rd party comes along and the complexity of the ‘fix’ caused another bug to surface.
This is a pretty extreme example, given that what Adobe had done in injecting code into Windows was pretty over the top. But I think it really helps reinforce several principles I have on API design:
- Principle 1: All successful APIs will be abused in ways you cannot anticipate.
- Principle 2: You will support a successful API forever.
The only way to ensure an API does not get abused is to ensure it is not successful. But if it is successful you will have to support it forever.
Of course if you don’t really want your API to be successful then you can ignore these principles.
I recently came across an excellent presentation by Joshua Bloch titled “How to Design a Good API and Why it Matters.” Anyone designing APIs should take the time to read and understand it.
My only gripe is that he does not put enough emphasis on the point he makes in bullet #3 on slide 2:
- Public APIs are forever – one chance to get it right
I believe early stage ventures should avoid creating APIs at all (unless they are purely a dev tool or framework). They certainly should not try to “be a platform”. You can read my deep and insightful (ha!) thoughts on this here:
But if you are going to ignore my advice and start exposing APIs, take this to heart:
You will do them wrong. Callers will use them in ways you don’t expect. And you will waste huge amounts of energy supporting them in the future.
When exposing APIs be absolutely certain the value you get from doing so is worth it. If you do design APIs, Joshua’s guidelines are good.
Update 4/19/12 – I added “unless they are purely a dev tool or framework” above.
RSS - Posts
I don’t buy it Charlie. Too much time spent in the Machine has got you thinking of this problem the wrong way. With a good business contract, proper versioning, and a sane support horizon (hint: not MS’s) you can manage this.
I’d much rather see companies offering APIs that will be changed, broken, and replaced on a 6 month cadence than not have anything. I’d also much rather repair my code every few months than work with an API ossified by maintaining compatibility for a bunch of developers unwilling to keep up.
Yes it means I’ll need to patch apps regularly. So? It’s not 1993 anymore.
What he said.
So Kav, you are really agreeing with the thrust of the post:
Building APIs is hard. You have to take care in building them. And when they are successful you will have work to do.
My point is that way too many companies are jumping into building APIs without the deep thought that you suggest. They are also building APIs without really understanding what the use cases are.
My opinion lies somewhere between yours and Kav’s. “Use at your own risk” is a perfectly valid SLA for startups’ web APIs.
Facebook’s API changed *a lot* and they broke *everyone* with weekly frequency. But their core product was popular enough that developers went where the users were. They put up with Facebook’s bullshit because they had no choice. “Go fast, break things” is a manifesto at Facebook, and with good reason.
Investing deeply in backwards compatibility is a bad plan for startups who have an API as a secondary concern. If your business *is* an API, then you damn well better understand the complexities of building and maintaining one. But if you’re building something that just happens to have an API because it’s a kinda neat side effect of your system architecture, then you’re crazy to concern yourself with world-class support for it.
In short: If your business depends on your API and developer ecosystem, then invest in it. Otherwise, don’t.
Brandon,
In my mind, your perspective echos my other post on building platforms “Be either an App or a Platform, Not Both”:
http://ceklog.kindel.com/2011/08/24/be-either-an-app-or-a-platform-not-both/
Facebook was an “app” first and foremost. The vast majority of the value of Facebook (early on) was the END USER EXPERIENCE. APIs were secondary (secondary actually puts it too strongly).
Developers put up with API-bullshit from Facebook because the USER EXPERIENCE was so great.
True, I’d just argue that in order to really get all the use cases the APIs will need to spend at least some time in the wild. At MS there was often an impression that one could think one’s way into the perfect design. While experience and intelligence can get you half way real use is needed to finalize things and even then some things need to go a few years down the line.
I watched the .NET framework team try every trick in the book to escape the legacy they’d created and that was slowly drowning them.I’m a bit overzealous on the side of change only because I think this is an optimization problem that will cause issues at either extreme and we sit at an extreme that privileges static, lazy, bloated, abandon-ware over rapid productivity, developer speed, app stability, and modern cloud/web/mobile app development and patching behaviors.
I also believe that we can solve for difficulty of dealing with API change. I don’t think we can do the same for eternal back-compat.
Kav, I don’t think you represent the vast majority of people building projects, who will build it using a specific API, ship it, and never touch that project again. Most consumers of APIs might not have a developer on staff. Not only that, but they have their own set of priorities and replacing something that was working before is a pain. What that means is that if you are doing it successfully (building an API for money), you’ll have an enormous pressure to keep supporting old stuff.
Imagine you get WordPress installed on your server and you install a Twitter plug-in and ship to the client and he’s happy and dandy. What if Twitter announces it will be deprecating the API used on the WordPress plug-in? It doesn’t matter if it’s 6 months from now, 1-year or 5-years. It will be a pain.
BTW, great post Charlie.
I can’t imagine using a wordpress plug in that is closed source and has been abandoned for more than 6 months. Nor can I really imagine using anything else that fits that profile for that matter. Maybe a game, but even then…
The world of software I’m willing to interact with is simply not that static.I do understand the thrust of your argument though Marcelo and the simple answer there is applications developed to be abandoned can’t use APIs that follow my model. You’ll note however I said that it should be clearly called out that these kinds of changes will happen. Also I think this is generally a cop out. The cost in developer hours and ridiculous bugs on the part of the API developer and the application developer of dealing with the API by far outstretch the cost of a reflector or two per API to bring it in line with sensible use cases.
A lot of this thinking is a Pre-Web fossil. There are no GM floppies any more folks!
You aren’t taking into account the change in commitment to users/devs that has occurred over the past few years (which is to say the change in the definition of “professionalism”). How much legacy support is Google providing for the Wave and Buzz APIs? And for products still active? Facebook provides a whole 90 days heads-up (a good chunk of which can be burned just scheduling planning meetings with stakeholders). I admire your assumptions, because they indicate concerns that I share — I just don’t think they map well against current trends: everybody and their mother wants their new gizmo to be the one gizmo to rule them all — and hacking an API is one way to get traction, even if you have no intention of supporting it going forward.
I do not think you should talk about API design at Microsoft. In case you didn’t know… everyone hates MS APIs.
I change shit in LoseThos for giggles, imagining I have users. http://www.losethos.com
I, of course, had to go check this out. Terrence, it is AWESOME that you’ve built LoseThos and even more awesome that you change shit in it just for giggles. Love it!.
Charlie, this was insightful and brilliant. I just disagree. 😉 Here’s my response, entitled “Build API’s!” http://blog.jamiesonbecker.com/post/21357486997/build-apis
Or you can be like Facebook and make breaking changes to your API’s at will, completely breaking applications that rely on them, just because you can.
If you write your own products right away on top of your APIs, the issue “Be Either an App or a Platform, Not Both” becomes moot.
No doubt building your own “apps” that run against APIs you create is a great
way to ensure the APIs are sold. However, I think you are missing two points:
You are not a real customer. Yes, it is a great practice to
have people in your own organization build (real!) things that exercise your
APIs. This will help ensure that the APIs really, truly, have use-cases, are
documented correctly, tested, etc… However, it is not sufficient, in my
experience. Your organization’s thinking is clouded and biased. A 3rd party
with independent motivations & priorities will provide much, much better
feedback. Example: I came across a startup building a product with a new web
API. I wanted to use the web API so I signed up. It turns out that the startup’s
mental model was “our customers are building mobile apps and will be calling our
REST APIs from mobile clients”. My use case was to call their APIs from my
SERVER. They had not accounted for this in their design and as a result their
API was challenged in my scenario.
Building End-User Value is behaviorally incompatible with Building
Developer Value. Go back and read Point #1 of my Be
Either an App or A Platform post. In my experience, asking a small team to
delight BOTH end-users and developers simultaneously is a recipe for
mediocrity. Yes, you can do it, but if you do, I’ll argue you will ultimately
fail because lack of focus leads to lack of excellence.
I would argue that you should build (well, more precisely, evolve) APIs from day 1 but certainly an early stage venture should not publish their API or make it available to the public until it’s well baked.
Yep, John, you get it. Right on.
It’s not having an API that’s wrong, it’s getting into a position where you have to maintain backwards compatibilty for every possible edge case. (“Compatibility means deliberately repeating other people’s mistakes”, I was taught as an undergraduate back in 1972). When I started on Saxon in 1999 I decided that the price people would pay for using an open source library was that I would make changes to the API at each release if I thought there was a good reason; I could afford to do that because my customers weren’t paying. Now that my customers are paying, I still do it, because I think it’s in their interest. They have a choice: they can adapt their code to work with the new version, or they can stick with the old version.
Allow me to disagree with you Charlie. Coming from a InfoSec background so you have to forgive my jargon. And the term Attack Vector as Pratyusa K. Manadhata defined it; a system’s attack surface is the set of ways in
which an adversary can enter the system and potentially cause
damage. Hence the larger the attack surface, the more insecure the
system.
Having that said, for sure an API, increases a systems Attack Vector, or may be Bug Vector if I may coin such word. But the problem is, we have to admit that the more ways people are allowed the interact with a system the more useful it is. Twitter wouldn’t have been such successful without their API, it gave people more ways to interact with the system, form the treadmill the tweets ones working-out achievements, to websites that uses twitter for signing in. And as Kav said, startups should adapt their SLA’s, and developers interacting with them should deal with it. This is exactly how agile your API should be.
For me an API, is how you let people you don’t know help you set and define the application you are creating, especially when you are either not sure or not capable of supplying all the needs people dream of.
I’m not sure what “Microsoft Windows without APIs” would have been. I doubt there would have been a release beyond 3.1. My take-aways from this story are that bugs are super-frustrating, but do not necessarily preclude success and that good support matters.
APIs are just too useful not to be there.
I have to say that I think the thrust of Charlie’s argument is spot on: focus on your product. All else is incidental. If your app relies on an API to function, fine (and see my counterpoint below for when I think that might be the case) but otherwise you’ll spend more time and money building and supporting it as a feature. Losing focus is the death of more startups than probably inadequate funding. Charlie really gets it. He’s been there and done that.
Or, you know, you could not be Microsoft. Be Apple and just don’t care about who you break–they deprecate half the OS every major release.
Having been one of the engineers on Windows who had do handle backward compatibility, I have to say I agree completely! Building a set of API’s that are robust and useful requires some serious consideration to explicitly documenting what will and will not be supported. It also requires some very strigent checking to insure that the parameters are enforced, and that you are not accidently exposing internal data structures. I once spent three weeks stepping through the debugger trying to determine why none of the Lotus apps wouldn’t run on Windows/NT. The hack involved to get the thing to run was impressive, sickening, and forever took a little off the top of performance of Windows.