What dooms software projects?

Everybody quotes the opening lines of Anna Karenina at some point. You know the one. It’s the bit about happy families being all alike in their yaddayadda-blabla and how unhappy families are all unhappy in their unique blablabla.

There’s plenty of stuff in Anna Karenina you could quote, but I guess most people give up on it before they get to the second half. Which is a pity because it’s a good book.

It’d be easy to reuse that line in a software context and say that software projects that succeed are all alike (well executed, designed, and marketed) while those that fail all fail in their unique ways.

Unfortunately, it’s the other way around. Every software product that succeeds seems to be its own story, have its own value proposition and work in its unique way from implementation to marketing. Software projects that fail, conversely, tend to careen off the road in similar and often predictable ways.

The biggest and most common mistake is to build software nobody has any need for. It doesn’t matter how well executed an app is if nobody has any need for it. No need, no buyers, no income. Product fails. This is surprisingly common, even in corporate projects for in-house use.

Then there’s the septic code issue. Some code is simply so bad that it’ll take the entire project down.

But one rule of thumb for spotting potential failures that I’ve been using is what I call the Google Wave Heuristic. (Just a rule of thumb, mind. Not a hard and fast law.)

If the product’s tentpole feature doesn’t directly help the user at the task they’re using the product for, then the product is likely to fail.

In Google Wave’s case the tentpole feature was real-time synchronisation and messaging but the products pitched were chat apps and document collaboration. Neither of the major use cases of Google Wave directly benefited from its tentpole feature. From the user’s perspective, Wave chats weren’t an improvement on a nicely implemented native client and document collaboration wasn’t any better than in Google’s own Docs.

Another example was a major product launch I was involved in years ago. A software company released a new major version of its application and the flagship feature that had consumed the majority of the team’s production hours was (drumroll please) … a new licensing system. Its main benefit was that it made the application hard to pirate. It was also incompatible with all existing off-the-shelf licensing and ecommerce systems.

The initial launch was a massive dud but recovered later on as the team knuckled down on improving the app. It finally began to gain headway once it showed substantial improvement over its predecessor. Of course, the next major release after that ended up getting delayed for years and upon collapse pretty much took the company with it—but that’s a story more appropriate to a pub context than public consumption.

(But, yeah, I’ve been involved in enough failed software projects to begin to recognise the smell. It’s very distinctive.)

Every application is judged by the user in terms of how much awesome it lets them do. If you are spending a massive amount of hours and efforts on a feature that you don’t know whether the user wants and has no obvious bearing on the awesome factor, you are probably doing something wrong.

There’s a potential opportunity cost to every feature built into an app. Every tentpole feature you implement that doesn’t directly address the user’s goals means there’s another feature you didn’t implement that might have made the user feel awesome at their task. Worse yet, a lot of these features are ‘invisible’ features where the engineering team spends untold work-hours papering over perceived flaws in the target platform. These are platforms that are generally already massively popular with countless apps where the users don’t seem to mind said platform flaws.

The idea that there is revenue-generating value in fixing these flaws is a completely unproven proposition and undertaking it is generally very risky. If solving the ‘flaw’ were easy or at all worthwhile, it’s likely the platform owner would have done so already. Attacking these gaps is likely to result in a feature that doesn’t work that well and doesn’t directly address the use case at hand.

Also, every new feature is a nail in the ground tying you down, making it harder to change your app. This makes it harder to iterate the app and adapt as you discover how to make the best tool the user can find for the task you address.

Examples of apps doing it right are Readmill and Aldiko. Neither of them were released with support for DRMed ebooks at launch, a feature many would say were a core requirement for any new ebook reading system. What they did is prove their core value proposition. Readmill proved the value of social highlights and ebook commenting. Aldiko proved the value of their UI and features. Then they iterated on increasing that core value. Not only did they avoid shipping with useless tentpole features at launch, they launched with just enough features to let them prove the core proposal of their app.

Interlude: People don’t know what they want…

But they do know what they tend to do.

Whenever I say the first half of that line people assume I’m just repeating the Ford adage (‘if I’d asked people what they wanted they would have demanded buggy whips with glitter or something’). That’s not what I mean.

If you ask people what they want from a word processor, they will simply proceed to describe Microsoft Word (something they know and is familiar). If all software companies did their research this way, all they’d end up doing is cloning existing apps.

(Come to think of it, that would explain a lot.)

However when you ask people what they tend to do whenever they tackle a specific task you tend to get much more interesting answers. You can get remarkably specific stories of how their current apps don’t work well, or how they tend to do X whenever they work on Y to get around problem Z that’s built into the app they currently use. Once you start to take them down this path a light bulb will go on above some users’ heads (not literally, of course) and they will begin to talk about their actual problems that they’d like to solve and not just describe their current work environment. But nobody knows what they want unless you walk them away from the ‘describe the familiar’ place they all start at.

Better yet, stalk user forums for the problem area. The more people write in the forum with the question ‘I can’t figure out how to do this and it’s vital for my project’ the more likely it is that an app directly addressing that will be successful.

Anyway… People don’t know what they want but what they do speaks volumes.

What is the web good at?

The web is absolutely rubbish at offline, pagination, parallel processing, intensive computation like image processing, security, and complex animation. There are various cutting edge browsers that can do some of those things but none of them do any of them well and sometimes they are quite simply disastrously bad.

A browser is really good at rendering scrolling text quickly, with images, decoration, links, and light declarative animation. And it does this on more platforms than I care to count. Everything about this task has been optimised to hell. Ever wonder why you can’t turn scrolling off on mobile by setting overflow: hidden on the body tag? It’s because browser vendors have optimised every pixel of the core document scrolling task breaking some web development edge cases in the process.

(Even with all that work it still goes wrong sometimes. Which does to show how risky it is to attempt to replace it with something hammered together by your engineers in javascript.)

The web is also really good at doing whatever it does cross-platform, on every major computing platform you can get your hands on, provided you stay away from cutting edge features. If structured properly it’s almost automatically accessible. The few built-in UI widgets it has (forms, textareas, buttons, etc.) come for free. A web app is always up-to-date. The only way to be certain a regular web page stays at an older version is to disconnect your device from the network—browsers often don’t cache even when you ask them to. Web browsers are also pretty good at pulling stuff down from a web server you control and getting better at pulling things down from other people’s servers.

So, one obvious thing browser-based apps are awesome at is be front-ends for stuff that actually takes place on your server.

The way people have been making web-based ebook reading apps is to treat all of the major features of a web browser like liabilities. Once you stop spending most of your engineering effort implementing and constantly repairing a feature or two (pagination and offline) and once you begin to treat the core characteristics of the web as advantages your project is much more likely to succeed.

Not guaranteed, just more likely simply by virtue of having more engineering hours to spend on making core tasks awesome.

Besides, how many normal users open a web browser and demand ‘let this be offline!’? Hardly anybody outside of geek circles is even aware of the technical possibility of an offline website. For most the very idea is an absurd paradox. To them the web is what’s online. And where are the queues of people demanding paginated website? You are more likely to find angry crowds demanding that crap pagination blog plugins be turned off as those seem to be universally rubbish. I’d bet that when faced with native, problem-free, pagination and scrolling, most people don’t care either way and will use whatever is default. (And then get used to that default and howl in anger whenever the app dev changes it, natch.)

I’m biased, obviously, by the fact that I have never encountered a javascript-based pagination implementation that worked comfortably and without issues in the default browsers on recent iOS and Android devices.

Engineers have a tendency to see software as a bag of features to be implemented. Most engineers I’ve met have had a strong inclination to assume that this or that feature should be implemented simply because that’s how it’s usually done. No thought given to the user task as a whole. Almost every engineering team I’ve encountered when tasked with making, say, a word processor would end up cloning an existing app instead of breaking the task down to fundamentals and asking themselves what is it really that we need to do? What is the user trying to accomplish? What skills does this app complement? Do we need to address every task that that Word does or can we pick one and do it really well? How do we make the user feel awesome at their task?

But, no. Ask them to do an ebook reader and they will attempt to reimplement iBooks, page curl and all.

Dismissing the potential of serious web-based reading based on existing ebook-oriented web apps is a bit like dismissing online chatting just because Google Wave failed.

If the core selling points of your web-based ebook reader is that it paginates and works offline—two features that native apps easily do much much better—then you’ve already failed.

What is really holding back web-based ebook readers?

There isn’t a need for them.

There is very little in general that web-based readers could do that trumps existing apps. You can’t make them cheaper (end-users aren’t paying for reading apps directly). Most reading systems are already cross-platform to an impressive degree. You’d be hard pressed to surpass native apps in convenience. They might serve to get past platform limitations such as the Kindle Fire’s ban on competing ereading apps or Apple’s app store limitations but those are more vendor concerns than user concerns. And you need the user to be on board because otherwise it’s pointless.

It’s not clear at the moment what problem from the reader’s perspective a web-based ebook reader would solve that existing native apps don’t.

Once upon a time I would have said that a web-based ereader would be great for the social reading thing, but Readmill proved you don’t need the reading app to be on the web to gain the social networking features of a web app. You can divide the responsibilities quite neatly between the native and web app, letting each do what it does best.

I’m convinced that if we didn’t already have ubiquitous ebook apps, a web-based ebook reading app could well be a smash hit. But we do have them and new apps without a unique value proposition won’t replace them, no matter whether the new app is web or native.

One way I can see it succeeding is if you found a specific group of readers who aren’t being served well by existing apps and would benefit from a specialised reading app. A web-based reading app might well be the most effective way to deliver a cross-platform app for that niche.

The other way it could work is if you make an ebook app for those who sincerely prefer scrolling over pagination and are willing to compromise to get it. Might not be a large group, but it might be enough to be worthwhile as a niche reading app.

The Smell of Death

Most software projects fail. No matter what some programmers might like to think, making software is more like art than engineering. As in, 90% of everything is crap and it’s often hard to know why.

The vast majority of projects begin to exude the stench of death well before their first public release and implode safely in the confines of a corporate server room while the management team tries to figure out who gets the blame.

Some get released and fill the room with the whiff of doom as soon as they see daylight.

Like Bookshout for example. That app had such a stench to it on its first major release that it’s amazing their PR reps managed to find a pundit to shill it in public. (Don’t bother to Google it. It’s not worth trying.)

• Tentpole feature nobody cares about? Check. (Their DRM circumvention screen-scraping hack.)
• Awkward and iffy UI design? Check.
• No unique value proposition? Check.
• Half-baked typography? Check.

And to compound everything the tentpole feature was extremely brittle and insecure as it relied on getting the reader’s Amazon password and scraping their account page.

Basically, it didn’t give us any reason to use it. The DRM circumvention feature was a way to make user migration easier, but that’s assuming the user wants to switch in the first place. They never gave them any reason to.

Of course, I haven’t tried it much lately, but that’s more to do with the fact that the latest version keeps crashing on me than anything else. I have a three crash quota before I give up on an app. Do I have serious doubts about Bookshout’s long-term survival? You bet.

Google Play Books on both Android and iOS is another pair of apps that have started to become a little bit too ripe for public exposure. When Amazon’s Kindle app looks more at home and is better designed on Android than Google’s own then you know there’s some serious upper management disinterest going on. Why Google persists in making their own app is perplexing. The rationale for both Android and Chrome is pretty clear: keep the market competitive. Google Play Books does no such thing. What would make the market more competitive is Google shuttering their own app and working with other vendors such as Bluefire, Aldiko, or Readmill to integrate Google Play Books directly. Bonus points for paying app vendors referral fees for every book bought. In the meantime, the current state of Google Play Books is just embarrassing.

Finally—and I feel bad saying this as I’m a fan of the people involved in these projects—every web-based ebook reader I’ve seen so far had at least a whiff of the old death-smell from the outset. These projects were all put together by amazing people capable of amazing things but in most cases there just isn’t a product there—no reason why a reader should care and switch.

What’s worse, the insistence of most of these projects on paginating every book has tended to limit them to the absolute cutting edge, which means they lose the cross-platform advantage of going with a web app and pits them against every odd new bug and every vague inconsistency in every immature specs they are relying on.

That’s without getting into the mess that passes for offline support in most browsers.

When even recent releases of Chrome Android and Mobile Safari aren’t close enough to the cutting edge to work comfortably you can quite simply be certain that you don’t have the foundations for a mainstream product.