The Google Wave Heuristic

What dooms software projects?

Everybody quotes the opening lines of Anna Karenina at some point. You know the one. It’s the bit about happy families being all alike in their yaddayadda-blabla and how unhappy families are all unhappy in their unique blablabla.

There’s plenty of stuff in Anna Karenina you could quote, but I guess most people give up on it before they get to the second half. Which is a pity because it’s a good book.

It’d be easy to reuse that line in a software context and say that software projects that succeed are all alike (well executed, designed, and marketed) while those that fail all fail in their unique ways.

Unfortunately, it’s the other way around. Every software product that succeeds seems to be its own story, have its own value proposition and work in its unique way from implementation to marketing. Software projects that fail, conversely, tend to careen off the road in similar and often predictable ways.

The biggest and most common mistake is to build software nobody has any need for. It doesn’t matter how well executed an app is if nobody has any need for it. No need, no buyers, no income. Product fails. This is surprisingly common, even in corporate projects for in-house use.

Then there’s the septic code issue. Some code is simply so bad that it’ll take the entire project down.

But one rule of thumb for spotting potential failures that I’ve been using is what I call the Google Wave Heuristic. (Just a rule of thumb, mind. Not a hard and fast law.)

If the product’s tentpole feature doesn’t directly help the user at the task they’re using the product for, then the product is likely to fail.

In Google Wave’s case the tentpole feature was real-time synchronisation and messaging but the products pitched were chat apps and document collaboration. Neither of the major use cases of Google Wave directly benefited from its tentpole feature. From the user’s perspective, Wave chats weren’t an improvement on a nicely implemented native client and document collaboration wasn’t any better than in Google’s own Docs.

Another example was a major product launch I was involved in years ago. A software company released a new major version of its application and the flagship feature that had consumed the majority of the team’s production hours was (drumroll please) … a new licensing system. Its main benefit was that it made the application hard to pirate. It was also incompatible with all existing off-the-shelf licensing and ecommerce systems.

The initial launch was a massive dud but recovered later on as the team knuckled down on improving the app. It finally began to gain headway once it showed substantial improvement over its predecessor. Of course, the next major release after that ended up getting delayed for years and upon collapse pretty much took the company with it—but that’s a story more appropriate to a pub context than public consumption.

(But, yeah, I’ve been involved in enough failed software projects to begin to recognise the smell. It’s very distinctive.)

Every application is judged by the user in terms of how much awesome it lets them do. If you are spending a massive amount of hours and efforts on a feature that you don’t know whether the user wants and has no obvious bearing on the awesome factor, you are probably doing something wrong.


There’s a potential opportunity cost to every feature built into an app. Every tentpole feature you implement that doesn’t directly address the user’s goals means there’s another feature you didn’t implement that might have made the user feel awesome at their task. Worse yet, a lot of these features are ‘invisible’ features where the engineering team spends untold work-hours papering over perceived flaws in the target platform. These are platforms that are generally already massively popular with countless apps where the users don’t seem to mind said platform flaws.

The idea that there is revenue-generating value in fixing these flaws is a completely unproven proposition and undertaking it is generally very risky. If solving the ‘flaw’ were easy or at all worthwhile, it’s likely the platform owner would have done so already. Attacking these gaps is likely to result in a feature that doesn’t work that well and doesn’t directly address the use case at hand.

Also, every new feature is a nail in the ground tying you down, making it harder to change your app. This makes it harder to iterate the app and adapt as you discover how to make the best tool the user can find for the task you address.

Examples of apps doing it right are Readmill and Aldiko. Neither of them were released with support for DRMed ebooks at launch, a feature many would say were a core requirement for any new ebook reading system. What they did is prove their core value proposition. Readmill proved the value of social highlights and ebook commenting. Aldiko proved the value of their UI and features. Then they iterated on increasing that core value. Not only did they avoid shipping with useless tentpole features at launch, they launched with just enough features to let them prove the core proposal of their app.

Interlude: People don’t know what they want…

But they do know what they tend to do.

Whenever I say the first half of that line people assume I’m just repeating the Ford adage (‘if I’d asked people what they wanted they would have demanded buggy whips with glitter or something’). That’s not what I mean.

If you ask people what they want from a word processor, they will simply proceed to describe Microsoft Word (something they know and is familiar). If all software companies did their research this way, all they’d end up doing is cloning existing apps.

(Come to think of it, that would explain a lot.)

However when you ask people what they tend to do whenever they tackle a specific task you tend to get much more interesting answers. You can get remarkably specific stories of how their current apps don’t work well, or how they tend to do X whenever they work on Y to get around problem Z that’s built into the app they currently use. Once you start to take them down this path a light bulb will go on above some users’ heads (not literally, of course) and they will begin to talk about their actual problems that they’d like to solve and not just describe their current work environment. But nobody knows what they want unless you walk them away from the ‘describe the familiar’ place they all start at.

Better yet, stalk user forums for the problem area. The more people write in the forum with the question ‘I can’t figure out how to do this and it’s vital for my project’ the more likely it is that an app directly addressing that will be successful.

Anyway… People don’t know what they want but what they do speaks volumes.

What is the web good at?

The web is absolutely rubbish at offline, pagination, parallel processing, intensive computation like image processing, security, and complex animation. There are various cutting edge browsers that can do some of those things but none of them do any of them well and sometimes they are quite simply disastrously bad.

A browser is really good at rendering scrolling text quickly, with images, decoration, links, and light declarative animation. And it does this on more platforms than I care to count. Everything about this task has been optimised to hell. Ever wonder why you can’t turn scrolling off on mobile by setting overflow: hidden on the body tag? It’s because browser vendors have optimised every pixel of the core document scrolling task breaking some web development edge cases in the process.

(Even with all that work it still goes wrong sometimes. Which does to show how risky it is to attempt to replace it with something hammered together by your engineers in javascript.)

The web is also really good at doing whatever it does cross-platform, on every major computing platform you can get your hands on, provided you stay away from cutting edge features. If structured properly it’s almost automatically accessible. The few built-in UI widgets it has (forms, textareas, buttons, etc.) come for free. A web app is always up-to-date. The only way to be certain a regular web page stays at an older version is to disconnect your device from the network—browsers often don’t cache even when you ask them to. Web browsers are also pretty good at pulling stuff down from a web server you control and getting better at pulling things down from other people’s servers.

So, one obvious thing browser-based apps are awesome at is be front-ends for stuff that actually takes place on your server.

The way people have been making web-based ebook reading apps is to treat all of the major features of a web browser like liabilities. Once you stop spending most of your engineering effort implementing and constantly repairing a feature or two (pagination and offline) and once you begin to treat the core characteristics of the web as advantages your project is much more likely to succeed.

Not guaranteed, just more likely simply by virtue of having more engineering hours to spend on making core tasks awesome.

Besides, how many normal users open a web browser and demand ‘let this be offline!’? Hardly anybody outside of geek circles is even aware of the technical possibility of an offline website. For most the very idea is an absurd paradox. To them the web is what’s online. And where are the queues of people demanding paginated website? You are more likely to find angry crowds demanding that crap pagination blog plugins be turned off as those seem to be universally rubbish. I’d bet that when faced with native, problem-free, pagination and scrolling, most people don’t care either way and will use whatever is default. (And then get used to that default and howl in anger whenever the app dev changes it, natch.)

I’m biased, obviously, by the fact that I have never encountered a javascript-based pagination implementation that worked comfortably and without issues in the default browsers on recent iOS and Android devices.

Engineers have a tendency to see software as a bag of features to be implemented. Most engineers I’ve met have had a strong inclination to assume that this or that feature should be implemented simply because that’s how it’s usually done. No thought given to the user task as a whole. Almost every engineering team I’ve encountered when tasked with making, say, a word processor would end up cloning an existing app instead of breaking the task down to fundamentals and asking themselves what is it really that we need to do? What is the user trying to accomplish? What skills does this app complement? Do we need to address every task that that Word does or can we pick one and do it really well? How do we make the user feel awesome at their task?

But, no. Ask them to do an ebook reader and they will attempt to reimplement iBooks, page curl and all.

Dismissing the potential of serious web-based reading based on existing ebook-oriented web apps is a bit like dismissing online chatting just because Google Wave failed.

If the core selling points of your web-based ebook reader is that it paginates and works offline—two features that native apps easily do much much better—then you’ve already failed.

What is really holding back web-based ebook readers?

There isn’t a need for them.

There is very little in general that web-based readers could do that trumps existing apps. You can’t make them cheaper (end-users aren’t paying for reading apps directly). Most reading systems are already cross-platform to an impressive degree. You’d be hard pressed to surpass native apps in convenience. They might serve to get past platform limitations such as the Kindle Fire’s ban on competing ereading apps or Apple’s app store limitations but those are more vendor concerns than user concerns. And you need the user to be on board because otherwise it’s pointless.

It’s not clear at the moment what problem from the reader’s perspective a web-based ebook reader would solve that existing native apps don’t.

Once upon a time I would have said that a web-based ereader would be great for the social reading thing, but Readmill proved you don’t need the reading app to be on the web to gain the social networking features of a web app. You can divide the responsibilities quite neatly between the native and web app, letting each do what it does best.

I’m convinced that if we didn’t already have ubiquitous ebook apps, a web-based ebook reading app could well be a smash hit. But we do have them and new apps without a unique value proposition won’t replace them, no matter whether the new app is web or native.

One way I can see it succeeding is if you found a specific group of readers who aren’t being served well by existing apps and would benefit from a specialised reading app. A web-based reading app might well be the most effective way to deliver a cross-platform app for that niche.

The other way it could work is if you make an ebook app for those who sincerely prefer scrolling over pagination and are willing to compromise to get it. Might not be a large group, but it might be enough to be worthwhile as a niche reading app.

The Smell of Death

Most software projects fail. No matter what some programmers might like to think, making software is more like art than engineering. As in, 90% of everything is crap and it’s often hard to know why.

The vast majority of projects begin to exude the stench of death well before their first public release and implode safely in the confines of a corporate server room while the management team tries to figure out who gets the blame.

Some get released and fill the room with the whiff of doom as soon as they see daylight.

Like Bookshout for example. That app had such a stench to it on its first major release that it’s amazing their PR reps managed to find a pundit to shill it in public. (Don’t bother to Google it. It’s not worth trying.)

  • Tentpole feature nobody cares about? Check. (Their DRM circumvention screen-scraping hack.)
  • Awkward and iffy UI design? Check.
  • No unique value proposition? Check.
  • Half-baked typography? Check.

And to compound everything the tentpole feature was extremely brittle and insecure as it relied on getting the reader’s Amazon password and scraping their account page.

Basically, it didn’t give us any reason to use it. The DRM circumvention feature was a way to make user migration easier, but that’s assuming the user wants to switch in the first place. They never gave them any reason to.

Of course, I haven’t tried it much lately, but that’s more to do with the fact that the latest version keeps crashing on me than anything else. I have a three crash quota before I give up on an app. Do I have serious doubts about Bookshout’s long-term survival? You bet.

Google Play Books on both Android and iOS is another pair of apps that have started to become a little bit too ripe for public exposure. When Amazon’s Kindle app looks more at home and is better designed on Android than Google’s own then you know there’s some serious upper management disinterest going on. Why Google persists in making their own app is perplexing. The rationale for both Android and Chrome is pretty clear: keep the market competitive. Google Play Books does no such thing. What would make the market more competitive is Google shuttering their own app and working with other vendors such as Bluefire, Aldiko, or Readmill to integrate Google Play Books directly. Bonus points for paying app vendors referral fees for every book bought. In the meantime, the current state of Google Play Books is just embarrassing.

Finally—and I feel bad saying this as I’m a fan of the people involved in these projects—every web-based ebook reader I’ve seen so far had at least a whiff of the old death-smell from the outset. These projects were all put together by amazing people capable of amazing things but in most cases there just isn’t a product there—no reason why a reader should care and switch.

What’s worse, the insistence of most of these projects on paginating every book has tended to limit them to the absolute cutting edge, which means they lose the cross-platform advantage of going with a web app and pits them against every odd new bug and every vague inconsistency in every immature specs they are relying on.

That’s without getting into the mess that passes for offline support in most browsers.

When even recent releases of Chrome Android and Mobile Safari aren’t close enough to the cutting edge to work comfortably you can quite simply be certain that you don’t have the foundations for a mainstream product.

Amazon’s biggest ally is Apple

I’ve never understood why people position Apple and Amazon as rivals in the ebook game. While it’s true that the two have clashed that conflict is a result of incompatible platform goals, not rivalry.

Conflict is not rivalry and two organisations that conflict occasionally may well be allies in the bigger picture where rivals may not.

Let me elucidate…

The Apple/Amazon conflict has presented itself in a variety of ways:

  • Direct competition. Apple entered the ebook retail game and broke the law when it enabled big publishers’ price collusion.
  • App restrictions. Apple prevents vendors from integrating ebook sales into their ebook reading apps.
  • Apple entered the proprietary format game by forking EPUB with iBooks Author books.

—How can you say those two aren’t rivals? I mean look at that!

Easy. These conflicts are not a result of an Apple/Amazon rivalry but of Apple’s ambitions for its platform.

Apple wants three things for iOS:

  • Lock in. It very much doesn’t want people to be able to switch easily to Android. (This is abundantly clear from the emails published as a part of the DOJ’s investigation into agency pricing.) An iBooks ebook platform that only works on Apple devices serves that end goal nicely.
  • It wants its platform vig. Apple would rather a digital transaction not take place at all than for it to take place on iOS and it not getting its cut.
  • It wants large-scale education contracts for iPads in schools. For that it needs textbooks and, since it wants its vig, it can’t just work with one of the existing vendors.

None of these things are specifically directed at Amazon except insofar as they are the biggest ebook vendor. Apple can’t claim it is doing these things out of concern for the consumer or in an attempt to make its platform more secure against fraud. If that were the case hey would have done what Google did with its Play Developer Program Policies: on Google Play services and cross-platform digital goods such as ebooks are specifically exempt from the requirement that apps use Google Play’s in-app billing service.

—Okay, okay. So, maybe Apple isn’t specifically targeting Amazon but there’s still conflict. I don’t see how you can claim that Apple is Amazon’s biggest ally.

Because Amazon is the default choice for ebooks. It is what your average consumer first thinks of when somebody mentions ebooks. They have a mindshare that is even bigger than their marketshare (which is big enough to begin with).

Amazon isn’t hurt at all by Apple’s demands. A reader who can’t buy an ebook in the Kindle app will just go to the website. It only hurts the Kindle app for iOS development team.

Apple’s demands hurt Amazon’s true rivals—other ebook vendors—much more that they hurt Amazon. These vendors would benefit enormously from being able to directly integrate ebookstores into their apps. They even have a standard that would let any vendor offer their ebook catalogue for sale within any ebook reading app: OPDS.

The EPUB faction of the ebook world is hurt more by Apple forking the standard and by its general platform behaviour than Amazon is. Apple’s tactics have weakened the standard and made it less competitive.

If Apple added an exemption for ebooks to their developer policies, something similar to Google’s exemption, we’d get a more competitive landscape for ebooks on iOS. If Apple went all in on EPUB, extending it instead of forking it when necessary, the modularised EPUB ecosystem would be stronger for it. If Apple made ebook development tools that targeted EPUB and not a proprietary format, the non-Amazon side of ebook retail would have more diversity of titles and would be more competitive. If Apple turned iBooks into an iBooksView, a general purpose widget for rendering ebooks, every single ebook app on the platform would improve as a result.

Doing all of these things would strengthen iOS by improving the apps available for the platform. That, in and of itself, should be enough reason for Apple to do so.

But, no. Apple wants lock-in and a vig.

Readmill versus Kindle – Readmill is worth the hassle

Last week I decided to reread a couple of books in Readmill that I had previously read in the iOS Kindle app.

Let’s see how the two compare.

Kindle for iOS

It’s a turd. There’s no way to express just how awful that app is while still couching your annoyance in polite language.

It’s not awful. It’s fucking awful.

Of course, some of the annoyance stems from general Kindle awfulness such as frequent bugs in how the platform does sharing and general disregard for basic typography.

It was boring anyway

It’s completely unacceptable that a reading platform should drop the last few words off an even short passage, but that’s exactly what the Kindle does (for iOS at least).

  1. Highlight a passage.
  2. Go to kindle.amazon.com.
  3. Discover that the last few words of all of your highlights are missing.
  4. Swear like a sailor who has banged his knee.

Reloadarama.

It reloads and re-renders constantly. Which wouldn’t be so bad if re-rendering didn’t lose your history. This means you can’t go back from a link if you follow it and the app re-renders for some reason.

Imagine if Mobile Safari, in addition to having to frequently reload the page due to memory and caching problems, it lost all of your history in the process.

Then imagine that it does this every time you put down your iPad or switch between apps. Now you know what the Kindle app is like.

Finger-painting with overcooked pasta

Like other Kindles it repaginates on every link. Reading, navigating, and highlighting is like sieving cold overcooked pasta with your bare fingers.

The economy kind of pasta, the type you get almost for free from the bottom shelf in the supermarket. The kind that disintegrates when you put it in the pot. Not the nice wholewheat kind.

There is no consistency in position or rendering. This compromises the reader’s visual memory and is generally annoying. Like when you end up navigating back and forth a bit (y’know browsing the book) and then get back to your spot to get back to reading, it’s repaginated so nothing looks the same. Very annoying.

Crayon scribbles have better typography

The default stylesheet makes the default font (Georgia) look even more pedestrian and uninspired than it is normally. And it’s a very pedestrian typeface to begin with.

Of course, in leu of a considered default stylesheet with thought out proportions, line-height to font size ratio, indentation, and margins, we get escalating configurability.

The thing is, no matter how you tweak the line-height, margins, or which of the provided fonts you select, the app’s typography remains subpar. It just isn’t nice to read.

The font selection is crap and leaves out a host of interesting built in fonts. Providing better fonts would be a start.

Your finger is an error-prone highlighter

Swipe highlighting is a gimmick that results in having to repeatedly highlight, then delete highlight, then highlight, then delete, etc. to get it right. It’s incredibly error-prone because it doesn’t offer any method for adjusting the selection. The only advantage it offers is the ability to highlight across page boundaries which, frankly, is useful. But, surely, there’s some other way of accomplishing that without taking away the ability to adjust your selection?

Swipe highlighting: rubbish gimmick that disregards everything we know about usability.

Nobody wants to steal your stupid book

Copy-paste is disabled in the Kindle apps, both on iOS and Mac. Couple this with the kindle.amazon.com bug above and this means there is no way for you to easily quote a passage without having to retype parts of it.

Why the hell do you need to disable copying and pasting? What the fuck is wrong with people?

Note to authors and publishers:

Quoting your books is a good thing.

It spreads the word. Gets you new readers. You know, all good stuff. You should be furious that Amazon is disabling copying and pasting. In doing so it’s just rounded all of your marketing staff out back and shot them in the head. You don’t want to be beholden to Amazon’s whims for marketing for the rest of your careers, do you?

Note: this isn’t a question of DRM. Even Copy-paste isn’t enabled in any title in the Kindle app, not even in the DRM-free titles.

No polish, because you know that old saying about turds and polish, right?

Kindle iOS really lets you down on the small details. For example, magnification when you highlight/select is blurry on retina devices.

There’s also this curious item in the settings called ‘Publisher Font’. The fun part is that it doesn’t seem to do a thing other than switch the font to Georgia no matter the ebook or whether it has a font embedded or not. The app doesn’t seem to support embedded fonts so that can’t be what that menu item is for. Why they ship it broken like this is beyond me.

Don’t get me started on the UI in general…

And then there are the odd updates like the one that removed margins show that the main reason why the Kindle for iOS developers add configurability is that they don’t have a clue what sane defaults would look like.

Their development process seems to look like this:

  1. Make a change to the defaults that a first year BA student in design would know is dumb.
  2. Witness a massive backlash from readers.
  3. Think that nobody can be pleased and ‘fix’ it by adding configurability without even trying to understand the underlying problem.

In short: Kindle for iOS sucks.


Readmill

In comparison, Readmill is a joy. The default font is an excellent, beautiful, choice—looks like a custom typeface, not one of the platform’s built-ins. It’s the only iOS reading app where I prefer the default font over a good embedded font.

It automatically adjusts the margins when you resize the text so that the width of the text-column never gets unreadably wide or narrow.

The typography in terms of font sizes, line height, and margins are all inter-connected that way to maintain optimal readability at all font-sizes. The result is an app that is a joy to read.

Quote, share, discuss—it’s all good

  • Here we have none of that nonsense of disabling copy-paste.
  • Everything regarding highlights, notes, online sharing (twitter, facebook, wordpress), and discussions is top-notch.
  • Highlights navigation, both in-app and on the web is dramatically nicer. The Highlights sidebar is a nice touch.

UI niceties

The Readmill chrome is nice enough for you to sometimes page through an ebook and forget it is there as you get engrossed in the book. This while still being usable and accessible. There is no higher compliment for an ereader UI.

Being able to swipe up or down to adjust brightness is a nice touch. You could live without it, but after getting used to it you don’t want to.

The page number indicator is a nice and simple way of showing your progress through the book. Since the ebook is rendered as pages it makes sense to count your progress in those pages. No needless reinvention of the wheel like the Kindle’s opaque location numbers.

The only thing missing is the iBooks-style “X pages left in this chapter” which I’ve found very useful. Of course, Kindle doesn’t have any progress indicator when the chrome is minimised and an opaque, inaccessible, one when the chrome is visible and in the way.

It is slightly annoying that after you press a link you have to switch the UI chrome on to find the back link. In theory that should be simple to fix. No reason not to show the back link in some way even when the chrome is switched off.


Sidebar on configurability

A love of preference toggles and configurability is an endemic among the engineeringly minded. It’s easy to see why: they don’t mind the cognitive overhead because it fits in with how they approach all of their problems.

Having some preference toggles is a good thing. But when you add toggles and sliders for margins, font size, indentation, and more, you escalate the complexity that users need to to tackle in your app.

A reader who wants to increase the font size will now have to deal with all of those other settings because they are interrelated. Font size, line height, and margins are all cogs in the same typography machine. Change one thing and you affect the other. So a simple task to change the font size can easily become a multi-minute odyssey among a forest of buttons, toggles, and sliders.

People are also very bad at assessing their own needs. A common ailment among my relatives is a tendency for muscle, joint, and other physical aches and pains due to bad ergonomics. (A curse that I thankfully am free of.) What makes the problem worse is that they used to be bad at figuring out how they should sit, what chairs they should be using, how to use their tools, etc.. They needed to be told, by an expert. Even then, following that advice was often uncomfortable at first, even though it helped address the problem in the long run. Without expert guidance, most people will choose ergonomics and positions that will actively harm them in the long run.

The same applies to reading system configurability. People will most of the time choose settings that will make them read slower and remember less.

Finally, by adding all of those preference toggles to your reading app you have turned it into a design tool. By letting the reader control all of the variables you have forced them to become a typographer and designer. They end up having to take on all of the complexity of designing an entire ebook for themselves.

Moreover, it’s a crap design tool. I know a couple of things about designing for reading and I can’t for the life of me get an optimal reading experience out of the Kindle app or Marvin (an otherwise excellent app). I can never get the proportions right between the width of the text-column, typeface, line height, and font size. I can get those apps to be readable, sure, but never optimal.

The only design configuration a reading app should have are font size, inversion (switching to light on dark), and contrast. These toggles are based on physical reader needs. Some eyes just work differently from others. They aren’t preferences but requirements for a section of your readership.

Everything else should be controlled by the system and derived from the variables the reader does control. That way you give them an optimal reading experience without excluding anybody.

Which means that the only thing missing from Readmill is a contrast slider.


API

The idea of having an API for my reading platform of choice is intriguing. It means that the platform owner doesn’t have to solve all of the issues the readers have. In theory, that should mean that I could put together a script that exports my Readmill highlights and notes into a usable format for writing. And that means that all sorts of apps out there will and have been rolling out Readmill features. It promises to increase the benefit I gain from the platform immensely.

Here come the downsides

Readmill’s single biggest flaw is the absence of anything resembling library management. Your books are presented in a very nice looking list that you can filter to show only finished books or those you’re currently reading.

Of course, Kindle for iOS’s library management is also rubbish for largely the same reason. You only have a list of your stuff that you can filter in rather limited ways.

Both apps need collections. They need the ability to browse the book by author. And because Readmill also supports PDFs you need to be able to filter by file type. Browsing by subject matter would also be nice, but the general quality of ebook metadata might not be up to snuff for that to work. Of course, if there was an app out there that used ebook metadata in interesting ways in its library UI, then maybe publishers would be more motivated to put in the effort to make it nice.

One serious annoyance is that sometimes Readmill will get the metadata seriously wrong (like marking all volumes of a book volume 1). Without an ability to correct that metadata in the app (while sending the correction upstream as a suggestion, of course) few people are going to bother to tell Readmill about errors when they happen.

Most people’s routine is a bit like this:

  1. Load the ebook into Readmill either via a ‘Send to Readmill’ button or opening it directly using the app on the device.
  2. See that the service seems to give some of the ebooks the wrong title.
  3. Either ignore the error and read on or switch to another app in frustration.

Very few are going to bother with going to the Readmill website, navigate to the book page and make a ‘suggestion’ there. Readers need to be able to correct a book’s title in-app

Finally, none of the major ebook vendors, at least those with any selection to speak of, offer direct Readmill integration. Why Kobo doesn’t let you set it up so that all of your purchases get automatically added to Readmill is beyond me.

The thing everybody does wrong

All highlights lose all formatting and don’t include images. This is the case for all ebook reading systems today. I know this is a difficult technical problem (well, so I’m told) but this is essential for any book that isn’t a prose novel. Formatting for many titles is integral to their meaning. Changing a list into a jumbled paragraph can render a highlight unreadable and this happens all too frequently.

Conclusion: Readmill is worth the hassle

Even without the social features or the highlighting features, Readmill’s UI, design, and typography are enough to make it, in my opinion, the best ebook reading app available for iOS devices.

When you add the platform’s general features such as social sharing, discussions, and an API, then it becomes an unbeatable choice. It’s so good that it’s worth whatever hassle you have to go through to get your ebooks into it. Even if that means using a notoriously crap application to convert your existing library to DRM-free epubs. Even if that means having to manually download the Adobe DRM files from Kobo to upload into Readmill whenever you buy an ebook.

I’ll put up with all sorts of nonsense now when I’m trying to get ebooks to read because reading in Readmill is nice enough to be worth the hassle.

Readmill is, quite simply, the benchmark for all future ebook reading apps.

Proprietary ebook formats versus DRM

Micah made this here statement on Twitter the other day, articulating neatly what a lot of us have been thinking for a while now:

Very true.

It’s something that has bothered me for years and years. I spent years arguing against the use of proprietary formats in interactive media academia (they were unnaturally fond of what was then Macromedia Director). Then proprietary ebook formats became my bugaboo. But tilting at windmills hasn’t gotten me anything but heartache and a reputation for being a bit of a jerk. I’ve now accepted the fact that proprietary formats are always going to be with us. If it doesn’t bother the buyer it doesn’t bother the buyer, simple as that. But I’ve often tried to figure out a good framework for discussing and analysing this dynamic between proprietary and standard formats. What’s the best way to think about this and find a way to combat proprietary formats?

One angle is that standardisation lowers cost for producers and lets them make more interesting products, but that’s not likely to sway Amazon who value the flexibility and power of an owned format and don’t bear the costs of production. And customers generally don’t care since they might not even benefit at all from lower production costs (some producers would just use the opportunity to increase their margins).

The other angle is interoperability and modularity, which increases the flexibility and value of the ecosystem as a whole. But that also changes the power dynamic in the ecosystem in less than predictable ways, something that the big dogs in the system won’t like. When you’re the biggest there’s no such thing as good unpredictable change. Amazon’s system is mostly vertically integrated anyway, leaving little room for interop. And many opportunities for really lucrative interoperability have been throttled in the crib by Apple’s stringent iOS policies. (Why ebook vendors aren’t doing more interesting things on Android where they aren’t held back by the platform owner’s policies is beyond me, but that’s a blog post for a different day.)

Then I stumbled upon the super obvious way to look at the problem. So obvious that it’s embarrassing that I haven’t pursued it as a serious argument before. Yeah, I know, I can be thick sometimes.


I didn’t hit upon it directly, but Micah’s above tweet did remind me of something I’d just read. From The Technique of the Novel – A Handbook on the Craft of the Long Narrative by Thomas H. Uzzell:

Ask any novelist in trouble with his plot what he intends the effect to be and he will answer something like this: “I intend to show that love between two such people is impossible.” This is material, not effect. Effect would be, say, the pathos or tragedy felt by the reader in a narrative about two people vainly attempting happiness in marriage. Amateurs in any art talk in terms of materials; professionals, in terms of effects.

Effects are subjective experiences; materials are objective experiences. Effect is response; materials are stimulus. Effects are the emotional qualities of things.

It’s features versus benefits all over again. In this context the materials the novelist uses are the features and the effects on the reader are the benefits. A writer should not think in terms of the materials (what you write) but in terms of the effects (how the writing affects the reader).

It’s ties into an adage from marketing, features are meaningless to the buyer, they need to be told how they benefit. But if there’s one thing I learned from my friends in marketing back in my software days it’s that this principle applies everywhere. No marketer can gloss up a Frankenstein monster app pieced together out of departmental hobby horses. Most software is a confusing turd made out of disparate components by a bunch of socially inept developers who can’t think in terms of user benefit. Moreover, they don’t really care about the user. Most developers think in terms of abstract beauty of the code and architecture, conceptual integrity of the components, and of ticking of checkboxes in a feature list. They don’t give a toss about the experience unless you can itemise it as a development checklist.

Bringing this back to ebooks…

Those who are trying to shift the market away from proprietary formats can’t try and market their way out of the problem. A tactic used by some is to harangue critics like me for pointing out important flaws in the EPUB ecosystem, but silencing critics won’t address the flaws. It will not change the fact that as a whole, the EPUB ecosystem offers readers fewer benefits than the Kindle ecosystem.

Offering equal benefits will not be enough to sway consumers. To change the status quo you need to outclass Amazon massively in benefits.

And in case you were wondering, Readium SDK is a feature, not a benefit. It’s what you do with it is what’s going to count.


My suggestion is simple: focus on the benefits Amazon can’t replicate.

If a reading app feature turns out to be a competitive advantage, Amazon is likely to copy it with ease.

Rendering or interactivity features aren’t likely to make a difference because sensible publishers will focus on making their titles cross-platform compatible. Amazon’s rendering and interactivity features are going to dominate as the lowest common denominator.

You can’t beat Amazon on selection or price. The Kindle’s ease of use is going to be hard to top. Their customer service is far above what others offer.

The one thing the EPUB ecosystem can offer that Amazon can’t, is tight interoperability between unrelated ebook vendors, services, and reading apps.

That’s it. That’s your only card to play.

  • A major retailer could implement Readmill’s Oauth integration API. Imagine buying an ebook from B&N, Kobo, or Google and having it automatically load into your Readmill library. Awesome, right? It would be even better if your Kobo library automatically synced your purchases with your Readmill library and vice versa. You wouldn’t even need new standards to do this, just the will to implement.
  • Apple could change its policies to allow in-app ebook browsing and purchase and enable more integration and interoperability in ebook reading apps.
  • We need a high quality web-based ebook reading app integrates with a host of relevant services. Most attempts I’ve seen to date are buggy, unusable, bare-bones, or half-arsed.
  • Apple could implement something like Readmill’s Oauth API, letting retailers securely send ebooks to iBooks. Or, again, better yet if they implement a library syncing API.
  • App developers could standardise rendering, including how overrides behave and how pagination affects existing web standards.

Basically, what I want is for the EPUB side of the ebook market to put their money where their mouth is. So far, they only seem to support EPUB because it isn’t Amazon and don’t take any advantage of the biggest benefits of standardisation. Namely, interoperability and modularity.

As I said, the one major advantage of a standard format is interoperability. The obsession ebook vendors have with silos and their antipathy towards easy interop is crippling their only competitive advantage over Amazon, the one big thing they can use to increase the benefit a reader gets from their ecosystem. Being able to easily mix and match reading apps with retail services and have them integrate tightly is something Amazon can’t replicate.

Copying Amazon’s vertically integrated stack when your only sensible strategy is interoperability is, quite frankly, insane.

With the way B&N, Kobo, Google, and Apple have been behaving, it’s a miracle that Amazon still doesn’t own more than 80% of the market.

Then again, what little headway they have made was largely due to illegal collusion.

B&N, Kobo, Google, and Apple separately can never compete with Amazon on price or range.

If I could hook all of their ebook retail services to Readmill so that all of my purchases are automatically added to my library, then I, as a consumer, can begin to treat them as a single market. Not having to worry about whether any given ebookstore is compatible with my chosen reading app makes me less resistant to try them all. Impulse purchases become more likely.

Together they can offer a competitive price and range. A book that isn’t available in Kobo might be available in B&N, Google Play, or the iBookstore. Proper interoperability will convert more readers away from the Kindle and so increase EPUB sales, to everybody’s benefit.

And, as I’ve been saying, the benefit is what counts.

Computers are too difficult and people are computer illiterate

There are quite a few things that prevent people from learning and acquiring new skills. This isn’t a bad thing since what many of these ‘blockers’ have in common is that they tend to help us in unfamiliar environments and new problems. Some of these ingrained heuristics are biological (i.e. infants begin to exhibit them early). Some of them are cultural.

We have an internal built in physics model that helps us interact with the world around us. It’s accurate enough for day to day living but it’s wrong enough to interfere with our understanding of how physics actually work. Many of our instincts and preconceptions prevent us from internalising a more accurate physics model.

Another habit of ours, at least in the west, is to perceive dichotomies—binary opposites—in our surroundings. It’s a tendency that can be useful: it helps us quickly pick one action among alternatives. “Do you want to go out for a drink or stay in tonight?”

Then there’s our tendency adapt everything into a familiar story, preferably one that reinforces our preconceptions and ideologies. Again, this is useful because stories are the oldest and most effective tool for information and skills transfer that humanity has, probably as old as language itself. That’s why you should always socialise with your colleagues during coffee breaks. The stories the old-timers tell contain valuable information for how the work is done.

These are all valuable tactics. As long as you aren’t trying to understand a complex system, that is.


The following is from a blog post written by a teacher trying to get across his frustration with just how computer illiterate most people seem to be.

Kids Can’t Use Computers… And This Is Why It Should Worry You:

The truth is, kids can’t use general purpose computers, and neither can most of the adults I know. There’s a narrow range of individuals whom, at school, I consider technically savvy. These are roughly the thirty to fifty year-olds that have owned a computer for much of their adult lives. There are, of course, exceptions amongst the staff and students. There are always one or two kids in every cohort that have already picked up programming or web development or can strip a computer down to the bare bones, replace a motherboard, and reinstall an operating system. There are usually a couple of tech-savvy teachers outside the age range I’ve stated, often from the Maths and Science departments who are only ever defeated by their school laptops because they don’t have administrator privileges, but these individuals are rare.

I know from personal experience (teaching, sitting next to the customer service guy at my last place of employment constantly headdesking all day out of frustration) that he’s completely right. Adults have worn their computer illiteracy as a badge of pride for many years now so it shouldn’t surprise anyone that their children share their digital inadequacies. Moreover, neither group is even willing to try to solve a problem when they encounter it.

As recent privacy and malware issues have demonstrated, there is a price to be paid for computer illiteracy. And nobody should ever be proud of being ignorant. If not knowing something is a nonissue, then it’s a nonissue, but don’t brag about it.


That blogger also demonstrates his linguistic ignorance when he explains that he likes to be an arsehole whenever somebody uses the term ‘internet’ to mean ‘my access to the internet’ instead of the internet itself. As in ‘the internet isn’t working’.

I mean, just how stupid do you have to be to not realise that almost everybody who says this knows very well that the entire internet hasn’t stopped working? It’s analogous to saying ‘the TV channels aren’t working’ when your cable TV set-top box is on the fritz. It doesn’t mean you think those channels aren’t broadcasting. It means that you don’t have access to any of them.

It isn’t just stupid to misunderstand language like this, it demonstrates a wilful ignorance of spoken English, wilful because he’s clearly heard the phrase often enough to understand what people are actually trying to say.


The following is from a blog post written by Jeff Eaton, a programmer with customer support and teaching experience:

The problem is not that a URL and a Search Term are two different things. The problem is that that particular distinction is one of thousands that are hidden under the surface of simple computer and internet tasks. What’s the difference between a “program” and a “web site?” What’s the difference between a local and a remote file? What’s a remote file? What’s caching? How do you tell the difference between a browser window that looks like a dialog box, and a modal window that contains a browser pane? Because guess what? All of those things matter at some point — and somewhere out there is a development team working hard to blur the distinction for their application, just for the hell of it.

His point is that computers are very complex things, more complex than those of us familiar with them think they are. A person can be intelligent, highly specialised, well educated, and still not be interested in learning how to properly use a computer. Why should they? Computers are more complex than they have to be and the payoff for understanding that complexity is, for most people, very limited.


Computers can both be too complex and people can be too lazy to apply themselves in computing. You can both criticise people for taking pride in ignorance and criticise computers for being needlessly complex. Despite what many commenters seem to think, pointing out the latter does not invalidate the former. And, conversely, pointing out the former doesn’t invalidate the latter.

Don’t fall into the trap of assuming that you have to accept the binary as presented.

There is a risk in presenting facts as a neat story. The problem is often that storytelling is often too effective a tool for presenting ideas and ‘facts’, trumping data, statistics, and research. Turning the usual ‘kids know more about computers than teachers’ story onto its head is a neat story, especially since it happens to be terrifyingly true, but it also obscures important problems with how modern computing works.

Namely, that computers are much too complex and difficult to use.

Why disruption goes unchecked

Good advice goes counter to established common sense

A few months ago, while I was filling an unexpected surplus of free time, I asked myself the following question:

—Hypothetically, what would your advice be if a medium-to-large publishing company came to you asking how they should handle the shift to digital?

On face value, this is an easy question with an easy answer because the literature and research tells us exactly what they should do.

Kevin Charman-Anderson, in an excellent blog post on the impact of digital on news organisations, outlines the answer neatly:

In April, I heard Gilbert speak at the International Symposium of Online Journalism in Austin about how he has applied the insights from the Innovator’s Dilemma to the Deseret News in Utah, and he laid out why integration was absolutely the wrong approach to disruption.

“In industries that are being disrupted, 9 percent of companies make it,” he said. Of the 9 percent that made it, 100 percent had set up a separate disruptive business unit.

Clayton Christensen, Clark Gilbert, and a host of others have established with a fair amount of certainty that there is only really one effective way to respond to a disruptive innovation. The incumbent needs to create a new business unit that is insulated from the rest of the company. The reason for the need for insulation is that existing business values are exactly what prevents the company from responding adequately to a disruptive innovation. The company prioritises existing business relationships over unproven new ones and existing processes over those with little testing and so, unless the new business unit is properly protected from the legacy business, the incumbent’s response to the disruption will always be compromised.

The problem, obviously, is the same issue that holds back every incumbent response to disruptive innovations: the best tactics for the new context go counter to common sense established in the old context.

A few examples from publishing:

  • Separating digital from the rest feels like spinning paperbacks back out into their on business units, a step back in the ladder of progress.
  • Having a firewall between print production and ebook production might compromise quality and result in an ebook that’s out of sync in some way with the print product.
  • Print and ebooks should be a part of a unified strategy. Ebook pricing should be set with print in mind.
  • ‘We tried separating ebooks out from print—we outsourced it to a company in India. It didn’t work.’
  • ‘We’re getting the hang of producing ebooks. Our QA processes hold them to a higher standard than anybody else. Our processes are top notch and well integrated into our Indesign process.’

The problem with all of these responses is that they assume that ebooks are the disruptive innovation and the extent of how book publishing will be affected by digital. I don’t think they are—not if you take them on their own. Digital on the whole—the web, apps and ebooks—are the disruption. (Yes, this is a refinement on my earlier thoughts on the issue based on feedback and pushback I’ve received.) Only focusing on ebooks is like facing an army and thinking you can pick which soldier you want to fight while ignoring the rest.

I think the web and apps are going to be more disruptive in the long run than ebooks. Today, they are targeting the low end and less profitable niches in publishing:

  • Subscription sites for specialised, high value, fields (photography, wine-tasting, investment).
  • The integration of tools with content (e.g. birdwatching apps, travel guide apps).
  • Niche artists that use Mike Masnick’s Connect with Fans and then give them a Reason to Buy (e.g. Thrillbent and too many web cartoonists and musicians to count).

Most of these tactics start with little to no investment and then iterate in response to feedback from their customers. These new entrants also sometimes combine approaches, e.g. both a subscription site and an app, both free website (CwF) and ebooks (RtB). They are often considered to be low quality, unprofessional, or even just plain rubbish by the publishing industry.

This should not be your response to the above list:

—None of those tactics work in my bit of the world so I’m safe

This should be your response:

—That’s a scary variety of approaches. Digital seems to come up with a tailored business model and approach for every market segment.

Don’t assume that the tactic a new digital competitor would use against your segment is going to be one of the existing ones.

Because that’s what digital has over print. There is no one fixed way that digital has to use to address a particular market segment. Print has only one solution for every problem: a book. In digital you can choose the most appropriate medium (ebook, web, app, all of the above) and the most effective business model. Publishers only offer two solutions (ebooks and print) and one business model (sell the damn thing).

This has already begun to affect existing publishers in minor ways. I know of one example where what was in my opinion the most effective tactic for that genre (subscription website) was taken off the table before the conversation even started. Why? Because one of the authors was already running a subscription website in that niche and they were doing it much much better on their own than the publisher ever would have.

Like I’ve said before:

It’s unrealistic to expect profitable niches to remain uncontested.

That competition won’t be coming from other publishers but from your own authors, self-published authors, web and app developers, tech companies, and specialists looking to capitalise on their expertise without involving a middleman.

They won’t be held back by established best practices and they won’t have to worry about complementing or integrating with a print process.


Aside on ebooks

The role of ebooks in this is, I think, going to be interesting. Ebooks are an excellent complementary product. They are relatively easy to produce as long as you don’t buy into existing tools within the publishing industry (namely Word and Indesign). Making an ebook can be incredibly easy if you aren’t making a print product in tandem (e.g. step one: write in Scrivener, step two: compile to ebook). They are cheap to distribute and sell (hosting is dirt cheap, most ecommerce solutions support them with ease, only four to five major ecommerce outlets). They make an excellent Reason to Buy for a variety of creators and fields.

However, they are extremely limited as a primary product. Their design capabilities will be limited for years to come, even if EPUB3 does get properly implemented. Apps and devices based on old versions of Adobe RMSDK will be around for years, if not a decade. Old Kindles will remain in substantial use for what will seem like an eternity. For a very long time it’s going to be very difficult for anybody to roll out an ebook with ambitious design and interactivity while still retaining the ebook’s biggest advantages (cheap production, wide reach). They are bound to one very limited business model (unit sales), one that seems to becoming less and less relevant in the digital age. They are bound by a tight vertical integration in separate stacks (the Kindle stack, Adobe stack, Readium SDK stack). Their silos prevent integration into other products, apps, or websites.

Even in the publishing industry context, ebooks add the most value when they complement something else:

  1. Print and ebook bundles.
  2. Synced ebooks and audiobooks.
  3. An economically efficient replacement to paperbacks that compliment hardcovers nicely, price-wise.

On the third point, the optimum for print would be a four-layered price stack:

  1. Expensive special edition hardcover.
  2. Just a hardcover.
  3. Hardcover plus ebook at the same price as a hardcover on its own.
  4. Just an ebook.

This would almost certainly result in a sales increase for hardcovers as the bundle would suck up a lot of ebook sales. The two layers above the bundle would make it look like a really, really good deal.

In short: ebooks are a complementary product that work best when the primary value is created elsewhere.


Edited to Add:

Suw Charman-Anderson wrote a blog post prompted by the same piece but focusing on the parallels between news media and the publishing industry. Recommended.

Make ebooks worth it

Turns out this post is a part of an impromptu series of blog posts. Didn’t plan it this way:

  1. Ebook silos and missed opportunities
  2. Ebook silos, update
  3. Ebooks and cognitive mapping

There’s a theme been running through all my blog posts this week. In fact, a single theme runs through all of my writing on ebooks; the driving idea behind all of my thoughts on the subject.

And, no, it isn’t ‘behave like an arsehole to people and criticise everything.’

The idea is very very simple:

We have to make ebooks worth it.

Print has massive benefits that shouldn’t be discounted. Ebooks have massive downsides that shouldn’t be discounted. The web provides a lot of what ebooks purport to provide.

We are facing the very real risk of limiting ebooks to the role, market position, and capabilities of mass market paperbacks. Remove paperbacks. Add ebooks. Keep the overall system the same with few changes, maybe a bit of consolidation.

This is what a lot of people and companies in publishing want and it would be a tragedy of massive proportions; the biggest lost opportunity in the history of digital media.

Fortunately there’s still time, still a chance, or otherwise I would just shut up.


What would make ebooks worth it?

  • A diversity of new modes of reading.
  • A wealth of new tools for reading and writing that are impossible in print.
  • The ability to enable new modes of learning and skills development (just adding interactive quizzes is a massive bankruptcy of imagination).
  • Democratised tools of publishing. It’s still too difficult to create good looking ebooks and distribute them widely.
  • A more peer-like, less hierarchical, relationship between the reader and writer.
  • A more symmetrical relationship between reading and writing. Reading, annotations, quotes and more should feed directly into writing systems.
  • A greater variety of models for how we extract value from writing, from gift-giving (pay-what-you-want) to subscription to dynamic pricing (like automatically increasing or decreasing prices the more people buy to create either scarcity or abundance, depending on what you want).

It won’t be worth it if all ebooks end up capable of doing is replicate print books with a bit of added javascript and video bling.

It won’t be worth it if all of the platforms keep offering only one business model for ebooks.

It won’t be worth it if the platforms keep every reader’s contributions, notes, and highlights under lock and key.

It won’t be worth it if ebooks become a inferior, partially incompatible, fork of web standards chosen only by the publishing industry.

It won’t be worth it if people keep having to go back to print because ebooks don’t have the capabilities or affordances necessary.

It won’t be worth it if we all switch away from ebooks to the web in a couple of years time because they just don’t do what we need.

Ebooks are only worth the effort if they are equally capable to the web in a unique and complementary way.

Ebooks are only worth the effort if they become something more than what they replace.


Edited to add:

Of course ebooks as they currently exist are fine for many people. But those who assume that this is acceptable are also assuming a stable media industry. In entering the digital arena books (e- or not) are brought into direct competition with not only other time wasters (games, video, etc.) but other forms of reading, namely the web and apps.

If the ebook ecosystem cannot support a diversity of content and interfaces, the web and apps will step in to fill the gaps.

They have already begun to take over areas of specialised analysis. You currently can find wine-tasters, lens reviewers, and economists running subscription websites with writing, analysis, charts, and data that is usually unavailable in either print or ebook form.

It’s unrealistic to expect profitable niches to remain uncontested. During the print era, most of the world’s expertise on how to target print media was consolidated among the world’s publishers. But these same entities are, if anything, among the more clueless companies around when it comes to digital, the web, and apps.

So, in the long term I don’t think it’s a question of publishers holding us back but of them slowly being encircled and swamped by outside competitors, leaving only the subsidised and unprofitable uncontested.

Aside:

Many non-fiction titles that are written to help the reader solve a problem could be replaced with a well-designed tool (or an extension to an existing tool) that enables the reader to easily solve the problem directly. And if the tools developer does their job properly, the tool can be designed specifically to make skills development natural and easy. Imagine a book on photoshop replaced by a series of interactive tutorials built as a photoshop extension: learning takes place within the context where the skills are to be applied.

Ebooks and cognitive mapping

The following series of tweets by @seriouspony (Kathy Sierra) tie into a subject I’ve been thinking about a lot lately, namely cognitive mapping.

The problem

To quote Wikipedia:

A cognitive map (also: mental map or mental model) is a type of mental representation which serves an individual to acquire, code, store, recall, and decode information about the relative locations and attributes of phenomena in their everyday or metaphorical spatial environment.

The spatial environment in this case is the book itself. It is a three-dimensional object that lends itself very well to this kind of mental model.

Like so many other readers, cognitive mapping is a major part of how I remember what I read and it’s a technique that’s utterly useless when you’re reading an ebook. I read a book and a part of how I remember what I remember is spatial mapping. I remember roughly how far into the book the passage was and where on the page. Combined with copious usage of small post-its this makes it very easy to remember passages and reacquire them when needed.

I know that for a lot of other people (my students, for example, back when I was a teacher) cognitive mapping helps them keep track of the book’s ideas and argument as they go forward. Without it a lot of them lose the thread they are following through the book and, as Kathy put it, ‘fail to carry context forward’.

The other key to the memory puzzle is to stop on a regular basis, close the book, and think about what you’ve just read. Go over the salient points in your head and if you have that nagging feeling that there’s something you’re missing, go back and look for it. This tactic fortunately works for ebooks.

It’s not that I have problems remembering things. I can remember most things I read without any effort (much to my sister’s annoyance who says that I have a ‘glue-brain’). What these techniques help me to do is to contextualise the information. The cognitive map preserves for me the overall context of the book (where a passage appeared, where it was in the extended argument, what line of thinking preceded and succeeded it, etc.). Closing the book and going over the book’s points places the book’s ideas in my own intellectual context; I remember them better because I have connected them with my own preexisting ideas.

I rely on these techniques so much that when it comes to reading books that contain ideas I want to remember, I have become hesitant to read them as ebooks. For example, I’ve been reading through John Gray’s books lately (False Dawn, Straw Dogs, and Black Mass so far, all excellent, although Straw Dogs was the best of the three) and I made sure to buy them in print. I made the mistake of buying and reading Taleb’s Antifragile on the Kindle and will probably end up buying the print version and reading it again so that I can retain more of it.

I’ve actually done this several times: bought an ebook, read it, then bought it again in print to read properly. (Bundling would be awesome for me. Read the book in print and still have access to full-text search.)

It takes a lot to make a longterm ebook fanatic, one who genuinely loves the format, to lean towards print but, when it comes to remembering stuff, ebooks are for me a bit crippled. A good search facility in an ebook reading app helps to reacquire a passage you already remember but not for carrying context forward or remembering the passage’s context.

The solution

Kathy’s perspective on this is trying to figure out how to make awesome books and in that context she is absolutely right. New books intended to provoke skills development in the reader should be written to remove the need for techniques such as cognitive mapping. They should absolutely carry the context forward through writing and design. We know so much more now about how memory is formed, how skills are developed, and how the mind works than we used to and we have an obligation to use this knowledge to make new books better. There is no downside to that.

However…

(You knew there would be a however, right?)

We can’t rewrite old books. You can’t rewrite John Gray and add sections at the start of every chapter that carries context forward. You can’t add this stuff to Seneca or Schopenhauer. So, I have to respectfully disagree with Kathy on the need to find a tech solution. The point is that this isn’t an either-or choice. We both have the duty to make better books and content and we need to improve the experience for reading older books. Our duty to preserve existing thought is equal to our duty to make better books.

The problem is that I haven’t the faintest idea on how to address this in the ebook reading app. It’s a problem that requires a lot of research because our instincts on what helps memory formation are likely to lead the app developer astray.


ETA:

To those of you who doubt cognitive mapping in the print book context:

Have you never read through a book, then held it in your hands, recalled a passage, and been able to guess roughly where it is in the book, down to whether it was on the right hand or left hand page? And you’ve been able to do this without remembering what exactly the passage looked like?

That’s cognitive mapping. It isn’t an abstract phenomenon or an artificial mnemonic technique but a side effect of our interaction with a three-dimensional object. It’s an aid. Your memory isn’t crippled or compromised without it, but can and does improve recall and help you keep track of things as you read. Especially if you practice it deliberately and add aids such as small post-its regularly as you read. Those post-its aren’t just markers but also landmarks, you’re able to remember where a passage was in relation to its nearest post-it.

Also note that your memories in this case aren’t entirely visual, as in you don’t necessarily remember what the passage looked like. This is one of the many reasons why progress bars in ereaders can’t serve the same purpose. Those progress bars aren’t visible all of the time (so chances are with any given passage that the bar wasn’t visible when you read it) and even if it was, you aren’t likely to remember it unless you are one of those rare persons with extremely strong visual memory.

Ebook silos, update

Yesterday, I wrote a post on ebook silos and missed opportunities.

Some people seem to be missing the core criticism in the post. The problem, as I see it, is in the infrastructure and in the market we have in place. This is not a lament for how lazy people must be or how stupid existing developers are for not implementing these things already.

Existing ebook reading apps are bland out of necessity. The ones implemented by silo owners need to appeal to and be useful to everybody. They can’t not be generic. Specialisation is not an option when you need to appeal to the lowest common denominator.

The independents have a different problem. They need to both implement detailed support overly complex file formats and they need to target one of the silos (generally Adobe’s playground, which is a fraction of the overall market).

Because the market involved is so small and the end result is that their apps have to have broad appeal. A specialised app targeting a fraction of a fraction of a market is not economically viable when you are employing a team of developers.

The solution to this conundrum is both simple and difficult (like so many simple things are).

Generally speaking, the way to promote variety in any given software genre is to make sure that it’s possible for a solo developer or a small team of developers to create and maintain an app in the genre. Only a solo developer (maybe a duo) can hope to make a living by addressing a fraction of a fraction of a market.

I’m convinced that, if a solo developer found a way to implement one a specialised ereader that was useful and valuable to readers, they’d be able to make a decent living from said app.

One such example is Marvin. It’s an ereading app that focuses on configurability, text analysis, Dropbox support, proper annotations export, and a lot lot more. It is quite different from most other ebook reading apps in the market and is quite excellent. It’s not for everybody, but it is unique enough for there to be a large group of people who both have been waiting for an app like this and are eager enough to use it that they are willing to break the DRM on their existing library to use it.

Anybody who is in any way dissatisfied with their current ebook reading app owes it to themselves to try it out.

Which brings me to the compromises needed to make the byzantine mess that are ebook file formats manageable by a small team. Marvin offers a few pointers in that direction:

  • No DRM support.
  • Only EPUB 2.0 support to begin with.
  • Limited support for ebook design and styling, which is okay for a small app because all of its users opt in to no styling. They know what they are getting.
  • No support for video or audio.
  • No FXL support.
  • No PDF support.
  • No attempt to achieve full spec coverage but instead a laser-like focus on implementing features that crop up in customer feedback.

Now, Marvin owes a bit of its success to luck, but I still think it offers us hope for how things could go in the future. Solo developers and small teams will experiment with apps. Some will fail. Some will succeed.


—Couldn’t Readium SDK help?

Possibly, but that’s not what it’s for. As I see it, the primary goal of the Readium SDK is to get large corporations who are competitors to collaborate on improving their support for the various incredibly complex flavours of EPUB (2, 3, FXL). This is hard enough without trying to address the needs of solo developers as well.

The cost of acquiring a license usable in a commercial product bears this out. After September you need to pay $60 000 to join and get a license or you need to contribute work and source code. Both are going to be way beyond the means of a solo developer attempting the already difficult and risky task of a specialised ebook reading app.

And that’s okay. Readium SDK has an already unenviable goal (achieve full format support in all major big-corp generic readers). Maybe once that’s achieved and all major apps have shipped with full EPUB3 support, maybe then the foundation will reassess the licensing terms. Before then they are more than justified in focusing on trying to solve the problem they set out to solve.


—What would really help?

More publishers going DRM-free would help a lot, especially those catering to less price-sensitive specialist subjects. That would open up the market for a developer to enter with a specialised app catering to those readers.

In fact, O’Reilly’s pioneering work with DRM-free might well be an opportunity for a developer-oriented ebook reading app, one with built in REPLs and consoles for various languages. Imagine being able to run and play with example code in a variety of common languages from a dev ebook just with a tap. Imagine having access to each platform’s documentation as you read a book on a specific problem area; having access to the official documentation for a method with just a tap.

More comics publishers following the example of Image Comics (they sell DRM-free comics direct, not enough of them yet, but it’s a start) or Thrillbent might open up the field for a specialised comics-oriented ebook reader, one that only supports FXL ebooks and PDFs. They might even be justified in only supporting PDFs and CBRs.


I’m convinced you don’t need full EPUB3 or DRM support to create an excellent app that is sufficiently useful to enough people to be a viable business for a tiny team.

The app would have to be specialised and solve very specific problems for very specific group of people with very specific needs. Just making an ebook reader with cool and unusual features isn’t going to work again, even if it did work for Marvin. You need to both delight and remove pain. Delight alone will not do.

I’m not saying it would be easy. It’s clearly a difficult problem. But it would be interesting.

Ebook silos and missed opportunities

ETA: I’ve posted a followup to this post that hopefully clarifies things and offers a few suggestions: Ebook silos, update.


Ebooks can be transformed by context. Print books cannot. No matter where you take the print book, no matter what room you read it in, it will remain in the same form and have the same affordances as it did on the day it was first stacked in the bookstore.

An ebook could, in theory, be reformed, rebound, and recast at will. A writer who is reading for research could open it in an ereader specifically designed to enable writing and integrates directly with writing tools. A student could use a specialised ereader that is full of mnemonic tools, structured note-taking, and export functions that integrate with common reference management software. A genre fiction reader might use an app that stamps all ebooks into the same aesthetic template, configured by the reader into the form they consider ideal.

It’s easy to imagine how these would work:

The writer’s ereader wouldn’t atomise the reader’s annotations but would present the annotations for a book as a single, freeform, document that could be edited, extended, filled with notes and exported in formats compatible with common writing tools (Word, Scrivener, etc.). Every set of annotations would be a commonplace book in machine-readable form. Bonus points for automatically syncing to Evernote and Dropbox.

The student’s ereader would integrate directly with Endnote and offer mnemonic features such as regular pauses for recollection, forcing the reader to note down what they thought the preceding pages were about (research shows that if you stop, close the book, and try to recollect what the preceding pages were about your recall will subsequently be improved). There’s a lot of research on learning styles and tools that would be a goldmine for this sort of UI experimentation.

Then there’s the potential for a specialised comics app. Fixed layout formats require dramatically different UIs for optimal treatment of annotations, clipping, highlighting, and browsing. It’s easy to imagine a comics ereader that makes it easy to clip and visually annotate sections from the comic book. It’s also easy to an app that specialises in the format would offer a much better experience than the schizophrenic status quo.

The genre reading app does not have to limit itself to design configurability. A reading app for crime and mystery stories could integrate an extensive database of firearms, poisons, historical crimes, police terminology, and common codes for crimes.

That’s without getting into the various features that nobody offers because everybody’s trying to be the same:

  • Scrolling instead of pagination. I’d jump on a well-implemented scrolling ebook reading app (iBook’s jerky crap is not it).
  • Autoplay/autoscrolling. Sometimes you want to force yourself to keep a reading pace.
  • In-text annotations. Sometimes I want to edit the actual ebook itself.
  • Dropbox syncing for my library and annotations.

None of these features require new standards or extensions to old ones. They could work with plain text if you wanted.

An app that is everything to everybody is bland. Generic is dull. Specialisation creates immense value.

But instead of specialisation and diversification among ereaders we see convergence. Ebook reading app become more similar with every release. They all aim to support the same rendering features, in roughly the same way (infuriating differences in style overrides notwithstanding), surrounded by the same constellation of widgets and tools.

  • Highlights? Check.
  • Highlights made more ‘natural’ by behaving like a highlighter? Check.
  • Notes? Check.
  • Notes mades social in some way, via sharing or a dedicated service? Check.
  • Highlights that lose all formatting whenever they are moved into another context? Check.
  • Offer a selection of four to five fonts (plus the ever present ‘publisher defaults’)? Check.
  • Sync all of this bland crap using a proprietary syncing service allowing no other alternatives.
  • Limit the export of all of this bland crap to something even blander and more useless than what you already offer like text-in-body email.
  • A neato brightness UI that makes people swoon because their Stockholm syndrome has lowered their expectations so far that they have to look up to see the bottom of the Mariana Trench.

Check. Check. Check. Any ebook reading app that doesn’t behave like this is aiming to. This is the ‘ideal’ they all seem to be striving towards and when pressed for answers on why they don’t try to solve hard problems (like proper annotations export), the only reply is that the standards for those hard problems aren’t there yet.

And they will never be there because the best standards are those that standardise existing best practice. Nobody offers proper annotations export and so there is nothing to standardise.

But just focusing on the individual features is the wrong way to look at the problem.

Which, obviously, is silos. You’re locked into the ecosystem you bought the ebook from. Nobody will ever create a specialised Kindle ebook reading app for writers or for students. There will never be much variety in how ebook reading apps based on Adobe’s RMSDK behave. The only app that will ever work with ebooks bought from the iBookstore is iBooks.

The one major and unique strength that ebooks have over print, flexibility and fluidity, the characteristic that has the biggest potential for adding value, has been thoroughly walled away by the silo mentality. Ebooks could have been a transformative sea change in how we read books but instead are nothing more than a second-rate alternative to cheap paperbacks.