Technology is not inherently good

I’ve never meet a self-proclaimed geek who understands this. Technology is not something that’s inherently good, where more of it solves more problems and improving it improves our lot. If we implement servile AIs and pervasive automation, that won’t be used to create a society of abundance and leisure but to make the rich richer while the unemployed starve. Technology is something that needs to be applied and generally reflect the economy and culture it was developed in.

This means that a society geared towards inequality and inequity will use technology to amplify them. This means that a police state will use it to decrease the freedom and privacy of the citizens. Theocracies will use technology to hunt unbelievers.

Technology does not make the unfair fair and it does not right wrongs. It is a tool and the only way to change the world is to first change the people who wield it.

ETA: Athena Andreadis made this here excellent point over on twitter:

Followup to ‘this ebook is a lemon’

There have been a few responses to my ‘This ebook is a lemon’ post earlier. Most of them either omit or misunderstand details from the post, which means that I probably wasn’t clear enough in the original. So here is a followup with a few clarifications based on issues raised.

It’s not an analogy

I was not comparing ebooks to cars. I was, like Akerlof in his paper, using cars to explain the ‘market of lemons’ dynamic. The other examples Akerlof used in his paper were from insurance and employment but the used car example was both simpler to explain and had the added benefit of being real-world evidence of the model.

Why didn’t I use ebooks to explain the theory? Because, as I stated in the post, I’m not 100% certain ebooks fit the model. It’s better to explain a model using examples that fit perfectly and then let people apply that understanding to the problem area at hand.

Well, except for the fact than none of the commenters or responders seem to have done that.

No, not all cars or ebooks suck, that’s not the point

Another common misunderstanding was that I was claiming that all ebooks were crap or that all cars from a certain era were crap. Absolutely not. The entire post and Akerlof’s model hinge on information asymmetry. The buyer can’t tell beforehand whether any given product is a lemon or not while the dealer can. Amazon, for example, has access to abandonment rates and return rates, as well as the ability to data-mine notes on a book for negative keywords. A publisher knows quite well how much effort they have put into a book. Dunning-Kruger aside, most publishers who are dumping crap on the market know it’s crap.

This information asymmetry is what gives rise to the bad publisher’s incentive, the customers demand for a lower price, and the good publisher’s disincentive. The bad publishers don’t take over the market until late in the process and I stated quite clearly in the post that I think we’re at the start of the process, not the end.

So, the point isn’t that all cars or books are bad. The point is that the buyers know that there are lemons in the market (might even have bought one or two in the past) but can’t tell if any of their current options are lemons or not. The existence of lemons combined with information asymmetry is what creates the dynamic.

“The returns policy you suggest is insane”

Possibly. There are only two ways to break the market of lemons dynamic:

  1. Information symmetry. The customer gets access to all of the information needed to help tell lemons from the rest.
  2. Returns and warranties. Which is an artificial way of shifting risk back onto the provider.

So, you need either a massive returns policy or information symmetry. I don’t think you’d need both.

And as to whether publishers could survive that sort of returns policy for ebooks. Of course they can, they live with that same policy for print.

You don’t have to work in publishing for long to see at least one sales forecast completely destroyed by a series of booksellers returning a book months later. Publishers are already geared for this kind of uncertainty.

“This will only impact self-publishers and push readers to big publishers”

Possibly. But what worries me is big publishing seems to be relaxing their quality standards (or they never had them, as in the case of ebook production). More and more they are acquiring new titles from self-publishing or fan-fiction, which would be fine if they were going for the good titles (of which there are plenty in both circles) but they are manifestly going for the crap most of the time.

But, yes, this could result in a two tier market where big publishers get away with charging $10 and everybody else can’t give it away.

“You didn’t mention X free bla”

I stated a general principle of the effects of free. If somebody disagrees with a common sense statement, bringing in examples won’t sway them, they will just come up with excuses for disregarding those examples.

“Piracy solves the problem”

One of the most interesting observations on piracy was Tim O’Reilly’s statement that ‘piracy is progressive taxation’. That is, it disproportionately impacts the more successful.

A corollary to that is that the promotional and marketing benefits of piracy disproportionately go to the more successful. Try to pirate anything beyond the recent and mainstream and you will run into difficulties. Most people know how to pirate a current TV series. They don’t have a clue of how to find anything less popular.

Moreover, piracy is a function of interest. You won’t pirate something you aren’t interested in. So, any product that has generated enough interest to benefit from the distribution effects of piracy has also generated enough interest to break out of the ‘market of lemons’ dynamic.

"DRM-free solves the problem

It’s a start since it does take away some of the long term risk for the buyer (‘will I lose this book in the future due to DRM?’) but it doesn’t address the basic information asymmetry.

“Prime members read for free”

Only for Prime members in the United States and the Lending Library only has a fraction of the books on the market.

The Lending Library may well be part of the solution as it lets people read as much of the ebooks as they want, but it comes with it’s own issues. Namely, most publishers can only get into it by giving Amazon exclusivity, which would make the Lending Library a massively iatrogenic solution. The cure would be worse than the sickness.

“The bad publishers will drop out”

Given that bad publishers have more incentives to publish than the good and that bad publishing requires next to no investment, this problem will not be solved by bad publishers dropping out of the market as their ‘get rich quick’ schemes fail. The incentives will make sure that several new bad publishers will be lining up to replace every one that drops out.

“You can easily tell whether a car is a lemon or not”

The idea here being that the car simply either works or not—it’s quality apparent just with a test drive.

A lemon is not a car that doesn’t work or handles badly. A lemon is a car or product that fails later on, after purchase. A car with a history of specific kinds of repairs and failures is more likely to fail than others (and is a lemon). This is the reason why used car dealers in many countries are required to disclose a car’s repair history. Those laws are in effect because of Akerlof’s paper. They weren’t on the books when he first wrote it.

Akerlof’s theory is not about ‘used’ versus new products

The used car market was just an example. Whether the cars were new or used doesn’t matter for the theory. The dynamic is orthogonal to the newness or how used the products are. That’s why his other example was a non-rival product like insurance.

“One man’s lemon is another’s lemonade”

Sure, that’s why I said that the following was one of the basic premises of the post:

Quality in this piece being defined as whatever the reader values, no matter how rubbish it looks to an over-educated twit like me. I’m not making any assumptions about writing, genre, or style.

That means that when I’m talking about quality I’m talking about it from the perspective of the buyer, e.g. a romance reader looking for romance ebooks. A lemon in this context is what that reader considers to be an unacceptably bad romance book, not a scifi or crime novel.

And I also stated that if you disagree with any of the premises then you really don’t have to read the rest as you’ll almost certainly disagree with it. I meant it. If you already disagree with the premise why bother responding to the post? I’ve already given you an out.

This ebook is a lemon

It’s no secret that I’m a huge fan of both Akerlof and Romer, not just the paper they co-wrote on looting in the financial system but also their work individually. Turns out one of Akerlof’s most famous papers is directly relevant to the ebook market.

For starters, a few basic premises. If you disagree with any one of these you can feel free to ignore the entire argument. I can easily pick apart any one of these statements myself, so I’d understand it very well if you disagreed with them. However, if you find them somewhat likely then the overall picture of the ebook market is a bit dark.

  • Ebook buyers buy more than they read. Book abandonment is high and out of proportion with the return rate.
  • Sampling the first few chapters is a lousy predictor of how much the reader will enjoy the book. You can only assess basic stylistic issues from a sample, not storytelling quality. Ergo the reader has to buy the ebook to assess the quality of the story.
  • Reviews are an extremely unreliable indicator of quality. The average quality of most reviews themselves is very low. Many reviewers are paid shills or just extremely overworked.
  • Luck is one of the biggest determinants of bestseller status.
  • Striking, marketable, differentiation is difficult in ebooks without having the reader actually read the book.
  • The marketing differentiation that is possible without having the reader actually read the ebook (sex, scandal, celebrity) is at best orthogonal to the book’s actual quality and at worst inversely correlated to quality.
  • Quality in this piece being defined as whatever the reader values, no matter how rubbish it looks to an over-educated twit like me. I’m not making any assumptions about writing, genre, or style.
  • The majority of ereader vendors implement style and design overrides to preserve a baseline of readability and usability, not to commodify their product’s complements. (I.e. they are well-meaning, rational actors.)
  • Distribution is becoming mostly self-serve with a very porous filter. (Like, for example, the self-publishing services run by Amazon, Kobo, B&N, and Apple.) Almost anybody with a computer has access to the publishing industry’s full ebook distribution chain.
  • Ebook development is underpaid and so will not attract experienced talent from the web industry.
  • It’s easier to make a bad book than a good one and so the vast majority of ebook supply will be bad.

The argument I’m about to make is that this situation gives publishers (both self- and non-self) an incentive to market poor quality books (remember the definition of quality I outlined above), that the average available quality of books will fall, and that the overall publishing market will shrink in terms of overall revenue (even though the the number of units sold increases).

First, the basics…

A market of lemons

It has been seen that the good cars may be driven out of the market by the lemons. But in a more continuous case with different grades of goods, even worse pathologies can exist. For it is quite possible to have the bad driving out the not-so-bad driving out the medium driving out the not-so-good driving out the good in such a sequence of events that no market exists at all. (Akerlof, 1970)

In 1970 Akerlof published a paper describing exactly why a new car loses a lot of its value as soon as you drive it newly bought out of the showroom.

With new cars you can assume that most cars of the same make will be of a similar quality and that if something goes wrong you are probably covered by a warranty.

Anybody who is in the market for a car doesn’t have access to the information that would let them tell the difference between a good car and a bad car (otherwise known as a ‘lemon’, hence the title of the paper).

If a new car is a lemon, i.e. has some sort of manufacturing flaw, neither the buyer or the seller are aware of the flaw and the manufacturer usually covers the repairs if the flaw is discovered within the warranty period.

However, because used cars have a history, the seller of a used car is likely to know which are lemons and which aren’t. This asymmetry of information in the used car market is, according to Akerlof, the primary reason why there is a pricing disparity between a new car and even the most recent of a used car.

Both bad and good used cars are likely to sell for the same price because the buyer can’t tell the difference between those that are lemons and those that are not. And because the buyer knows this there will be s strict upper limit to what they will pay for a used car—the seller of a good used car will not get its full value.

The sellers of good cars have an incentive not to sell while the less well-meaning sellers of lemons have an incentive to sell. The information asymmetry means that less scrupulous will, at least while the market is still maturing, get more than their ‘lemon’ car is worth.

The Lemons model can be used to make some comments on the costs of dishonesty. Consider a market in which goods are sold honestly or dishonestly; quality may be represented, or it may be misrepresented. The purchaser’s problem, of course, is to identify quality. The presence of people in the market who are willing to offer inferior goods tends to drive the market out of existence- as in the case of our automobile “lemons.” It is this possibility that represents the major costs of dishonesty – for dishonest dealings tend to drive honest dealings out of the market. There may be potential buyers of good quality products and there may be potential sellers of such products in the appropriate price range; however, the presence of people who wish to pawn bad wares as good wares tends to drive out the legitimate business. The cost of dishonesty, therefore, lies not only in the amount by which the purchaser is cheated; the cost also must include the loss incurred from driving legitimate business out of existence. (Akerlof, 1970)

What worries me…

Reliable information about ebook quality is increasingly hard to find in the market. Reviews have almost completely been gamed; a casual reader has few reliable indicators that tell them whether a review is an honest one or not. Rubbish books, ones that most buyers don’t even read to the end before giving up, shoot up the bestseller lists due to viral marketing. Bestseller lists themselves are increasingly either gamed by publishers or by ebook retailers themselves who are trying to shift their sales in one direction or another.

Even some big publishers are getting into the game by dumping cheap OCR converted ebooks full of errors onto the market. Again, a casual reader has no way to know whether this particular big publisher is one that does a quality ebook version or one who pumps out ebook ‘lemons’ by the virtual truckload.

The same applies to self-publishing. The casual reader doesn’t have access to the information to help them tell the difference between the self-publisher who has invested substantially in the quality of their book and one who is dumping something onto the market looking for a quick profit that requires next to no cash outlay. That is without mentioning the publishers and authors who have been paying for reviews, engaging in sock-puppetry, and astroturfing left, right, and centre.

My worry is that the ebook market has all of the hallmarks of an early stage ‘market of lemons’. The information asymmetry—exacerbated by the information hoarding done by the big ebook retail players—the growth in dishonest actors, and the increasing disincentive for honest actors to even participate at all, make ebooks an ideal candidate for the lemon dynamic.

What this would mean, if true, is that publishers and self-publishers will begin to experience massive pressure to lower prices if they are to move their product at all.

I think this is already happening with self-publishers.

Alternatively, revenue might become a function of your reputation and your maximum addressable market. That is, once you’ve surpassed whatever necessary lower reputation bound that is required by your addressable market, the price becomes elastic within the bounds set by the market (e.g. there are only so many steampunk fans in the universe).

Below that reputation bound you will have problems even giving away your product.

If the former model is correct (i.e. the ‘we’re all screwed’ model) then the ebook market can only be saved by the ebook retailers. They’d have to begin to practice complete information transparency and put in place aggressive returns policies (yes, even more aggressive than the one Amazon currently practices). Sales information, return rate, who exactly is that reviewer and where did they come from—anything and everything about the book would have to be shared in a digestible manner. A culture of secrecy at this stage would risk killing the market off entirely.

If the latter model is correct (i.e. you can only charge when you’ve built up a reputation) then we enter a world where the publishing industry needs to learn how to engage directly with readers to build their audience. This means that they would have to give a lot more stuff away for free.

A note on free

Free creates the most value when it is specifically being used to build a community and decrease information asymmetry—transferring the burden of risk from the reader to the publisher. Short term free offers are of little use. You need to go long term and do it with work that is representative of what you do.

Free at the moment is used by exactly the crowd that churns out the most rubbish (Smashwords) and Amazon makes offering titles for free needlessly difficult.

A free offering is always preferable to a cheap offering for the seller because it suspends the buyer’s value judgement temporarily but in exchange buyers can assess the quality of the good at their own leisure. A mixture of free and higher or variable priced goods is likelier to result in a fairer exchange of value between the reader and author than an oversupply of cheap (the free offerings build reputation).

What to do?

If Akerlof’s theory is right and is applicable to the ebook market then it predicts that prices in the market will be driven down below what can sustain the good actors and investment in research and development will cease.

I.e. the books will all be rubbish, look like shit, and the big tech companies will cease to invest in their ebook platforms.

This isn’t a problem that can be solved through code or UI design. It requires a fundamental change in business tactics. You can’t solve structural business problems with code.

The biggest measure retailers could take to ease this asymmetry is an extended, no questions asked, refund policy. If a reader asks for a refund they should get it even if they bought it ages ago. A year should be enough. A market that is developing ‘lemon’ dynamics requires a generous refund policy.

(This is in addition to the radical information transparency I mentioned above.)

This would obviously shift most of the economic power back to the reader but it would also have several consequences. Low quality books would be hammered in the market. Publishers would no longer have incentive to market bad books. Prices would rise to account for the returns and the portion of readers who dishonestly return ebooks but readers would likely accept the rise because of the generous refund policy. The market would contract sharply at first as the bad actors get shaken out but would begin to grow aggressively as the good actors, who are rewarded with both a higher price and a lower return rate, reinvest their profits in product development.

That’s the theory, anyway.

I hope we are in for a world where reputation becomes the key to survival, where publishers and authors with a good reputation in a market segment can make a decent profit, because the alternative is horrifying. I don’t think it is likely that ebook retailers will take the measures necessary to correct the dynamic once it becomes more apparent that the ebook market is a market of lemons.


One thing worth mentioning:

If you assume that the above applies to the iOS app store as well then that would mean that the best pricing strategy for a new app is freemium. That is, the app itself should be free to use/play with variably priced add-ons and features you can buy once you know you like the app.

ETA 2:

I’ve posted a followup to this post addressing some of people’s criticisms and misconceptions.


Perceptions of society

Descriptions of a society is like the proverbial blind men describing an elephant; a partial insight is indistinguishable from a lie.

Iceland just had a referendum on its constitution. Over two-thirds approved and, moreover, over two-thirds voted in favour of specific reforms that they felt the new constitution had to have.

The story told about post-crash Iceland is almost like a fairy tale. A nation that threw out the banksters and cleaned up after being betrayed by its financial class. A left-wing government was voted into power. The debts of the public were written down. Financial crime was prosecuted and corrupt government officials brought to justice. A new constitution has been crowd-sourced and written by the people themselves. The government refused to kowtow to the demands of the UK, EU, and the IMF, and through unorthodox economic policies has returned to economic growth. The government has rejected austerity. They passed a media freedom/free speech law that was written with the advice of Wikileaks.

That is the story told about Iceland in foreign media.

And it’s almost entirely a lie.

Some small fry in the finance industry have been prosecuted and convicted. The big fish not only all got away but continue to own large chunks of Icelandic society. The so-called left-wing government followed the IMF playbook to the letter, receiving praise from the IMF for the work they’ve done. Most of the policies they have put into place would make Thatcher proud. Two officials were prosecuted. One got away with a slap on the wrist (no punishment). The other’s crimes were so blatant that despite the best efforts of various factions in Icelandic society, they had no option but to convict him for insider trading.

The rest all got off scot-free, despite receiving large amounts of money from the banks.

The debt write-downs were almost instantly rolled back by index-linking. Icelandic society is in the middle of the greatest mortgage crisis in its history. It’s a powder-keg waiting to explode.

The constitution wasn’t crowd-sourced but written by a committee appointed by parliament (itself largely composed of the same corrupt members of parliament who ran the place pre-crash). The committee draft, which is what the Saturday referendum was about, is non-binding. The referendum question was whether the new constitution (which will be written by MPs) should be based on the committee draft, not whether it should become our new constitution. The Independence Party, the likely winner of the election next spring and so the party that would have to implement a new constitution, is already claiming that it isn’t bound by the results of the referendum because they don’t feel like it. (Essentially, their logic is as tenuous as that.)

The Icelandic government has already paid most of the Icesave debts, despite what the media may tell you. The dispute between the UK, Iceland, and other EU countries is entirely about how much interest is owed, not about the validity of the debt itself. The supposedly left-wing government tried time after time to accept the UK/EU demands but were overruled as the President forced a referendum on the laws. Which is a Presidential power that is very likely to be dropped from the new constitution.

Yes, there’s a good chance that the gloriously democratic crowd-sourced constitution will be used to roll back the reach of democracy in Iceland.

By any calculation, the Icelandic government has engaged in unprecedented cutbacks in social services, healthcare, education, and support for the elderly.

The growth of Iceland’s economy is probably down to capital, trapped in the country because of the currency controls, overheating the real estate market.

The Icelandic Modern Media Initiative that was passed by the Icelandic parliament was a non-binding resolution, which is a handy way of disposing of things you never intend to implement. The actual media laws that were passed require the registration of every single media outlet, TV, print, radio, or website, that intends to deliver news and commentary to the Icelandic public. Failure to register will result in a fine. The same media law includes new provisions for blocking the IP-addresses of illegal content.

Most of the news you hear about Iceland is a partial truth that, like the blind man describing an elephant as a snake, becomes a lie.

The truth is that Iceland is a corrupt oligarchy that, unless it changes its course radically, is on the verge of collapse.

Why is this important to fantasy and science-fiction writing?

Because every story is composed of partial truths, a weave of limited perspectives that build a picture. Even an omniscient narrator can only describe one thing at a time.

A single society can be represented in so many different ways, can be shown to be so many different things. Who you choose to describe and follow, whose position and perspective is central in the story, is as important to world-building as the actual details of the world themselves.

The details of education and wealth won’t come into it if the lead character comes from a class without access to schools and money.

Interior decoration trends and detailed architecture is a sign of a high culture and a sophisticated society if your lead is a member of the elite but becomes an emblem of corruption and sleaze if your lead is an unprivileged labourer.

And, finally, a country can seem to be an utopia to outsiders who only stay for a few weeks when they visit, but feel like a confining dystopia to those who can’t escape it.