Winner takes all versus the Matthew effect

Winner takes all

There’s a vague notion going around. For some it’s a suspicion, for others it has become a certainty, the rest of us worry and hope it’s wrong.

It’s the idea that the internet exaggerates the sales inequality of media markets. That, by massively enabling word of mouth and social networking, the web means that we will only get mega-bestsellers or flops, with little to nothing in between. The market becomes just bestsellers leaving the long tail with scraps.

In theory, this should be a simple question to answer. Somebody with access to detailed numbers from the market could calculate the Gini coefficient for book sales revenue over the years. If it used to be lower and is now almost one (or higher, since it’s theoretically possible for books to have negative revenue through returns) then we probably have a winner takes all situation on our hands.

If, however the Gini coefficient has remained the same or stays broadly equal to society’s income Gini coefficient, then we probably just have a regular “the world isn’t fair, boohoo” situation and there’s no need to blame the internet.

My suspicion is that the book market is only about as unequal as the economy in general, that the sales difference between J.K. Rowling’s books and the rest is about the same as the wealth difference between the top 1% and the plebeian masses (us).

But, something has changed!

Everything changes, all the time. In uncontrolled circumstances you can’t reliably map specific changes and claim that one causes the other. A lot of the time when we do that we get it the wrong way around (if we’re lucky), and end up claiming that wet pavements cause rain.

There is a related concept that might explain some of the sales patterns we’re seeing but, again, it’s hard to come up with conclusive proof given that big data lets us see what we want in the numbers and a market doesn’t lend itself to double blind experiments.

It is something that has been observed in plenty of other systems.

The Matthew effect

The idea is very simple: the rich get richer and the easiest way to get more popular is to be popular in the first place.

How it would work in a market could be described like this: every sale of a copy of a book increases the probability of selling other copies independently of other variables in the market.

If you couple it with Reeds Law which states that “the utility of large networks, particularly social networks, can scale exponentially with the size of the network” then you get this:

If a book has sold twice the number of copies of another book, it will have four (22) times the sales clout of the lesser selling title. A book that has sold ten times more will have a hundred times the sales clout. And so on.

What does this mean for publishing?

If this theory is even remotely true this has several major consequences:

  • Minor and random variations early on in a title’s sales history can snowball it into a bestseller. There won’t be any logical rhyme or reason for this and predicting these successes will be impossible because they are completely stochastic.
  • Hopping onto known successes (i.e. pushing an already big snowball further down the hill) will have much bigger payoffs than building up sales from scratch. Why go with the lottery ticket probabilities of a new author or completely new title when you can earn so much more by turning a one million dollar bestseller into a ten million dollar blockbuster?
  • Since the big payoffs are governed by randomness and the moderate payoffs by hard work, publishers have an incentive to cut down on the hard work (editorial, acquisition, design) and focus exclusively on the logistics of printing and shipping shit-tons of Fifty Shades of Grey (or whatever the next big thing is).
  • Self-publishers and small publishers become responsible for the research, development, and discovery of new successful titles. And once a title is a proven thing, a big publisher will swoop in and buy it up, promising the author more money than they’d get doing it alone or with a small publisher.
  • The big losers are small to medium-sized publishers who do all of the R&D but don’t have the resources to scale sales up into the stratosphere when they hit a success.
  • The winners are self-publishers and self-pub coops who can build sales up the slow and hard way for as long as it takes and keep a lion’s share of the upside in the unlikely event that they do find a major success. The bargaining position of an already successful author has never been this strong in the history of publishing.

Or, of course, it could all just be wild speculation based on a wild theory with no basis in reality. Time will tell.

Followup to ‘this ebook is a lemon’

There have been a few responses to my ‘This ebook is a lemon’ post earlier. Most of them either omit or misunderstand details from the post, which means that I probably wasn’t clear enough in the original. So here is a followup with a few clarifications based on issues raised.

It’s not an analogy

I was not comparing ebooks to cars. I was, like Akerlof in his paper, using cars to explain the ‘market of lemons’ dynamic. The other examples Akerlof used in his paper were from insurance and employment but the used car example was both simpler to explain and had the added benefit of being real-world evidence of the model.

Why didn’t I use ebooks to explain the theory? Because, as I stated in the post, I’m not 100% certain ebooks fit the model. It’s better to explain a model using examples that fit perfectly and then let people apply that understanding to the problem area at hand.

Well, except for the fact than none of the commenters or responders seem to have done that.

No, not all cars or ebooks suck, that’s not the point

Another common misunderstanding was that I was claiming that all ebooks were crap or that all cars from a certain era were crap. Absolutely not. The entire post and Akerlof’s model hinge on information asymmetry. The buyer can’t tell beforehand whether any given product is a lemon or not while the dealer can. Amazon, for example, has access to abandonment rates and return rates, as well as the ability to data-mine notes on a book for negative keywords. A publisher knows quite well how much effort they have put into a book. Dunning-Kruger aside, most publishers who are dumping crap on the market know it’s crap.

This information asymmetry is what gives rise to the bad publisher’s incentive, the customers demand for a lower price, and the good publisher’s disincentive. The bad publishers don’t take over the market until late in the process and I stated quite clearly in the post that I think we’re at the start of the process, not the end.

So, the point isn’t that all cars or books are bad. The point is that the buyers know that there are lemons in the market (might even have bought one or two in the past) but can’t tell if any of their current options are lemons or not. The existence of lemons combined with information asymmetry is what creates the dynamic.

“The returns policy you suggest is insane”

Possibly. There are only two ways to break the market of lemons dynamic:

  1. Information symmetry. The customer gets access to all of the information needed to help tell lemons from the rest.
  2. Returns and warranties. Which is an artificial way of shifting risk back onto the provider.

So, you need either a massive returns policy or information symmetry. I don’t think you’d need both.

And as to whether publishers could survive that sort of returns policy for ebooks. Of course they can, they live with that same policy for print.

You don’t have to work in publishing for long to see at least one sales forecast completely destroyed by a series of booksellers returning a book months later. Publishers are already geared for this kind of uncertainty.

“This will only impact self-publishers and push readers to big publishers”

Possibly. But what worries me is big publishing seems to be relaxing their quality standards (or they never had them, as in the case of ebook production). More and more they are acquiring new titles from self-publishing or fan-fiction, which would be fine if they were going for the good titles (of which there are plenty in both circles) but they are manifestly going for the crap most of the time.

But, yes, this could result in a two tier market where big publishers get away with charging $10 and everybody else can’t give it away.

“You didn’t mention X free bla”

I stated a general principle of the effects of free. If somebody disagrees with a common sense statement, bringing in examples won’t sway them, they will just come up with excuses for disregarding those examples.

“Piracy solves the problem”

One of the most interesting observations on piracy was Tim O’Reilly’s statement that ‘piracy is progressive taxation’. That is, it disproportionately impacts the more successful.

A corollary to that is that the promotional and marketing benefits of piracy disproportionately go to the more successful. Try to pirate anything beyond the recent and mainstream and you will run into difficulties. Most people know how to pirate a current TV series. They don’t have a clue of how to find anything less popular.

Moreover, piracy is a function of interest. You won’t pirate something you aren’t interested in. So, any product that has generated enough interest to benefit from the distribution effects of piracy has also generated enough interest to break out of the ‘market of lemons’ dynamic.

"DRM-free solves the problem

It’s a start since it does take away some of the long term risk for the buyer (‘will I lose this book in the future due to DRM?’) but it doesn’t address the basic information asymmetry.

“Prime members read for free”

Only for Prime members in the United States and the Lending Library only has a fraction of the books on the market.

The Lending Library may well be part of the solution as it lets people read as much of the ebooks as they want, but it comes with it’s own issues. Namely, most publishers can only get into it by giving Amazon exclusivity, which would make the Lending Library a massively iatrogenic solution. The cure would be worse than the sickness.

“The bad publishers will drop out”

Given that bad publishers have more incentives to publish than the good and that bad publishing requires next to no investment, this problem will not be solved by bad publishers dropping out of the market as their ‘get rich quick’ schemes fail. The incentives will make sure that several new bad publishers will be lining up to replace every one that drops out.

“You can easily tell whether a car is a lemon or not”

The idea here being that the car simply either works or not—it’s quality apparent just with a test drive.

A lemon is not a car that doesn’t work or handles badly. A lemon is a car or product that fails later on, after purchase. A car with a history of specific kinds of repairs and failures is more likely to fail than others (and is a lemon). This is the reason why used car dealers in many countries are required to disclose a car’s repair history. Those laws are in effect because of Akerlof’s paper. They weren’t on the books when he first wrote it.

Akerlof’s theory is not about ‘used’ versus new products

The used car market was just an example. Whether the cars were new or used doesn’t matter for the theory. The dynamic is orthogonal to the newness or how used the products are. That’s why his other example was a non-rival product like insurance.

“One man’s lemon is another’s lemonade”

Sure, that’s why I said that the following was one of the basic premises of the post:

Quality in this piece being defined as whatever the reader values, no matter how rubbish it looks to an over-educated twit like me. I’m not making any assumptions about writing, genre, or style.

That means that when I’m talking about quality I’m talking about it from the perspective of the buyer, e.g. a romance reader looking for romance ebooks. A lemon in this context is what that reader considers to be an unacceptably bad romance book, not a scifi or crime novel.

And I also stated that if you disagree with any of the premises then you really don’t have to read the rest as you’ll almost certainly disagree with it. I meant it. If you already disagree with the premise why bother responding to the post? I’ve already given you an out.