It’s the Cover, Stupid! Why Publishers Should A/B Test Book Covers

Expert publishing blog opinions are solely those of the blogger and not necessarily endorsed by DBW.

Audience + InsightNever judge a book by its cover. So the saying goes, yet consumers do it all the time. Every publisher and bookseller knows that covers sell books. But do consumers also form expectations from looking at the cover? Well, based on the results of some of the initial reader analytics data at Jellybooks, we think they do.

The phrase “You can’t judge a book by its cover” is attributed to George Eliot’s The Mill on the Floss (1860), in which Mr. Tulliver uses the phrase in discussing Daniel Defoe’s The History of the Devil, saying how it was beautifully bound. Eliot was of course not a man, but rather the pseudonym of Mary Ann Evans, who knew that a book was viewed differently in her time (and maybe even today?), if written by a man as opposed to a woman. Appearances matter, as first impressions shape expectations.

In this post I will discuss two recent cases in which the cover was central to the interpretation of reader analytics tests Jellybooks conducted for book publishers.

The first case shows how a cover shaped readers expectations. The cover was well designed and clearly aimed at grabbing book buyers’ attention both online and in physical bookstores. This particular cover had a single dominant color and “told a story” with a single image. But it seemed to have told the wrong story.

Users who picked the title in one of our test reading campaigns told us afterward that, based on the cover, they had expected to be reading a crime novel or perhaps a spy thriller. However, they were surprised to discover that the book was instead a work of non-fiction.

The narrative described a slice of post-war history at the Central Intelligence Agency. The spy and crime themes were there, but not in the novelized form readers had expected. As a result, the drop-off in reader engagement we measured during the initial chapters was swift and sharp (over 60 percent) when readers realized that this was not the book they had expected.

We draw two tentative conclusions:

i. when presented with a choice of 20 titles or more to choose from, test readers were often guided by the visual cues the cover provided and not the synopsis of the book, which requires more time to read and absorb, and
ii. covers shape reader expectations about the content within the book that feeds through into completion rates. The book cover is not just packaging, but a visual summary of the book itself that needs to be accurate.

The second example comes from a different reader analytics promotion. Once again, readers were allowed to choose from up to 20 titles on offer, but instead of being able to choose only one title, they were allowed to pick up to five. One title frequently chosen, though rarely as first or even second choice, was a psychological thriller with a somewhat quirky cover: a cat staring at the consumer. Astounding, though, were the results for this book relative to its sales.

The book was a debut novel from recent years for which the publisher had held very high expectations. However, sales had been poor. In a rare case of post-mortem “what went wrong,” the publisher had the courage to investigate whether the content or the publisher’s marketing was at fault. The reader analytics showed that the content was not responsible for the poor sales performance:

i. The book had an exceptionally high completion rate: more than 75 percent of those who tried the book finished it—a rate achieved by fewer than 5 percent of books that we test.
ii. The thriller had a very high velocity. Readers were glued to the pages, and a majority read the book from start to finish in just a few days.
iii. The book had an excellent recommendation factor based on the Net Promoter Score concept. Readers were strong promoters of the book, recommending it to their like-minded friends.

However, many test participants also noted that the cover was “weird.” Readers indicated that they really enjoyed discovering the book, but that based on the cover, they would never have picked it up in a bookstore. Well, why not?

The cover was clearly outside the genre conventions readers were familiar with and accustomed to.

The digital book jacket did not call out to the kind of reader who was looking for edge-of-the seat suspense drama. A cat on the cover, no matter how viciously the feline stares at you, just does not hold the promise of an edge-of-the-seat psychological thriller. It was a case of great content let down by its packaging. In other words, it was the cover, stupid!

So what are the consequences?

In the first case, the publisher learned that being too smart with the cover can backfire. The cover drew people’s attention but then disappointed relative to the expectation it raised. The content was great, but the wrapper let it down. Those who finished the book gave it high marks, but it seduced many into trying the book for which the content was “not their cup of tea.” And those readers would tell others “that is not the kind of book you want to buy.”

In the second case, the publisher is now considering how to relaunch the book with a new cover (and a lower price to draw attention). A relaunch is, of course, never an easy proposition, especially as scarce publicity and marketing resources are often soaked up by the never-ending line of new books to be released.

Now, other reasons may have been behind the poor sales, like the launch of another book overshadowing the launch of this book, bad luck, etc. But it was certainly not the content.

The best solution is to try to get it right the first time by picking the right cover. The effort spent up-front is small compared to lost sales or corrective action later.

But how do you know if you have the right cover? Test it!

Thus, we at Jellybooks are preparing to introduce A/B testing for book covers. In the 21st century, the humble cover is no longer just a dust jacket, but a key element in the sales and marketing campaign for any book. Given its importance, the covers require more attention than ever before to make sure that it can fulfill its central sales and marketing role.

How does Jellybooks design A/B tests for book covers?

Test readers are invited as normal, but half the test audience is randomly selected to receive one cover, while the other half is selected to receive an alternative. Readers think we are measuring their reading behavior, as usual—which we are—but in addition we also look at:

i. Whether the cover influences which book is picked by test readers from a selection of available titles to read (remember that test readers do not know that there is more than one cover in play). Does one specific cover make it more likely that the book will be picked compared to an alternative?
ii. How completion rates, velocity and recommendation factor might be influenced by the cover, as the cover has created a particular “promise” or expectation for the reader. In other words, we are very specifically measuring whether a cover can influence the results of a reader analytics test and how strong that effect is. This kind of potential reader bias depends in part on whether readers form their expectations primarily based on the cover or on the synopsis of the book.

An A/B cover test might not perfectly replicate the environment, stimuli and selection pressure that occur in an online or physical bookstore, but for the first time it allows us to judge—in a reasonably objective way—how readers’ expectations are shaped by the cover, if one cover might be a better fit for a book, and to what extent a cover might shape a book’s future success.

If you are interested in participating as attest reader in one of the upcoming campaigns that Jellybooks organizes with publishers, then sign up at Jellybooks. There are some exciting titles coming soon, as we are launching several test reading campaigns in the UK, the US, Germany and other countries in the next couple weeks.

If you are a publisher and interested in undertaking A/B cover tests yourself, then let us know. We will be sending out a newsletter in irregular intervals about what we learn and we will of course also invite selected publishers to participate in the beta tests of this new approach to optimizing marketing.

Earlier posts in the data-smart publishing series:
“The Internet of Bookish Things”
“Reading Fast and Slow – Observing Book Readers in Their Natural Habitat”
“Start Strong or Lose Your Readers”
“What Books Have the X-Factor? Measuring a Book’s Net Promoter Score”
“Men Are from Mars, Women Are from Venus, But What About Readers?”
“How Does Age Affect Reading?”
“8 Reasons Why People Buy Books”
“Data Vs. Instinct – The Publisher’s Dilemma”

Note: All the data reported in this post was collected in test reading projects financed by Innovate UK. EPUB3 files were modified with candy.js by Jellybooks so we could record, store and extract the user’s reading behavior when using iBooks, Adobe Digital Editions and selected Android reading applications. We have since extended support to VitalSource and other reading applications with more to come. The data stored within the ebook file was extracted when the user clicked the “sync reading stream” button at the end of chapter or the end of the book. All users were informed about and consented to the presence of the analytics software.

To get all the ebook and digital publishing news you need every day in your inbox at 8:00 AM, sign up for the DBW Daily today!

7 thoughts on “It’s the Cover, Stupid! Why Publishers Should A/B Test Book Covers

    1. Andrew Rhomberg

      no doubt, the story would be so much more interesting with the actual covers, but publishers don’t permit it.

      Under our model, the publisher holds the rights to the data collected with Jellybooks have only a license and that license does not include discussing or identifying any specific title, author or publisher without adequate permissions (which are usually NOT granted)

      sorry – and a deep sigh t the same time

  1. Emily Labram

    Brilliant stuff! I’ve been involved in A/B testing covers before, but never with the additional data on corresponding reader analytics, which makes this so much more powerful.

    Keep on doing what you’re doing.



Your email address will not be published. Required fields are marked *