Why Proprietary Ebook Formats Work

Expert publishing blog opinions are solely those of the blogger and not necessarily endorsed by DBW.

EPUB ebook formats FirebrandAs someone who has been creating ebooks for more than a decade, I am often asked whether I think Amazon will someday let go of its proprietary Kindle format and switch to EPUB. My response has been the same since the question was first posed to me in 2009: I don’t see any reason why they would want to.

I also find it somewhat amusing that everyone singles out Amazon for this supposed faux pas of digital publishing, when Amazon is hardly the only company with a proprietary ebook format. Apple has iBooks Author and Barnes & Noble has proprietary formats for children’s books and fixed-layout nonfiction, just to name a few.

Why do these companies choose to use proprietary formats instead of (or in addition to) the open source EPUB standard?

Because proprietary formats just work.

When an ebook retailer builds its own ebook format, it steps beyond controlling just the reading system or device to actually controlling the code that is loaded into that reading system. That level of control brings with it a level of certainty about what the retailer can expect to receive, in turn giving it more control over features, functionality, and—importantly—the quality of the reading experience.

This is the same reason why some EPUB retailers will run every EPUB file they receive through an ingestion process that changes the markup in specific ways and forces the EPUB to meet certain criteria. Essentially, those retailers are ingesting EPUB files in a proprietary way, all with the goal of making the reading experience better.

While I am a strong advocate for EPUB and other open initiatives in our industry, I think we do ourselves a disservice if we try to downplay the benefits of proprietary formats. Instead, I think we should be working on ways to make open standards more consistently supported and easier to implement—in other words, learning the lessons that proprietary formats teach us.

It is encouraging to me that we are doing just that in the ongoing development of the EPUB standard.

There are active efforts to bring the EPUB standard even more into line with prevailing web standards, allowing EPUB files to be first-class citizens of the web and as easy to use as any webpage. There are also efforts to make EPUB work better for educational content (specifically the EDUPUB profile), essentially using “proprietary” (note the air quotes!) markup to extend the abilities of an EPUB file in a space that has some very specific requirements. And, finally, there are efforts (like Readium and epubtest.org) to make the display of EPUB files on different devices as close to each other as possible, bringing the goal of one EPUB file within ever closer reach.

So fear not, friends. If we continue down this path, there may come a day when it actually does make sense for all ebook retailers to use the same ebook format. But even if they don’t, we will at least have learned the lessons of proprietary formats, and we will be much the wiser for it.

Update: An earlier version of this post incorrectly stated that Inkling uses a proprietary, XML-based ebook format. Rather, content authored on Inkling’s platform is in HTML5 with flexible CSS styling, allowing users to export their content into other formats.

15 thoughts on “Why Proprietary Ebook Formats Work

  1. Mike

    I completely disagree.
    The best format for the future is HTML5 offline docs that surpass all proprietary formats out of the box.

    Reply
    1. Richard Stallman

      Many proprietary ebook formats implement digital restrictions
      management (DRM), which makes them an injustice.

      Commercial ebooks are typically unjust in other ways too.
      See http://stallman.org/ebooks.pdf for more details.

      Digital books do not have to mean a step backward, but the sellers
      want to make them one. Please join me in rejecting any ebook that
      respects readers’ freedom less than an ordinary printed book.

      Reply
  2. Michael W. Perry

    Pretty words, but words with little relation to reality. Amazon hasn’t used its proprietary format to create a better user experience. Mobi is dreadful and KF8 still falls far short of Epub 3.0 in features. Amazon in many ways has a bizarre view of publishing. Because it makes most of its money from fiction sold cheaply, it has shown little interest in adding features that more complex books need.

    You saw that just days ago when Amazon announced—insert trumpets blaring—that their Kindle app would now handle drop caps and would make justified text a bit less ugly. Amazon fanboys leaped to their soapboxes to praise features that should have been present years ago. The very fact that Amazon had to make a Big Deal about changes that long overdue illustrates just how impoverished Amazon’s formats are.

    Even more important is the fact that creating documents that support Amazon’s proprietary formats is a pain in the neck. The industry-standard tool for creating print books, InDesign, now does excellent fixed-layout and reflowable epubs from the same document. It doesn’t do Amazon’s proprietary formats because Amazon refuses to help Adobe develop those export capabilities. The result is a lot of unnecessary bother and expense for publishers.

    I’m quite aware of the game that’s being played here. It’s quite similar to the 1990s, when there was a huge industry of software support companies that made their living correcting all the inadequacies of Microsoft Windows. They were hardcore Microsoft fanboys.

    Amazon is playing a similar game. When I queried Kindle support about developing feature-rich ebooks, I was told to go the third-party developers who were, like their counterparts with Microsoft, were growing rich doing what Amazon refuses to do, which is to make it easy to use standard tools such as InDesign to create Kindle books. Amazon would rather that publishers either:

    1. Waste all their ebook development funds creating Kindle editions through third parties, leaving no money for editions for other retailers.

    2. Use Amazon apps that create only specific kinds of ebooks and only for Kindles. Again, all the money a publisher has invested in creating a Kindle edition gives them no leg up on editions for other retailers.

    The short of it is that Amazon may or may not adopt an industry standard such as epub. But its reason for doing so isn’t defensible. Amazon wants to inflate the cost and trouble of creating ebooks with proprietary formats to the point where many publishers simply can’t afford to create ebooks for Amazon competitors. They lose money. Amazon makes more money. It really is that simple.

    Amazon isn’t a friend of authors or publishers. The same is true for readers. Amazon’s strange-seeming royalty scheme is carefully crafted in inflate the price of ebooks, to enrich Amazon, and to put the appearance of onus for higher ebook prices onto authors and publishers. Amazon is Amazon’s friend and Amazon’s friend only.

    And in saying that I am not praising epub. The format is covered with the fingerprints of media pundits who’re obsessed with gimmicks like multi-media and of geeks, who wouldn’t know an attractive page layout if it hit them like a ton of bricks. There’s a host of ways epub needs improving from auto-handling of windows and orphans to an ability for those doing epub layout to specify how images are handle. In many ways epub is going through the same follies as HTML went through in the mid-1990s. It desperately needs input from people who know how to make a printed page or screen look good.

    Reply
    1. Jonathan Mattson

      I’m confused, amazon is inflating the price of ebooks? While everyone else (publishers, “main stream” authors, and many in the media) accuse amazon of lowering the price of ebooks to where no one can make any money on them. Which is it?

      How is a “scheme” where they ask independent authors to price under 10 dollars for a 70% return raising the prices of ebooks when many books from big publishers are priced over 10 dollars? Look at any of my wishlists on amazon, they are all stuffed full of publisher books priced anywhere from 12 to 25 dollars. All with the notation that the prices were set by the publisher.

      Or for other proof, read the verdict from the lawsuit against amazon on the price fixing conspiracy.

      I understand you hate amazon, fine. I hate lots of companies that other people do business with also. But at least be logical with your criticism and complaints.

      Reply
    2. iceberg

      Note that this is a very different question from “why the ƒ@#$ doesn’t the Kindle natively support the EPUB format” in addition to whatever proprietary formats work best for their needs

      Reply
    3. kabosh

      Actually, on ebook pricing, you could not be more wrong. Amazon’s ebook pricing, for works from regular publishers, is set by contracts with the publishers; the royalty split is also set by contract. Amazon is actually prohibited, contractually, from discounting ebooks from the Big 5. Amazon is well known to use ebooks as a loss leader, steeply discounting them, a practice the mainstream publishers hate, as it creates competition with physical books– hence the contractual prohibitions in the current round of contracts. Let me gently suggest you are somewhat misinformed.

      Reply
  3. JustAReader

    I think that the differences between EPUB & Mobi /Azw for off-line reading are far less troublesome than the proliferation of formats and interfaces for on-line book services. Every textbook publisher seems to have their own proprietary approach, many of which are so unstable that I can’t keep a link to interesting content for more than a few months, if at all.
    Now with the rise of subscription services for general ebook reading, I’m less worried about whether EPUB and Mobi will merge than I am about whether they will get squeezed out entirely.

    Reply
  4. Private_Jet

    I will offer that proprietary ebooks work if you are the type of person that only purchases that specific format for a particular device, however, this is very shortsighted!

    What if I told you that you can convert most formats to another format of your choosing, regardless of the device? My Kindle has many converted .doc files, .html files, .epub files , and .mobi files that are now ready to read on my device. For folks wanting to do standard/traditional reading with the occasional illustration, these formats work pretty dang well. If you want to make the basic written word do cartwheels and add animation and sound effects, guess what….. you have a completely different format that is going beyond the idea of an ebook (which is to read literature).

    I have many ebooks in multiple formats and the one format causing the biggest trouble are Adobe PDF’s which do not work well for basic Kindle readers (perhaps because PDF’s are some type of image?) and take up a much larger amount of memory space to save when compared to .epub ebooks. But hey, PDF’s are great for Kindle Fire’s, color screens, laptops, computers, etc., so I have no real problem with PDF’s.

    Remember one thing: most ebook formats can be converted to whatever format you need or prefer, for free.

    Are some formats better? It seems this is largely opinion at this time based off of whatever criteria you perceive as valid. Of course a company will try to \box you in\ with their proprietary format and perhaps (just perhaps), a company will somehow make ebooks better but you are missing the larger point: the literature and author are more important than the ebook format, not the other way around.

    Save every ebook you buy on a computer or cloud drive! Why? So that when you go to another ebook reader that you like or can afford, you can convert all copies to the new device with minimal hassle and free of cost. After all, it is your personal library.

    Case in point: Project Gutenberg offers free ebooks in .epub, Kindle (in .mobi format), plain text, and .html. Presumably because conversion between those 4 formats are pretty darn easy with a computer and they just say \Here you go dear, it’s so easy to convert an ebook you shouldn’t stress yourself.\ And Project Gutenberg is absolutely correct!!

    I largely find proprietary ebook formats to be something akin to convincing someone that Coca-cola is better than Pepsi or vice versa. Just enjoy your \cola\.

    Reply
  5. Barry Marks

    I’m not qualified to say what’s best for the book sellers but as a reader I think what would be best for me would be for book sellers to all use the same format and DRM so I could read their books on any device I choose. That would give non-ebook sellers a chance to develop hardware and I’d be able to pick the device that suits me best and buy my books from the stores I prefer.

    Yes I do realize that ereaders would be more expensive this way and, even though I can’t afford that I’d manage it and I’d be glad to pay for that kind of freedom.

    Reading is an important part of my life. I buy books mostly from Amazon and I read mostly on Kindles although I do have Kobo ereaders as well. None of these devices really has the sort of features I can find in a variety of Android apps and as long as ereading devices are manufactured by the book sellers that’s not likely to change. I have to use e-ink because LCD screens limit my reading time.

    This isn’t a rant against Amazon. They’re doing what works best for them and they’re doing it well and they’re quite considerate in the way they handle things. But it is a bad system. Amazon popularized ebooks but they didn’t really create this system. Proprietary formats existed before Amazon was born.

    I think from reading various blogs and forums that a lot of people have the same problems I have but don’t realize that proprietary formats and devices from book sellers are standing in the way of solutions.

    Barry

    Reply
  6. Rob Siders

    It doesn’t do Amazon’s proprietary formats because Amazon refuses to help Adobe develop those export capabilities.\

    And there are software makers, much smaller and much less well-funded than Adobe, that have accomplished export to Amazon’s reflowable and fixed layout formats without Amazon’s help. After all, both of those formats have spec documentation readily available. Surely Adobe is capable of downloading and reading the doc and then developing an export function to produce those formats. If it’s a licensing restriction for Kindlegen, that’s one thing (and I suspect this is probably the issue). But let’s not pretend that Amazon needs to help Adobe figure out how to make an export feature that produces mobi files from InDesign.

    Reply
    1. John

      there is absolutely no chance since they don’t even read HTML, CSS, JavaScript and ePub specifications.

      Do files InDesign outputs pass epubcheck? Sure.
      Are those files output with specs on mind? Not at all.

      Adobe should seriously hire people who know those specs to make sure InDesign’s developers don’t build something that could prove a catastrophe for the entire standard. It is a natural process that browser vendors have naturally followed to make sure everything is OK.

      But no, they just outsource this to India, putting software in the hands of the most terrible project managers you can find on the market and as a result, InDesign has become a huge problem (according to inside sources in distribution, reselling and e-reading development).

      Reply
    2. Ben Denckla

      Rob Siders wrote:

      Amazon’s reflowable and fixed layout formats …. have spec documentation readily available

      Where is this documentation? I try to not ask questions that can be Googled, but I honestly cannot find it.

      Reply
  7. andrew updegrove

    What the proprietary formats in eBooks teach us is that the eBook industry hasn’t learned the lessons that virtually every other area of technology has – that proprietary extensions and formats are simply tools to limit consumer choice and maintain vendor (or, in this case, supply channel) dominance. The problem isn’t that proprietary formats are necessary, but that those using them are not participating in, and committing to, making the open format standards better.

    For the last 130 years, just about every industry in the world has adopted open standards, and then implemented them. Suggesting that eBooks are somehow different from much more complex technologies is nonsense.

    Andy Updegrove

    Reply
  8. Timothy

    ePub’s broad acceptance is a good thing, but when various vendors have their own version of how to render the same ePub, that’s not. The &quo;middle path&quo; is to use simpler (some would say &quo;dumbed down&quo;) formatting that will render predictably on the various ePub platforms.

    Proprietary formats can offer features not otherwise attainable, but at a cost if they don’t fit into a regular workflow. One problem we have with proprietary formats is that they can be a moving target–what was once acceptable usage is no longer. Another problem is that troubleshooting problems with a proprietary format can sometimes be a black box, because there are hidden things happening that aren’t in any documentation and which the vendor will not disclose. For example, a file that builds in KindleGen with no errors or warnings, and which tests perfectly on a Kindle device and *using the software they provide* (ahem), can have problems that are introduced by the proprietary process Amazon runs on them after they’re uploaded. In a philosophical sense that’s akin to the different flavors that result from the same ePub, but in practical terms it’s a nightmare to solve (whereas ePub issues are more predictable) because of the lack of transparency.

    Reply
  9. S Winklbauer

    Very short-sighted arguments and I do not agree with the author. Reality looks different. The reason proprietary formats work for this retailers is that is forces people to sue their ecosystem reducing choice. Amazon tries to control the market and destroy any competition.
    They are not offering a better format.
    They should sell books in other reader’s format, and then we can see if people would choose a Kindle and its ebook format as a better choice.
    I think that Amazon is very anti-competitive in this area and is making a big mistake. They are already misusing their power to make sure that they control publishers and the market. This is a cartel.

    Reply

COMMENT

Your email address will not be published. Required fields are marked *

*