What We Learned (repost)  

Earlier this month we took our Hiptype.com analytics service offline as we transition Hiptype to a new home where we’ll be able to have a much larger overall impact and achieve our mission of bringing data-driven tools to book publishing.

We have learned throughout the last few months that there are major challenges facing innovations in eBook publishing.

 Discovering an Opportunity for eBook Analytics

Back in the spring of 2012, we were just a couple startup geeks at the initial stages of writing an eBook about analytics and data. As we researched the various options for publishing our book, it quickly became apparent that eBook publishing was years behind the web and native apps in terms of the types of insights and analytics available to publishers.

While publishers of apps and websites can now capture sophisticated data about their customers and how they use their products, eBook publishers typically have little information about how readers are engaging with their titles.

In other environments like the App Store and the web, third-party analytics plugins have addressed the demand for data by allowing publishers to easily include analytics capabilities with a few lines of code. It occurred to us that a similar concept could work with eBooks.

We got in touch with a few contacts in the publishing business, and their frustration with the lack of data available was palpable. This was especially true among authors and publishers, and we were surprised to learn that even the largest publishers in the world didn’t have even the most basic data about how their books are used.

Our attention quickly moved from the eBook itself to this fascinating question of whether it’s possible to add a third-party analytics plugin to an eBook document in a way that would work with any of the most popular eBook readers. What if we could help authors and publishers learn about the most and least popular sections in a book, and the various segments of their audience? We simply had to try it.

 Implementing the Impossible

We decided to give ourselves a couple weeks to build a prototype for an analytics plugin that could be easily added to an eBook. But this was a daunting task, as none of the most popular eBook formats contained an officially supported way to execute javascript, load tracking pixels, or do anything else that could obviously be used for analytics purposes.

We first narrowed down to a couple of specific eBook formats that were both popular and more likely to yield some way of supporting analytics features. These were the ePub-based formats that are used by Amazon, Apple, and Nook.

The latest ePub specifications contain support for several features that could be used for analytics purposes. And while formats like Kindle Format and iBooks are proprietary, they are modified versions of ePub that are easy enough to work with for a developer familiar with HTML.

Within a few days of intense reverse engineering of these ePub-based formats, we found a way to get the exact kind of analytics data we’d been seeking titles published in Apple’s iBooks format. We quickly started scheduling meetings with eBook publishers to get the analytics service deployed.

 A Promising Launch

Within just a few days of completing our prototype, we submitted an application to the Y Combinator Summer 2012 program and within a few weeks we went for a secondary interview and were accepted into the batch.

Our team moved to Mountain View for the summer, and we quickly got to work out building out our service. We created an online dashboard at Hiptype.com, where you could upload a book document and download a version modified to include the Hiptype analytics plugin. Once the plugin was loaded into one of the supported eBook readers, the dashboard would instantly fill up with information about the audience of the book and how they were engaging with the title.

It wasn’t long before word got around within publishing circles and we found ourselves being contacted by executives from the world’s biggest and best publishers.

We launched in the first week of August, with coverage provided by Fast Company, Techcrunch, PandoDaily, Publishers Weekly, PaidContent, and other top publications. Within a week of our public launch, thousands had signed up for an invite to try Hiptype.

 Working with eBook Platform Vendors

Unfortunately, the launch spotlight also forced the delicate conversations we’d been having with eBook retailers to speed up. The vendors controlling the largest eBook platforms had some legitimate concerns about a developer ecosystem springing up around their services, which were not designed with this type of third-party development in mind.

The primary concerns we discussed with our contacts at the eBook platform vendors were privacy and performance. If we could address both of these areas, it looked promising that we’d be able to work out a way to get our analytics plugin available to millions of titles on these platforms.

 Privacy Concerns

Privacy is a huge issue affecting all types of technology. There is currently little in the way of regulations and laws about tracking, analytics, personalized advertising, and other uses of data that many people feel invades their privacy.

The privacy concerns surrounding Hiptype were not substantially different than those for any other type of analytics product. There needed to be a clear disclosure letting readers know that their eBook contained an analytics plugin, and an opt-out preference that would work across all titles. Additionally, vendors asked that we kept all analytics anonymous, and didn’t attempt to retrieve any kind of personally identifiable information (PII) such as names and email addresses.

We updated the analytics plugin so it contained a disclosure about Hiptype’s anonymous usage statistics, and included a link to more information and a simple opt-out button that would set a preference across all books.

We also made some technical changes to the analytics plugin to make data shared with our servers encrypted for security, and replaced a demographics panel in our online dashboard with a “Personas” panel that would provide the same amount of insight to publishers and authors but abstracted away the exact demographic details to lower the “creepy factor”.

 Performance Concerns

The second type of concern shared by the eBook platform vendors was that eBook analytics would make the reading platforms sluggish, reduce battery at a faster rate, and use unnecessary amounts of data bandwidth. These are truly important issues among mobile device vendors, and we took it upon ourselves to minimize the footprint of our analytics plugin.

Over a Redbull-fueled weekend hackathon, we rewrote the analytics plugin code to make it faster and leaner, and the interface to send analytics back to the server was re-architected to batch requests to only send data to the server once every 30 seconds so that less battery and data would be required.

 Obstacles and Setbacks

While the conversations with our contacts at the eBook platform vendors initially seemed promising, by mid-September it was clear that getting approval for Hiptype would be no cakewalk. It didn’t help that publishers and retailers were dealing with antitrust lawsuits from the US Department of Justice - there was concern that if any data happened to be shared between parties, it would appear that publishers had formed a monopolistic “data cartel”. While it wasn’t our intention to form such a cartel, we were surprised by how often this was brought up in conversations.

Just a few days after speaking with a representative from Apple, an iBooks update was released that removed the support for analytics we previously had. Progress with Amazon and Barnes & Noble had also stalled. In all three cases, the eBook platform vendors also had corresponding app platforms that publishers have been using when they desired features that weren’t supported by eBooks, sometimes with great results.

The problem is that from all three vendors, we were eventually told the same thing - if we wanted a feature that wasn’t supported for eBooks, then we should be looking at their app platforms instead. We were referred to app store representatives and told that we would be supported as an analytics plugin if we followed the developer guidelines for these platforms.

But our mission wasn’t to provide yet another analytics plugin for mobile apps. These existed already. Our mission was to provide an analytics plugin for eBooks, so that data that was previously locked up could be made available where it was needed most.

Maybe in a year or two there will be more pressure on eBook platform vendors to open up their eBook environments, even a little bit, for third-party services that abide by best practices for privacy and performance. One simple way to do this would be to support ePub3, the new generation of ePub that allows for tracking pixels and javascript code to be included within books.

But right now, there’s no indication that ePub3 or a format with support for similar features will be adopted by the eBook platform vendors within the foreseeable future. With Hiptype, we were too early. And in the fast-paced world of startups where you often don’t have more than a few months of runway to prove out an idea, being too early is the same as being wrong.

 Next Steps

While we’re sad to shutdown Hiptype in its current form, we couldn’t be more excited about the new home we have found for the service. We’ll be announcing details shortly.

Stay tuned!

 
31
Kudos
 
31
Kudos

Now read this

The Magic of Silicon Valley

I read TechCrunch, have since I was 12 years old – during middle school I would religiously sit and go through tech blogs for hours every day. The stories were awe-inspiring; people creating amazing things that change the way people... Continue →