Chartbeat: The Next Generation

Three years ago this month, Chartbeat launched at the web 2.0 summit. Today Chartbeat is beating under the hood of sites across 37 countries and takes the pulse of around 5 million visitors across the web at any one moment. Fantastic clients like ESPN, Fox News, The New York Times, Forbes, Fab.com, and Gilt Groupe have partnered with us and helped us to understand the contours of this new world of data.

Today, I’m excited to announce the next chapter in Chartbeat’s history: a new round of funding, new look, new dashboard, and a whole bunch of new features. That’s a lot to get through, but I promise to be brief.

A New Round and A New Partner

We’ve just closed a $9.5m Series B round of funding led by Josh Stein at Draper Fisher Jurvetson and Saul Klein at Index Ventures, and they’re joined by some of our favorite Angel investors.

Saul has been a constant friend and guide over the last three years and helped us take the time to find the right partner that will continuously help us push Chartbeat and the real-time revolution forward. We found that partner in Josh Stein. Josh took the time to get to know us over the course of a year, has incredible experience of building great SaaS companies like Box and SugarCRM. Most importantly, Josh just plain got it. Thanks Saul, thanks Josh, we’re going to make you proud.

A New Look

The first thing you’ll notice when you hit Chartbeat.com today is a very different look. We’re growing up. And growing up means making sure we’re as good looking on the outside as our dashboards are on the inside.

We’re also making things simpler. Newsbeat has become Chartbeat Publishing: the command center for all serious publishers of media and content.

A New Dashboard

We’re also rolling out a brand new dashboard with advanced functionality to all Chartbeat users over the next few days. When you get your hands on it, you’ll see a few key elements that we’ve focused on:

1. Real engagement

Common understandings of engagement have always relied upon explicit actions (a like or a share) taken by a few people or traditionally inaccurate guesstimetrics like “time-on-site.” At Chartbeat, we wanted to accurately understand how engaged each person who visits your site is, not just the people who hit the like button.

So we built Engaged Time. Window open in another tab? Doesn’t count. The visitor isn’t actively viewing, reading, commenting on the page? Doesn’t count. With Chartbeat’s unique way of measuring, Engaged Time counts the amount of time someone spends actively interacting with your site and enables you to see your site from a completely different perspective.

For the very first time, you can finally go beyond eyeballs and clicks and understand which content is making an impact, what’s engaging people’s heads and hearts.

2. Data in context

When you’re looking at your data – whether Chartbeat or otherwise – I bet the first question you ask yourself is “Is this good?”. You generally know if you’re doing well compared to yourself, but what about your competitors — those that are fighting for the same eyes and hearts that you are?

We’re introducing Peer Stats, which does just that.

Now, you can see your data, put in context with the anonymized, aggregated stats of your sites like yours all in real time.

Are we getting as much social traffic as we should? Is our page load speed fast enough? For the first time you’ll have a frame to understand your performance in the context of the wider web, so you know where to focus and when to celebrate.

3. Your data wherever you are

When something important happens, you want to know about it — whether it’s on your site, your iPhone or iPad app or across the social web. So why should you only see real-time stats of who’s on your site? You shouldn’t.

Chartbeat is going beyond the site. From now on you’ll be able to get real-time data from your iPhone or iPad app as well as your site. This feature is currently in beta so we’re going to need your feedback, but we’re super excited about it.

We’ve also launched a new Social View that brings the social data you need into one dashboard. You’ll get loads of insight into not just who’s talking about you where, but how that’s affecting your site traffic.

We hope you like all the new things we’ve been working on. The only thing we ask of you in return is your feedback. Let us know what you think about your new dashboard as you’re rolled into it over the next few days.

We can’t thank you enough for the love and support thus far. We’re excited to get to work and take this thing to an even higher level.

Post-mortem of today’s DNS outage

For those who might not have been following or affected, chartbeat just suffered, and is recovering from, a major DNS failure that affected our users’ dashboards. I wanted to give some insight into what happened and explain how we will do things differently in the future.

Yesterday evening, one of the nameservers at our DNS provider started reporting 0.0.0.0 as the IP address for static.chartbeat.com. As you can tell, this isn’t a real IP address and we were stumped as to why it was happening since we had not made any changes that might affect it.

Static.chartbeat.com holds all of our static assets, including our images, css, and javascript for our dashboards and the javascript we use to report visitor statistics to our servers. Because of this DNS error, many people were unreachable. This didn’t have any effect on people visiting our customers’ sites, but it did mean the visitors who were hitting the bad nameserver weren’t being reported. As a result, dashboards showed a dip in traffic.

After being immediately alerted by Nagios, we identified the offending nameserver and reached out to our DNS provider to find out what the hell was happening. At the same time, we removed the entry for that nameserver from our system, taking it out of circulation.

We monitored the effects of the changes and everything seemed to go back to normal until early this morning, when our DNS provider began to pull the same trick on a larger scale across multiple nameservers. For some reason, the lifetime of some cached assets (TTL) was being set at 12 hours instead of two hours, meaning any change we made would take at least 12 hours to fully propagate across the web. The wall still bears indentations from my head at this point.

It became quickly apparent that our current DNS provider wasn’t going to be able to fix the situation in the timeframe we needed, so we reached out to Dynect, the DNS provider behind Twitter and bit.ly. Dynect was great and we were able to move our entire infrastructure over to their services before the morning was out. The changes would take a while to propagate because of the rogue TTL setting at our old DNS provider, but at least we knew that when the changes rolled out we’d be on a much more bulletproof DNS system and everyone’s traffic would be back to normal.

And that brings us to now. Dynect and Akamai were both awesome and super responsive throughout, and the bit.ly guys were a great source of advice and support. We were also blown away by the response from our users, many of whom tweeted or emailed incredibly kind messages. Some of them were captured in Erin Griffith’s Adweek piece today.

What did we learn?

Aside from the immediate lessons around which DNS provider to use, I’d say we were probably too optimistic at first about how easily this would be resolved. Once we acted to fix the first bad nameserver, we implicitly assumed things would get better, not worse, and missed a valuable window to have prepared for more extreme options. We should have reached out to Dynect much earlier and had an alternative prepared just in case the situation recurred, rather than simply reacting when everything went crazy a few hours later. We should have (and will be implementing) a protocol to explore several scenarios and what we need to do to mitigate them, rather than simply assuming any crisis is going to follow the path we implicitly think it will.

In the end, it doesn’t matter whether it’s an external service or an internal bug that fails, the responsibility for providing you with the service you deserve is ours and we let you down. We’re incredibly sorry that our users were affected by these issues, we’re humbled by the response and we’re grateful for your support.

Tony Haile, General Manager

When Mubarak met the Internet

We watched in awe as Al Jazeera helped to topple a dictator in Egypt. Chartbeat was behind the scenes powering the analytics and dealing with the flood as millions came to the sites to watch a regime crumble. It was awesome to watch and we thought it would be awesome to share, so with the kind permission of Mohamed Nanabhay, we present the 2011 Egyptian Revolution in online traffic. Click on the image for the full view.

 

 

Breaking 3,000,000 and the Mubarak Effect

It feels like less than a month ago we were celebrating crossing 2 million concurrent visitors on chartbeat sites; that’s because it was! It took us 16 months to get to 1 million, five months to get to 2 million and now less than a month to break 3 million. Safe to say it’s the fastest growth we’ve ever seen at chartbeat. We’ve seen a flood of new sites and users over the last month but what put us over the top was the incredible events happening in Egypt.

We’ve been working hard with Al Jazeera throughout what I guess we can now call a revolution. In a recent MediaWeek article, Mohamed Nanabhay talked a little about how he used chartbeat to react swiftly to what was going on in Egypt and we’ve watched in awe as a News service ignored by many of the major cable companies has become, at least for a while, the most important news site in the world. Congratulations to Mohamed and his team, and all the other news services whose reporters have been covering these events. We’re proud to have worked with you during this momentous time!

 

Breaking 2 million!

Breaking 2 million

After chartbeat launched in April 2009 it took us 16 months to get to the point where we were recording the digital heartbeat of more than one million people at any one time. I’m psyched to be able to say that just five months later we have hit two million simultaneous visitors. That’s not two million in the course of a day or an hour, that’s two million people concurrently at a single moment in time.

Thank you to all our incredible partners who’ve helped us get here, had the courage to try something radically new and given us so much advice and guidance; we wouldn’t be here without you.

Hey Publishers, Chartbeat is getting personal!

We’re proud to announce that the alpha release of Newsbeat, the first analytics service built entirely for Publishers. Newsbeat is real-time analytics purely focused on what’s important for those people creating and editing content on the front lines of media sites.

Personalized Dashboards

Every editor and writer can have their own dashboard showing them what’s happening on their content, and how to improve its performance to maximise traffic and engagement. We’re opening up early Alpha access to a select few so if you’re a Publisher, Editor or Writer who wants to find out more, go to Newsbeat and see just what it can do for you.

Chartbeat raises $3m from Index Ventures and more!

  • 5 guys,
  • More than 1.2 million concurrent visitors being tracked right now,
  • More than 2,500 paying customers,
  • And three million dollars from the best investors on the planet…

We’re incredibly excited to announce that we have raised $3m in Series A funding from Index Ventures and some of the best investors in the world including:

  • O’Reilly AlphaTech Ventures
  • Jeff Clavier
  • Freestyle Capital
  • Lowercase Capital
  • Founders Collective
  • SV Angel
  • Lerer Ventures
  • Jason Calacanis
  • Josh Stylman & Peter Hershberg
  • Betaworks
  • Alex Zubillaga
  • Fritz Lanman

It’s an incredible show of faith in our team and what we have built thus far and we’d like to thank everyone for their support. We’re going to take this team of hardcore C coders, genius Data scientists and all-round Python hackers and build Chartbeat into the ‘Intel Inside’ of websites everywhere.

Chartbeat: the quick guide

Chartbeat is a real-time analytics service used by  more than 2,500 paying customers including Time.com, Gawker Media, Groupon, Threadless, the Chicago Tribune and the New York Times to make their sites more adaptive in real-time. To see an example of what chartbeat can do when integrated into a site take a look at Gawker’s new beta site where Editors use chartbeat to make real-time decisions about what stories to show where and even reflect that data back to their users. There’s a 30-day free trial so there’s really nothing stopping you seeing what chartbeat can do for you.

Breaking one million

There’s been some celebrations around chartbeat today as we can now say that at any one moment during the day we are listening to the pulse of more than one million people across the web. You can see the current number of people chartbeat is interacting with by going to our front page.

As I know from painful experience with my personal site, for the vast majority of sites on the web the average number of people on that site at any one time is approximately zero, so this is a big milestone for us. We’ve been lucky to work with some awesome sites and companies since we launched a little over a year ago and in particular I want to call out a few chartbeat users who’ve gone above and beyond with feedback, encouragement and subscription fees! Thank you, Gawker media, Fast Company and Inc, Time.com, The New York Times, AOL, Forbes, Deltatre, Groupon and Threadless.

Thanks also to companies like Talking Points Memo, the Onion and the Chicago Tribune for putting the API through its paces and using chartbeat to make their sites automatically adapt in real time. We’re really very fond of you all!

We’re working hard on the evolution of chartbeat right now (and hiring!) and we plan on continuing to show just what you can do when you take data real time.

Announcing two new historical APIs

Although chartbeat has always emphasized the value of real-time data, we know that past performance is really important for understanding what today’s numbers mean. It’s why we brought historical data into the new dashboard, showing how today is doing compared to last week, and how a given referrer, page, or other metric has trended over time.

Many of you have contacted us asking for programmatic access to this trend data for your own purposes, and in fact chartbeat has a philosophy of trying to release all the APIs we use to build our own interface. Although the APIs are still provisional, we’ve decided to release an early version for you to try out and provide feedback. You’ll find them on api.chartbeat.com: data_series and day_data_series.

As always, please feel free to offer suggestions or ask questions to via support emails or in the Google Group. We encourage anyone using our APIs to join the Group, which we will also use to announce any deprecations or changes.

Another note for the coders out there: our change last week to support bookmarking and back buttons of specific views in the dashboard has an interesting side effect. You can now build links straight into the dashboard for a particular page from your own internal tools. For example, you can use http://chartbeat.com/demo/#Page::/a_vc/mba-mondays/ to link straight to that page’s date on Fred Wilson’s dashboard.

The four design decisions behind chartbeat’s v2

The chartbeat dashboard recently underwent its first major revision since launching a year ago. In this post, betaworks’ User Experience lead Neil Wehrle will share some of the process and design decisions that got us here.

The team has a strong vision for chartbeat, and to bolster our vision I led some quick-and-clean research into how current and prospective users view chartbeat. Our plan included heuristic evaluation, in-person usability reviews, and group-based cognitive walkthroughs, all to provide insight and momentum. The team arrived at a set of first principles to guide development, and we quickly settled in an iterative design-build-test cycle, where the fidelity of each step evolved as we built out the site. I’ve highlighted some of the core principles and selected design decisions below:

Structure the site around user goals
chartbeat is a tool for front-line workers, not just internal analytics teams. We designed the layout around a set of use cases, so users could walk through the data in a logical fashion, understand causality, and take action. The triad of panels at top allows users to understand “how many people, how did they get here, and what are they looking at?” in a snap. We also heard our users love the kinetic nature of chartbeat, so we extended that a bit with a Matrix-like stream of raw hits in the right-most column.


Data should be appropriately dense, clear and actionable

Data should be rich and deep, without compromising ease of use and clarity. As an example, the tree map in v.1 was challenging for users – they liked the intent, but it was difficult to interpret. Sites with extremely low or high traffic or with few pages skewed the chart so that it was impossible to analyze. We decided to use a small range of fixed sizes, ensuring the display of most pages, and using dots to represent visitors. Larger numbers and page titles increase legibility, while isolating the page modules with white space makes it easier to read them as units. We also standardized and gave meaning to the range of colors we used, so users can more associate meaning across the panels.



Everything should be on a single page
A key interaction design challenge chartbeat faces is letting users drill into richer data whithout resorting to traditional hierarchical navigation schemes. We came up with the notion of “pivoting” around a selected data element, where the entire page changes to reflect just that element. This way, chartbeat can serve as a site-level analysis tool and easily shift to isolate a page with a single click.

Use historical data as context, but keep it a real-time tool
Users love the real-time aspect of chartbeat data, but a unanimous request is to provide some context to what they’re viewing. Some amount of historical data was in the previous version, hidden behind a tab at the top of the page. Hidden, too, was a powerful replay feature that lets users isolate events and walk through it (“Tivo for your website” as Tony likes to say). By bringing the replay to the fore, we signaled to users that historical data is available by showing a trend chart. When users pivot on data, we also show a thin historical chart that can be expanded for more detail.

We’ve been doing some followup visits with users to understand their long-term usage and how it is fitting into workflows. One big finding has been that the clarity of the data brought many new features to the foreground for users, giving them new reasons to use chartbeat. We hope that our customers have enjoyed this revision and let us know what we can do to keep improving it.