Designing Data – Part 5: Launch and Iterate

We’re asked about our design all the time – usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Today’s the last post in this series…though iterating never ends, so maybe there will  be more down the line. Stay tuned!

We make changes constantly. Seriously. Every day.

Sometimes they’re small and only our most dedicated users notice them; sometimes they entirely change the way people interact with our data. We don’t look at these changes as regressions, dead ends, or a waste of time; we look at this as the right way to design products.

We vet everything intensely, but we can only do so much in the confines of our office with a few dozen users in pre-launch testing. And the only way to know if we’ve succeeded is to put it all out there into the real world (even if it sometimes results in a few harsh Tweets headed our way).

So, when a new design is launched, we can’t just pick up and move onto the next thing. We listen. We read every email coming into customer service, we consider every tweet, we grab everyone who wants to talk to us about what we’ve done and we listen. We figure out where things rocked and where things sucked and then this whole cycle starts again.

If you’ve been following this design blog series, you know that in our initial design we tried to answer the question our users were always asking, “what’s special happening on my site right now?”  with the new dashboard Overview.

We stripped out a whole bunch of detail and put the focus on Notable Pages (an algorithm that detects what pages have unusual traffic) and Peer Stats (information from similar sites.)

With the combination of those two stats our dashboard was always able to alert our users when something surprising, unique, or interesting was happening.

But when it got out there into the wild…it turned out that people wanted to see more.

They didn’t realize it at first, but it wasn’t just “is something special happening?” that they wanted answered rather it was “is everything normal?”

While the questions sound similar, there is a world of difference between the two. We had built a dashboard that always caught and called out the big stuff, but it had trouble distinguishing between a normal day and a really bad one.

We thought users would turn to the other views by clicking down their left-hand nav to understand these more nuanced things, but we were quickly told they wanted to put these new things on a second screen to always have a pulse of their site, even if it was just to tell them that everything was going smoothly.

So we went back to into research mode. We asked what it meant when something was running smoothly, we watched how people used a dashboard when they weren’t paying attention, and we found out what screen they put our dashboard on.

We learned a lot of little things (like that people liked to put our dashboards on big screens, but there wasn’t enough contrast in the colors to display information well enough at that scale).

We got some bigger ones, too, that weren’t solved by changing colors.

It turned out it wasn’t just the surging pages they wanted to see… they also wanted to know if anything unusual was happening with their more always-trafficked content.  So we brought back Top Pages and showed them with the Notable ones as context.

Now the dashboard doesn’t just show you when something special is happening – it also lets you know when you’re doing alright and everything’s just fine.

But that only solved things on the individual page level when the information about the people sending it was every bit as important.

So we changed the visualization of Traffic Sources to show two new things: the top referring domains and search terms and how your performance today compares with your average performance.

Now, no matter if your day is normal, great, or terrible, you can tell right away. And that was the point all along.

So welcome to our new Overview page. We think it answers the three questions that were most important…

  1. How many people are on my site?
  2. What are they looking at?
  3. Where are they coming from?

 

…but maybe you don’t. That’s awesome. We want to know. This process won’t stop until we do…

Designing Data – Part 4: Design and Prototype

We’re asked about our design all the time – usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Yesterday we looked at the data, and today we’re talking about designing against it.

Once we know that we’re solving real problems and using good data, we get to the part that everyone sees, touches, and (hopefully!) falls in love with: The design.

While we love delighting users and get pretty jazzed when people ohh or ahh over what they see on screen, we make sure to have a few people on our product design team who are solely responsible for making sure that the design uses and highlights the data we’re displaying in the best possible way. So that it’s not just pretty, but usable.

Although this is the stage where pixels are pushed, lines of code are also written. Usually, we need to prototype a live design to make sure we can touch it, vet it, and sometimes get user feedback on it, regardless of how rough it is.

Whether it’s quickly building a navigation system to make sure it functions instinctively or prototyping a visualization with real data pumping through it to see what it looks like — a functioning, built design lets you see things that you never would in the static world of Photoshop.

For this very reason, every designer on our team is also a developer.

When pulling together the original Chartbeat dashboard Overview, we went through a bunch of prototypes that were pushed out to small groups of our users.

Well, they didn’t love everything, that’s for sure. That sucked. We’d spent hours/days/weeks designing and prototyping….and it just didn’t work. But we don’t try to make it fit or fall on our sword and go with it anyway. We made that first version of the Overview better.

From the beginning, our goal for this view was to provide a high level, at-a-glance view of the site so it was obvious if something interesting or unusual was happening.

Some of our early concepts were knocked down by users mainly because we weren’t answering the questions they were asking, the needs they had at that instant level.

Honestly, many of these concepts were visually much more appealing than the design we launched with but, as always, the goal isn’t only to make things pretty; the goal is finding the delicate balance between beauty, usefulness, and delight. To us, that’s the core of good design. That’s is what makes people who don’t have time for raw data able to understand, use, act on, and enjoy our products.

So what were those pretty (but not so useful) designs that didn’t make it for Traffic Sources?

The radar chart is a beautiful visualization and has the advantage of being able to quickly compare shapes, but it falls apart in the eyes of many users who are trying to get precise values of individual traffic sources.

A radar chart is intended to show balance, when in reality most sites don’t have (and shouldn’t have) an equally balanced set of traffic.

So we canned that guy and moved on to the dot chart.

The dot chart was definitely cool, but  it didn’t work for us a different reason. We liked the idea of being able to show a shape, or thumbprint, of your site’s traffic and added a connecting line between the values. The line didn’t have a value – it was just meant to aid the eye in spotting the dots easily.

But users were interpreting it is a relationship between each traffic source, which it certainly wasn’t meant to be. We took a hit on that one for sure. The last thing we ever ever want is to launch a misleading visualization!

Then came the circle chart. This guy had a lot of promise. Circle charts can be beautiful and usable, but its main weakness really shows when trying to compare values on different radii. The eye gets a little tripped up. Not to mention that it’s tricky to grasp at a quick glance – the whole point of the Overview. So we inevitably trashed this guy, too.

How did we land on our “final” design? Check back tomorrow to read our last piece of Designing Data: Launch and iterate.

Designing Data – Part 3: Look at the Data

We’re asked about our design all the time – usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Yesterday we identified problems through user research, and today we’ll look at the data to validate our thinking.

We’ve identified the problem. Next step is to start sketching or to open up Photoshop and start throwing pixels on the screen, right? Nope. Negative. Wrong.

We always look at the data before even thinking about design.

We’re a company that eats, sleeps, and dreams information and insight, and we design from what we have — not what we wish we had.

It’s so easy to spend days pushing pixels in Photoshop making the perfect design for the perfect scenario, but it’s all meaningless if the result doesn’t carry over to reality. Great mockups are easy. Designs that work well with real data of all kinds are hard. Some days, really hard.

When we were designing the Overview on the Chartbeat dashboard, we took each question people were trying to answer and made sure the data we were tracking matched up against those questions to provide real, unique value.

We questioned and vetted every decision as to whether or not to put each data point on the Overview, a secondary page, or not include it at all. We looked at client after client, page after page after page trying to find places where our data fell down.

  • People wanted to know where traffic was coming from. But did that mean dropping a list of each referring site was be enough?
  • How did the dashboard look if your site had very low traffic, like 0-10 concurrents?
  • What if it had a ton, like 500,000+?
  • What if everyone was coming from search? Or social?
  • What if you got a flood of traffic all at once? Would the visualization work? Would it tell you that something special was happening?

To some people it can seem like overkill. But when you’re working with data, you can’t make assumptions and expect the design to match every possible form of data across thousands of sites.

Otherwise, it’s a ridiculous waste of time. Which is why we’ve gotten into the habit of  ending meetings the second someone says “Well, if the data says ____” by responding with “Let’s go look at it.” And all gather around my screen, or Isaac’s (our chief data scientist’s), or Matt’s (our UI designer).

We tear apart every single concept that makes it to this second stage in the process. When they survive this grueling data-review round, the fun part begins: Designing and prototyping – which we’ll dive into tomorrow.

We’re On TV!

We’ve been working closely with our good friends over at Patch since Chartbeat was born, essentially. They’re always testing out cool stuff and truly understand how to put real-time data to work.

So when they come to us with new ideas of how to use their Chartbeat data, we pretty much always jump at the opportunity.

Our latest love child? Putting online data on TV. That’s right — three-screen Chartbeat, baby!

Patch and PIX11 News took their Chartbeat Big Board and used it to share what people are reading and talking about in New York, New Jersey, and Connecticut right now. Using online content data to inform offline/on-air topics. Awesome, right?

What ways are you guys using the Big Board? Up in your office? On your own personal second/third monitor? Share the Big Board love!

Designing Data – Part 2: User Research

We’re asked about our design all the time – usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Yesterday we covered the right mindset, today’s all about user research. Check in tomorrow for the looking at the data portion of our design process.


We don’t make a single move without making sure it will solve a real problem for real users.

Whether they’re sitting here at the Chartbase telling us what makes their jobs hard, on a screen-share showing us how they use the dashboard , or through behavioral data from our APIs — everything we do comes from real people and their real usage. Not best case, hypothetical personas who might one day open up our dashboard.

That’s why we’re so serious when we talk about learning from our users. They are as much partners as they are anything else and we just couldn’t deliver without them.

The biggest thing we’ve learned is to always be listening (ABL, if you will). And really listening. Not just hearing what they say and delivering that exact feature. But truly understanding what will make them better than they every thought possible. We can’t build anything until we really get that.

But understanding and solving are very different things. Don’t try to solve everything or even anything right off the bat.

In our early user research, we’re just trying to figure out the common problems people bump into.

We don’t ask “what don’t you like about our dashboard?” we ask “what makes you come to Chartbeat? Why do you want to see data at all?” The former results in feature requests while the latter uncovers the things that make life harder — and we’re way more interested in those.

Until we know why people turn to us we can’t solve their problems, we can’t make something that will make them better.

For example, in talking to people about our recent Chartbeat dashboard redesign we realized that, at the highest level, people had three questions they turned to Chartbeat for:

  • How many people are on my site?
  • What are they looking at?
  • Where are they coming from?

In short, they needed the pulse of their site.

So, naturally, we decided to build something that delivered just that.

Where next, you ask? Well, looking at the data of course! We’ll get there in tomorrow’s edition.

Designing Data – Part 1: The Mindset

We’re asked about our design all the time – usually in an incredibly kind way full of high fives and “how’d you do that?!”s but sometimes in a “ugh, did you even think about talking to a customer??” kind of way. So, we decided to give you a week-long deep dive into our design process in this “Designing Data” series. Stay tuned every day this week for a next step in our design process – and let us know what you think about every one of them.

Design is pretty important to us.

We have one job: Taking a crazy amount of data and turning it into something meaningful, so people on the front-line – people who don’t always have time for numbers – know exactly what to do and when to do it.

Without good design that’s just not possible. If it takes them 10 minutes to figure out what the numbers stacked in front of them mean, let alone what they should do with them, that data might as well not exist.

But it’s not just about pretty. Design is way more than the visual. Design is a process of understanding. Where do people want to be and how can we make it stupid easy for them to get there?

When we say design we mean everything: from the look of things when your traffic spikes to why you want to know that in the first place. Design is a holistic process that’s about people and solving their problems – and making it beautiful while we’re at it.

So we start, unsurprisingly, by talking to these people, watching these people. We don’t lock ourselves in a room and run through scenario after scenario with fictional, ideal users and refrain from putting anything out there until it’s completely finished. We actually get out there and find out what kind of problems people are facing and solve those first.

But we don’t stop there. After we’ve got a solid idea and confirmed that it works with a handful of our users we do something that’s a little bit riskier…

We iterate in public.

We put out stuff that isn’t done, and sometimes we’re not even totally confident in it. Because we – and our early testers – don’t have all the answers.

But we’re getting ahead of ourselves. Tomorrow, we’ll get into the nitty-gritty of the user research portion of the design process and explain all of this launch-into-the-wild stuff later this week.

Meet the Five Women You’ll Be Working for in 2020

Today, we had the most fun, rewarding, inspiring day at the Chartbase in…well…maybe ever.

We invited five young ladies from The Young Women’s Leadership School of Astoria to join us for a job shadowing day that the school runs for their sophomores.

TYWLS of Astora, Chartbeat (and G-raph), Tumblr and Betaworks crews

To be totally candid, we had no idea what to expect : Would they be used to using the internet? Would they have Facebook pages? Did they understand what a startup was? Had they taken HTML classes or surprise us by being JavaScript geniuses? Hell, they could be Chartbeat customers already for all we knew.

Because that’s the problem with technology in schools right now. It varies so much. And while the tech sector continues to blow up in NYC and all over the country/world….we’re not teaching it (at least not consistently) anywhere. Which completely blows my mind.

The Young Women’s Leadership Schools are doing an amazing job, actually. We learned that these gals had classes on social media available to them and a bunch of them had Tumblr blogs of their own. But still none of these girls knew what coding was. So we started there with our day.

We decided to walk them through some of the basics of what it takes to build a startup:

  1. Identify a problem
  2. Know your audience
  3. Have a simple idea
  4. Work with smart people to build a solution
  5. Market that solution
  6. Learn from your users and iterate

Not only did they get it, but they built their own start up! A project called StudentConnect that would be a website and app that is a student’s best friend – sharing everything from what’s for lunch in the cafeteria that day (complete with real pics of those Sloppy Joes) and real-time communicating snow days to giving your friends a heads up on that 3rd period pop quiz and a calendar of homework and upcoming after-school activities.

By the end of the day, these little ladies learned a lot of things. But the one comment that brought a tear to our eye “Before I came in here, I didn’t even know what coding was, now I want to learn it today. This was awesome.”

So, internet community, we’ve created a Tumblr of ways they (and everyone) can get started: http://usefulstartups.tumblr.com/

We’re asking you to submit other startups, apps, blogs, ideas that we can add to this page and constantly keep it growing, so the next generation has no choice but to beat us at our own game. And we’ll like it.

“Journalism: It’s Just Like Parenting”

Tony chatted with Sarah Marshall of Journalism.co.uk about the role of analytics in the newsroom.

They talked about everything from uncovering hidden interest points in lesser-known players during the NFL draft and Glenn Beck using online audience interests to inform his radio and TV content to the fear of the “tyranny of the popular” in the newsroom and how being great journalist is just like being a great parent.

Have a listen to it all, and let us know how you think data has impacted how you create and consume the news.

EU Cookie Legislation and Your Chartbeat Data

What’s a cookie and how does Chartbeat use them?
Most data providers track certain visitor behavior by using cookies. Cookies can tell things like the difference between a new and returning visitor. That way you can deliver content that makes sense for them.

That return-state of a visitor is the one and only way Chartbeat uses cookies, never for any kind of personally identifiable information collecting or monitoring. You can read all about how we pull data in our technical docs.

How does the EU Cookie Legislation affect Chartbeat?
Recently, the EU passed legislation governing how websites can collect and use these cookies (as well as similar visitor tracking techniques like HTML5 LocalStorage). This law discusses website visitors’ options regarding the cookies set on their browser.

While each country in the EU has yet to finalize how and when they’re putting the law into effect, the UK has done so already.

Lots of our client partners are in the UK, so we’ve made this transition an easy one for you guys. We’ve already updated our JavaScript to work without setting cookies.

Here’s how to do it. Within your Chartbeat JavaScript, just set an option in your configuration to disable cookies:

    _sf_async_config.noCookies = true

With noCookies set, the Chartbeat cookie (_chartbeat2) will never be created.

How does this change affect what I’ll see in my dashboard?
As we mentioned, Chartbeat currently only uses cookies to track if a user is new or returning. So, when you set noCookies to true, Chartbeat will not be able to tell if a user is new or returning. In the future, other metrics may depend on the presence of the cookie, but we’ll keep you posted on anything that does.

Depending on your implementation, you may have some visitors allow cookies and some not. If that’s the case, Chartbeat would report new and returning values for only the visitors who have allowed cookies.

As always, if you have any questions hit up our Chartcorps on Twitter or via email.

Your Site, the Socialite: Introducing Social View & iOS Tracking Beta

The world outside your website is tough to manage. You have die-hard fans extolling you in 140 characters. You have newbies who aren’t quite your target stumbling upon you via, er, StumbleUpon. And your regular guys constantly checking up on you through their iPhone apps.

For a long time people have looked at the social audience, in particular, as its own discrete segment living in a vacuum to be treated totally differently from the rest of your audience. But really, they affect your site traffic and engage with your content like everyone else, don’t they? Shouldn’t you know just how much they’re affecting your site? We sure think so.

So within the new Social View on your Chartbeat dashboard, you’ll see the whole world of your social visitors. There’s a ton of easy-to-digest information about what’s going on with your site as seen through the social web.

In a single view, you can see what people are saying about you, how many of your current visitors came from social channels, and what pages are powerhouses on social networks.

Social traffic can make the difference between an ordinary day and an extraordinary day. It’s important that you can now dig through that list of your most social pages and figure out what makes them different, so you can get the same kind of oomph out of the rest of your pages.

If you see a big spike in traffic at the same time as you’re blowing up on Twitter, embrace it – see what people are saying, engage in conversation, retweet. People want to interact with you – make it easy for them to know there’s a real person behind all that HTML…

…and behind all that iOS code.

Your iPhone and iPad app is kind of like an entirely different site. It reaches a potentially totally new audience for you. It’s important to track the differences between your site and your apps.

So, we’re beta testing an iOS app tracking feature, too. If you have an iPhone or iPad app, install away through your dashboard sites page and let us know what you think. (Remember: the key word is “beta,” so we’re looking for your feedback and understanding that it’s not done yet).

And don’t forget to sign up for the Chartcorps virtual workshop to get a more in-depth look at social, iOS tracking and everything in between.