Sep 29, 2016 by James Strategy

Asking Hard Questions

This is how I imagined it. Ben Chestnut, MailChimp’s CEO, sits me down and asks:

“So, what value does technical content bring to MailChimp?”

I’ve been at MailChimp for a little more than a year now. For most of my time here, I would have floundered at that simple question. In the back of my mind, I knew technical content was important. I knew my role was important. But I couldn’t articulate those ideas.

Simple questions are hard

Last year, our team got together to answer a similar question: “What makes a good knowledge base?” The conversation was difficult from the start because none of us had a clear idea of what a good user experience looked like. First, we focused on things we could measure: scroll depth, time on page, and click-through rates. By the end, we were just listing off old gripes: better navigation, more videos, a snazzy site redesign.

All of those things were important, but they didn’t capture our true mission. We’re here to help users, but the same could be said for any team at MailChimp. It just didn’t feel specific enough for us. I left the meeting less sure than when I started, and that theoretical CEO question was still picking at me. So I pored over research, hoping to find some guidance. An embarrassingly easy thought exercise emerged:

Imagine MailChimp’s technical content doesn’t exist. What looks different?

The answer was obvious—more support tickets, longer support wait times, bad customer survey results. All of which leads to frustrated users and more money spent on support efforts. And just like that, I knew the answer to the question.

Technical content reduces support costs, saving MailChimp money.

With this answer guiding us, our team was able to identify our key metrics: support costs and customer satisfaction.


A point of contention during our conversation was the proper role of metrics and data in our work. Will ROI and KPIs turn us into stuffy, corporate drones? Can we still focus on users’ best interests?

That concern is not completely misplaced. UX folks know dark patterns often emerge when we focus on bad metrics instead of users. But when used properly, data is just too powerful of a tool to avoid. The new metrics gave our team a focus and a baseline measure to improve. They let us make decisions based on real user experiences instead of anecdotes.

The focus also helped us discern which secondary metrics matter. Pageviews? Bounces? Sessions? Before we knew our key metrics, it was easy to get lost in a sea of data. We were never sure what was important. But now we had a simple heuristic: if a metric impacted support costs or customer satisfaction, it mattered.

Cheerlead your team

Understanding and measuring the value of our work was a big win. But it also had an unexpected benefit—our work got noticed more. With a clear mission, our team now had a clear message, and delivering that message to the rest of MailChimp became much easier. We no longer had to lean on vanity stats. To be honest, even our team wasn’t sure why pageviews or new article counts mattered. But happier users and fewer support tickets? That made sense to everyone.

We also overhauled several of our internal reports, keeping the metrics that mattered and cutting the cruft. Plus, we created a dashboard using Google Data Studio. And that allows us to share progress with our whole team—and the rest of the company—in an easy-to-read format.

More than cost cutting

Ben’s imaginary question no longer makes me sweat. But the answer has already led me to a new question:

Does our technical content do more than just save money?

I’ve never thought of technical content as a cost center. If that’s true, even our best work can only reduce costs so much. It’s a zero sum game. Luckily for us, Google’s ZMOT research confirms how good “how-to” content can make money. Clear, easy-to-find content, like good technical content, convinces users to try new features and sign up for new services. For companies like MailChimp, powering that type of growth is important. The revenue will continue to pay off long after cost savings have been wrung out.

Moving forward

Don’t be fooled. None of this stuff is easy. MailChimp has a big technical content team, lots of smart people, and plenty of support from leadership. We have every resource we need to succeed. And yet we still struggle with the value question.

On the upside, though, our quest for an answer was a great opportunity for our team to ask some other important questions:

  • Are we spending our time on work that meets users’ needs?
  • Are we tracking the right metrics?
  • What else can our team do to create value?

Those questions deserve good answers, too.