Newsletters continue to be a valuable tool for cutting through the noise of shared online spaces and engaging audiences in direct and meaningful ways. To learn from a couple of experts in the field, Chartbeat’s Brad Streicher sat down with Elaine Piniat of Reuters and Maya Neria of The New York Times at The Audiencers Festival in New York.

They talked about how their roles help bridge editorial and business teams, how newsletter growth correlates to site KPIs, and what they find challenging about their work.

Can you start by walking us through your roles? 

Maya: I’m the Director of Product for Messaging. Messaging here refers to both push and email. Our work is really to be a bridge in many ways between the business side and the newsroom with email strategy.

We have over 150 newsletters that span lots of different categories. They also vary very much in format. Certain newsletters are really all about the inbox and very much about journalism in your inbox. Other formats are really focused on optimizing engagement with our on-site journalism. We live in a world where there is really limited real estate, especially on Mobile, so we really see emails as an extension of the homepage and a way to personalize the homepage for readers.

Elaine: I’m the Newsletter Audience and Revenue Strategy Manager at Reuters. My role is a bridge role with all of the different departments working on newsletters–Editorial, Product, Design, Sales, Marketing, Data–and I essentially act as a product manager. When I started at Reuters, I was the only employee working full time on newsletters, and now we have a cross-functional team that’s made up of individuals from all these departments and we’ve hired a newsletter editor to be my editorial counterpart.

Editorial and sales are the two departments that I work the most with. On the editorial side of things, I work with the team on building templates, A/B testing, surveys, and communicating those findings. And on the sales side, I’m working with planners and sellers on pricing, data, and providing audience info for pitches.

At Reuters, our goal is to attract professionals in the areas of sustainable business, auto, health, and legal. Essentially, what we do is we’ll have deep coverage with a specific topic, and we’ll marry that to a newsletter. 

What correlation have you seen between newsletter growth and hitting key metrics on the site itself?

Maya: I think email plays a bigger and bigger role in engaging readers that would have otherwise come organically via search. And we’ve seen huge success in terms of email being a situation lever for folks. We’ve done a lot of analyses that show real incremental impact when it comes to both conversion and retention. And that’s true for both emails that drive readers to site as well as emails that are all about inbox journalism. We tend to subscribe to a philosophy that our subscribers should have no friction so our subscriber-only newsletter portfolio is really about the most premium feel in the inbox. We’re not trying to get them to click into the site by design.

More of our free offerings tend to try to drive people to the site and we have a smart meter that will serve a paywall when it makes sense. For KPIs on the newsletter side, we look at email clickthrough rate as a leading indicator. We then track weekly active users, and we’re also tracking conversion and retention. So every single time we launch a newsletter, every single A/B test that we’re making on major formats of newsletters, we’re really able to test those lagging organizational KPIs and be able to attribute real email impact to those.

Elaine: For Reuters, when I first started our focus was on the surface-level metrics – open rate, click through rate – and to find more meaning in newsletters we transitioned over time to site metrics. We’re setting goals against sessions and estimating revenue from users who click from newsletters to the site. We’re paying closer attention to that impact. Clicks and sessions are key for us. Over the last two years clicks have grown 39% and pageviews have grown 50% so the growth is there. 

We’re also seeing that newsletter subscribers are much more active on site than the average visitor. They have 25% more engaged time on site, 75% more pages per user, and a lower bounce rate. So all of these metrics that are kind of core to measuring engagement and loyalty are higher with our newsletter subscribers, and we’re still looking for ways to measure loyalty in a more meaningful way, but we see the value there in our newsletter subscribers.

How do you conceptualize new newsletters?

Elaine: We consider this as a Newsletter Cycle, where these things kind of happen all at once but can also be considered as steps to work through.

• Grow the list of newsletter subscribers, whether through smart sign up points, social media, outreach, etc
• Engage audience once you launch, with a welcome series, strong subject lines, compelling content
• Of course, you’ll inevitably lose users overtime – those who unsubscribe or through list clean ups
• So you have to figure out how you can re-engage users with re-engagement emails for instance
• And ultimately optimize the newsletter itself through testing, analyzing data, surveys
• …before going back to growing the list

What tips do you have for publishers who want to start adding personalization to their newsletters?

Maya:  I think the real opportunity is how you match the audience and the content, and that’s where we’re really investing our resources. For instance, we believe geopersonalization is a strategy worth investing in, and have recently launched “Your places: Extreme weather”.

Obviously that’s very resource intensive so we’re also developing models that are not about one-to-one personalization but one-to-many. So even just saying, “What does a registered user get versus a subscriber?”, or on the geopersonalization side, “What can we offer international readers” and that’s a much easier personalization problem.

So, going back to that example of a subscriber-only newsletter that’s not really about driving readers to site, you can create a version of that newsletter that’s all about driving people to site, and it’s the articles that were referenced in that email, but now in a digest preview that you send to the registered list. It’s a way of doing this that’s not as difficult to pull off as one-to-one.

We’ve talked a lot about some of the successes that we’ve seen, but I think we would be remiss to not address challenges that you’ve seen in your roles as well.

Elaine: Getting resources can sometimes be a challenge. Newsletters are a small piece of what the rest of the team does so when we have a vision for our newsletters, we may not be able to do it in the time span that we were hoping to.

Another is the accuracy and truth behind data. When it comes to open rates, when it comes to clickthrough rates, and, if you’re using a tool like SailThru, metrics that now filter out Apple MPP and bots, how much do we trust those metrics? How do we disseminate those metrics? Do we share that information, and when do we share it? We don’t want to discourage people who are working on newsletters when they see the truth behind some of the metrics.

Maya: One is figuring out what the business needs are and what the user needs are and how you get those people aligned. There are times when the business wants to guide as many people to the site as they can, and if you’re working with a newsletter editor, they want to make sure that they have the best journalism in the inbox. Navigating those tensions is really important.

I think the other thing from a product perspective is how do you not get stuck in a world where you’re just doing incremental improvements–small A/B tests that have 1%, 2%, or 3% lifts– and give yourself the space to think bigger and actually get into a larger, potentially riskier area. I think that’s where we are with geo-personalization, both the infrastructure that we have to build to support that and the culture that we need to change.  Figuring out how you actually give your team the space to innovate there is really challenging, because it’s much easier to just continue to do A/B tests and just get a little bit better and a little bit better.

While striking this balance is the responsibility of a good editor, one of our responsibilities is to uncover data and insights that make those decisions easier. It’s in that spirit that our Data Science team recently investigated the relationship between word count and engagement. 

We sought to find out how Average Engaged Time changes as word count increases and how editorial teams can use this data to optimize their content. More on those findings below.

Average Engaged Time increases with word count, up to a point

In the undertaking of this study, Chartbeat’s data scientists analyzed millions of articles of 10,000 words or fewer that were published between January 2019 and April 2022. With a global network of publishers, languages, and grammar rules in the same dataset, this was not as simple as scraping HTML and calculating engaged time, but our team was up to the challenge and delivered a trove of data illuminating the relationship between article length and engagement.

When we plot Average Engaged Time by word count, two clear patterns emerge:

1. Between 0 and 2,000 words, Average Engaged Time increases as word count increases.

2. Once word count grows beyond 4,000, the variability in engaged time also grows, and the return on additional length is less certain.

In other words, we can confidently say that for articles of less than 4,000 words, the longer the article, the more engaging it will be. Beyond 4,000 words, however, the interval of expected engaged time varies much more widely, and we can no longer conclude that engaged time will increase with word count. 

While this doesn’t mean that there is no more engagement to be had beyond this point, it does mean that performance will depend more heavily on how well optimized a page is for engaged time after publication. This data also allows you to benchmark your articles of various lengths against our global averages.

Decoding the binned scatterplot

Since we frequently use bar and line graphs in our research presentations, it’s worth pausing here to explain the graphic above. Rather than plotting millions of articles on one graph, the binned scatterplot uses one dot to represent the Average Engaged Time of all the articles published at a given word count. For dots that are intersected by a line, this denotes the expected range of engaged time for an article of that length. From 0 to 2,000 words, the variation is only hundredths of a second and the dot is essentially also the line. Around 10,000 words, the range visibly stretches to almost 9 seconds.

If we look at the 6,000 word mark as an actual example on the graph, the dot tells us that the Average Engaged Time for all articles of 6,000 words is 80 seconds. The line through the dot then shows us that the least engaging articles of 6,000 words receive about 77 seconds of engaged time, and the most engaging receive about 83 seconds.

The majority of articles in our network are far less than 2,000 words

After discovering that articles between 2,000 and 4,000 words get more engagement than shorter articles, we wanted to see what proportion of total articles fall in this range. Not only did very few articles register more than 2000 words, but the majority of articles came in at fewer than 500 words.

While the Average Engaged Time for 500 word articles is in line with recent global engagement benchmarks, the difference between a 500 word article and a 2,000 word article is nearly 30 seconds of engaged time. 

Word count and engagement by loyalty

Since engaged time is a strong indicator of loyalty, we also looked at this data by visitor type. We found that the average percentage of loyal readers, or readers who have visited a site at least 8 of the last 16 days, is highest on articles of 2,500 words or fewer. 

We’ve previously found that loyal readers tend to read more pages per visit but spend less time on individual pages, so it’s not surprising that longer articles would see more new and returning visitors.

Word count and engagement by device type

When we look at word count and engagement by device type, we see that the crest of Average Engaged Time for both devices occurs around the same word count as the larger study. 

Though mobile referrals generate more pageviews than desktop, they haven’t quite caught up when it comes to engagement. As the graph shows, this doesn’t mean that readers aren’t willing to read longer articles on mobile, and engagement strategies should be mindful of the different experiences by device as you execute your optimization strategy. 

Using this data to optimize your content

As you incorporate these findings into the other data and experiences informing your audience engagement strategy, remember that curation and optimization – not word count – will ultimately dictate how much of your content gets consumed. These benchmarks are an additional tool for diagnosing article performance, and when used with real-time analytics, can make your content more engaging before and after publication.

Here are our key takeaways from the research:

1. Between 0 and 2,000 words, Average Engaged Time increases as word count increases. 

Our analysis shows that up to almost 4,000 words, the longer article, the more engaging it will be. If your articles are falling short of the benchmarks we’ve shared, a real-time optimization tool like our Heads Up Display can show you how far readers are scrolling and give you an opportunity to make changes at the point of exit.

2. Beyond 4,000 words, variability in engaged time grows, but that doesn’t mean there’s a ceiling.

As we see with our year-end list of the most engaging stories, unique topics can require more depth than daily reporting. This doesn’t mean you should shy away from covering them. It just means you’ll need to devote more attention to optimizing these pages for engaged time.

3. The majority of articles published across our network are less than 500 words.

Not every story can fit snugly between 2,000 and 4,000 words, and we’re not suggesting they should. Articles of all lengths have their place in your content strategy, and can be used together to increase recirculation and engagement. For example, if a brief piece of breaking news goes viral via Search, use well-placed links to drive that traffic further into your site to more engaging articles on similar topics.

4. Engagement should be evaluated on all types of content.

Even after all of our reporting on the way Average Engaged Time increases as word count increases, this article totals only about 1,300 words. If we had additional data and insights that added value, we would gladly include them and expect to further engage our readers. Since we do not, in the days after publication we’ll be monitoring our Historical Dashboard to see if it’s generating around a minute of Average Engaged Time, the benchmark for articles of similar length.

---

Chartbeat is the leading content intelligence platform that gives publishers the data to build loyal audiences with real-time and historical dashboards, in-page optimization tools, and robust reporting across desktop, social, and mobile platforms.

Read more research and insights on their blog or book a demo of Chartbeat’s platform.