I'm Alexander Held, Senior Data Scientist at DER SPIEGEL, and this is the third article in a series on our audience engagement strategy. 

> In part one, we explained how we built our engagement score, a blended metric for measuring reader interaction (link to part one). 

> In part two, we showed how combining the score with surveys and interviews helped us identify our most important engagement drivers (link to part two). 

> In this article, we describe how we apply the score across our product and channels, and why a one size fits all approach to a diverse audience is inefficient for sustainable revenue growth. 

Short recap: Our engagement score ranges from 0 to 100 and gets calculated for  every user who has visited spiegel.de in the last 30 days, subscribers and non subscribers alike. By combining signals such as visit frequency and reading time, the score provides a holistic view of involvement and defines four audience segments:  Occasional (0–25), Regular (25–50), Active (50–75), and Fans (75–100). 

We use these segments to tailor subscription offers, adjust our paywall to balance reach and conversion, prioritize product features that drive engagement (especially during onboarding), and improve targeting on external platforms. The aim is a more relevant experience for casual readers and our most dedicated fans, which supports short term conversions and long-term retention.

The following is a collection of our key use cases. 

Dynamic offers 

We built a dynamic offering system that adapts subscription offers to audience  segments. Experiments showed three things:

• Each engagement segment responds best to different offers
• Higher engagement correlates with interest in discounted long term plans
• A notable share of orders, especially among higher-engagement readers, comes from former subscribers. 

Based on these findings, we introduced dynamic pricing. Footer banners and paywalls now present tailored offers aligned with each reader’s engagement score. This moves us away from one size fits all and toward a strategy that maximizes both initial conversion  and long-term retention, ultimately capturing the highest possible revenue per reader  over time. 

From left to right:

• Paywall with tailored offers for low engaged users
• Paywall with tailored offers for mid engaged users
• Paywall with tailored offers for high engaged users

While dynamic offering has shown promising results, challenges remain. For occasional readers, while conversion rates improve with low-barrier offers, retention rates remain low, underscoring the need for improved onboarding and engagement strategies to  sustain these subscriptions. 

Dynamic paywall 

We use the engagement score to adjust access to premium content by reader  segments. Our experiments showed:

• Highly engaged users had a 54% higher conversion rate when exposed to more paywalled articles on our front page
• Even the lowest engagement segment saw a 19% lift.
• But as expected, paywall exposure also reduced article views and time spent, especially among highly engaged readers.

Overall, conversion gains outweighed the losses, but we are still working to optimize this tradeoff  (more on that in the outlook). 

Nevertheless, the results show that a dynamic paywall converts highly engaged users efficiently while not (too) significantly deterring less engaged readers. It helps us  balance reach and monetization. 

Prioritizing product features that drive engagement 

To focus development on features that matter, we ran audience research using  interviews, surveys, and behavioral data analysis. This work surfaced the key drivers of engagement. You can read more about our approach here. 

We are now increasing interaction with these drivers through product changes, including targeted nudges in onboarding journeys and greater visibility on the front page. In A/B tests, a  tooltip-guided tour outperformed a banner-based onboarding, lifting the average engagement score by 9 percent and increasing usage of the features introduced during onboarding. ** 

We also restructured our email onboarding based on the study’s insights. These tests have not shown a meaningful uplift yet, but we are evaluating next steps. 

Other use cases and the path forward 

We export the engagement score to external platforms such as Meta, enriching platform profiles with on-site signals for more precise audience targeting. Furthermore, we use the score internally to enrich surveys with engagement data and to report how segments evolve over time, giving marketing, editorial, and product a shared metric to align growth, retention, and reactivation. 

Next, we are combining the engagement score with additional signals, such as content interests, so personalization reflects both involvement and intent. In parallel, we are moving beyond a short lookback by incorporating long-term engagement and the increasing reality of re-subscriptions. To support this, we are testing a proof-of-concept for a customer lifetime value model that feeds into the same use cases: offers, access, onboarding, marketing (and even more). The goal is a longer-term perspective that captures sustained engagement and repeat subscriptions while making the product more relevant for every reader.

Thanks to Simon Schwandner for proofreading and Valentin Espagné for the images (and for driving so many of our A/B tests and innovative experiments at SPIEGEL).

The post How DER SPIEGEL uses its engagement score: our key use cases  appeared first on Audiencers.

Regular commenting below articles has always been a frequently-used engagement feature on DER SPIEGEL. In fact, Product Manager Laura Badura shared that at the peak 1.7M comments were left in a month… too much for AI and human moderation to manage and too much for users to read . 

Not only this, but conversations on a single topic were happening in multiple places across the site, diluting the potential value of this feature. Debates on the US elections for instance took place in the comments section of several articles covering the topic a day. 

So, when the time came to review the feature and evaluate tools used, Laura and the DER SPIEGEL product team decided to run some user research to understand the popularity of comment sections, why users turned to DER SPIEGEL for debating instead of X or Reddit, and find ways of improving the feature.

A lot of research later… 

Quantitative analysis meant asking questions such as “How much do you use the space?” and “Do you prefer active or passive participation?”, revealing that only 5% of users are commenting on a regular basis.

This is supported by industry research in Germany by Reuters Institute. The graph below shows active participation in news reporting since 2013, with the yellow results representing commenting.

Qualitative research found that users associate trust and safety with these conversations when they happen under the DER SPIEGEL branding. There’s a mutual appreciation for the publisher’s work, and a reputation that users appreciate. Moderation means that users know that not every comment gets through to be published, and that journalists can vaguely guide discussions if needed. Overall, interaction options are highly appreciated by users as they have a clear desire to engage and know others’ opinions.

For those users who haven’t yet participated, the term ‘safe space’ kept coming up, particularly amongst women. Some mentioned interest in using the feature but a worry of publishing under their real name as well as deliberate choices over which topics they want to participate in.

Re-building the feature with this research in mind

To build trust with users and ensure they feel confident that they’re in a safe space, clear branding and custom design allows the debates to fit perfectly into DER SPEIGEL’s site.

Their tool for implementing debates into the site, Logora, allowed for plenty of features, but it was important for DER SPIEGEL to evaluate which would be the most relevant to their audience. 

For instance, one of Logora’s features is a form of rewards system for high commenters that sends a congratulatory message after a certain number of comments. But the team decided not to implement this at the launch to give readers the chance to test and interact with the debates un-interrupted. However, today, one year on from launching, and they’re considering ways of rewarding high engagement such as this. 

The team also decided to omit ‘dislikes’ on comments to support the creation of a safe space. 

Interestingly, they’ve also decided against making it obligatory for commenters to use their real name to allow users to feel safe. However, they have an account verification process, even for free members, and are continuously evaluating this decision. 

Real name commenting at The Times

At The Audiencers’ Festival London, The Times & Sunday Times shared how they did the opposite, aiming to reduce toxicity by reducing anonymity in their comments section. The idea: “people should know who they’re speaking to if you want a high quality conversation.”

At the time, only 37% of commenters were using their real name, today they’re at an impressive 98%.

“It was a long process involving many different departments at The Times, and lots of communication around it. Subscribers were told early on that they needed to use their real name, and there was a 3-step process to do this. But since making the change we’ve seen toxicity come down – it’s a safer, more welcoming community, and attacks on writers were down 25% – and an increase in the number of comments. People who don’t use a real name stick out for a bad reason – they’re now blocked and then become compliant.“

Debating in practice

For SPIEGEL Debatte, you only need to create a free account or log in to read the debates. 

But in order to actually comment, react, see the distribution of votes (before the debate has ended) or suggest a debate topic, you need a paid subscription. 

The feature is promoted in subscriber onboarding to increase regular usage amongst subscribers.

And debates are integrated into editorial content and vice versa to encourage recirculation. Notice the high visibility of premium (S+) content here to promote the value of subscription.

This premium article below is accompanied by a debate topic open to all, allowing free members to engage in the debate to an extent without being able to benefit from the full experience and accompanying content – a great balance between engagement and frustration to encourage conversion.

To reach new users and draw attention to the digital platform, they sometimes show comments from SPIEGEL Debatte in the print magazine and link to the Debate Space via a QR code. And finally, they also draw attention to the debate on social media channels such as LinkedIn or Instagram by publishing quotes or voting results, linking to the debate.

The goal wasn’t necessarily to increase conversions or retention, but commenting and interactive engagement has been an important feature of DER SPIEGEL’s site, so improving this feature was important, especially for subscribers who have full access to debating.

Some initial results

• Within the first 7 hours of launching, 10,000 users logged in within the debate space
• In the first year, the team created over 900 debates with more than 3M votes and about 400K comments.
• The quality of comments has increased with feature changes. “That is a qualitative observation for now. But it is also reflected in the rate of accepted comments, which has risen from 80-85 to 85-90 percent.”

Thank you Laura for taking the time for this interview!

The post Going further with interactive engagement: introducing the debating feature appeared first on Audiencers.

This article was co-authored by Alexander Held,  Senior Data Scientist, and Angelika Zajac, qualitative researcher in the audience research department, at DER SPIEGEL. 

In the subscription industry, keeping customers engaged is key to success. Earlier research indicates that engaged readers are more likely to subscribe and less likely to cancel their existing subscriptions. So how do we manage to increase the frequency and intensity of our users in a provable and sustainable way? 

Besides our efforts to improve our product and business through continuous A/B testing, we are committed to gaining a comprehensive understanding of what drives engagement. Therefore, in a recent audience research initiative at DER SPIEGEL, our objective was to identify key engagement drivers. We aim to understand which editorial and product features lead to regular and deep usage, so that we can promote and explain them during onboarding and/or incorporate them into our product design considerations.

To obtain answers to these questions and identify key engagement drivers, it is important to understand the factors that influence a sporadic user’s transition to an engaged reader, including what product knowledge engaged users have and which components they have habitualized in their regular use. We do this by analyzing which features our most engaged readers use and value. 

This text outlines the methodological approach we used to answer this question.

> Also by the DER SPIEGEL team: How we measure audience engagement at DER SPIEGEL

Creating a laboratory 

To identify key engagement drivers, we established a laboratory-like environment through three conditions described below: 

Condition 1: We start by identifying all features available to our users. “Features” can be understood as functional containers, aimed at enhancing user experience and interaction with our platform, giving users a clear and structured way to consume our content. As news constantly changes, editorial topics are explicitly not covered by these features.

Condition 2: We establish a laboratory-like setting by focusing on features that are consistently accessible on our site. Additionally, we concentrate on our most engaged users, identified through a segment based on our engagement score. This way, we create a laboratory environment characterized by the continuous presence of features and active user participation. 

Condition 3: We set a threshold: A feature is considered “regularly used” if it is used by a user at least once a week (over the course of four weeks). 

How do we define audience engagement? 

We use an engagement score as our central metric for audience engagement. Every user (with/without subscription) who has been on spiegel.de in the last 30 days gets an engagement score attached to their user profile. The score is between 0 and 100, the higher the score, the higher the engagement on spiegel.de. The score uses various key figures (e.g. frequency of visits & reading time over the last 30 days) and merges these into an overarching metric. The engagement score can be used to divide readers into segments according to their level of engagement:
- Occasional: 0-25
- Regular: 25-50
- Active: 50-75
- Fans: 75-100

Methodology

Based on the laboratory conditions described above, we have chosen a mixed methods approach which enables us to gain a comprehensive understanding of our user base, their behavior, and their needs. 

Hypotheses generated from guided in-depth interviews are incorporated into a subsequent survey to gain insights into which features our users are aware of, which they use and how they rate them. Looking at the behavioral data not only provides us with information about which features are used regularly and most frequently, but also tells us whether the behavior of the survey participants matches the information in the survey or whether it deviates from it. The findings from the interviews might offer explanations for these deviations. 

Results 

A usage-preference matrix forms the central presentation of results for this audience research project. All features are mapped in space using two axes and can therefore be compared with each other by their positioning. The Y-axis shows the observed usage values from our website tracking tool, while the X-axis shows the preferences that fans indicated in the survey.

From the ratio of behavioral and survey data eight feature groups or individual features can be identified that boost user engagement. The following interpretation is supported by a) findings from the in-depth interviews, b) by analyzing specific sub-segments, like young readers, subscribers or low engaged readers and c) by several findings from previous research: 

App & Push 

• Around 75% of fans use the app, making it a significant driver of engagement. It also serves as an engagement booster across different age groups. 
• Push notifications are regularly used, particularly among fans under 40. Interviews showed room for product optimization, as there is a desire for more customization of which push notifications users would like to receive. 

News & Update 

• News features like our headlines section or our daily morning briefing “Lage am Morgen” are highly popular among SPIEGEL fans and are frequently accessed. 
• All news formats are currently freely available and used by both users with and without a subscription or login. 
• Live blogs (not shown in matrix) are especially favored by fans under 40. 

Opinion 

• Comments and columns are highly valued and regularly read. Even though positioned less prominently, it became one of the most sought-after product features on our front page. 

Debate 

• Interaction options are highly appreciated by users as they have a clear desire to engage and know others’ opinions. Our newly created one-topic-a-day debate feature attracts younger (under 40) users compared to the older comment section centered around each article

Quiz 

• Our quiz has particularly good regular access numbers and preference ratings. It also works well for users with varying levels of engagement, including those without a login. 

Video 

• A relatively high percentage of fans regularly watch videos, although it is not deemed as important based on preference ratings. 
• Regular video consumption is higher among users under 40 and videos have one of the highest retention rates among all features. 
• We know from interviews that people particularly enjoy watching videos if they offer clear added value to what they have already read. A broken dam or the facial expressions and gestures in a political discussion can be conveyed more impressively via the medium of video.

Thematic Entry 

• Thematically focused pages have high access rates among fans. Usage patterns suggest they are used when readers have ample time, possibly without paying active attention. 

Recommendation Boxes 

• Certain recommendation boxes, like “The Best of SPIEGEL+” and “More on This Topic,” are highly utilized, especially among subscribers.

What’s next? 

Through behavioral analysis and audience research, we have shown editorial and product features that have a proven positive impact on engagement. We now aim to increase interaction rates with these key engagement drivers. First, we plan to increase the number of engaged readers by strategically nudging those with lower usage towards key engagement drivers and support the adaptation of ritualized interactions with these features. We do this by highlighting them in our mailing and onsite onboarding. Secondly, our objective is to enhance the visibility and ease of access to these engagement drivers on our front page. 

Research Process and Learnings 

This project provided significant insights and learning opportunities. For the first time, we documented all features on spiegel.de. This comprehensive inventory helped us understand our product offerings and identify areas for enhancement. 

Combining different research methods proved highly effective. Our mixed approach gave us nuanced qualitative insights and robust quantitative data, offering a well-rounded view of user behavior and preferences. By correlating interview findings with survey results and behavioral data, we validated our hypotheses and uncovered more in-depth insights into user engagement drivers. Discussions between researchers of each method were particularly fruitful. 

The project generated interest across various departments at DER SPIEGEL. Its broad scope, touching on editorial, product development, marketing, and user experience, provided a holistic view of engagement across the organization.

Thanks to Jakob Halm, Sophia Egbert, Henning Bumann and Simon Schwandner for editing and proofreading. Special thanks to Oliver Scholz for his major contribution to the quantitative work in this study.

The post Audience Research: identifying Key Engagement Drivers at DER SPIEGEL appeared first on Audiencers.

By focusing on delivering a meaningful and relevant experience to our readers, DER SPIEGEL has built a loyal and engaged community of subscribers who actively seek out our brand and value the content we provide. We are successful in acquiring new subscribers, yet we are facing a significant challenge in maintaining the retention of these readers. Some of our subscribers stop engaging with our content, leading to churn. This article dives into how we measure audience engagement and its impact on key business metrics.

Audience engagement metrics

A method we employ for measuring audience engagement is the blending of various metrics to create a comprehensive indicator. This concept, initially popularized in the field of e-commerce, involves the renowned engagement score known as RFV (recency, frequency, volume), pioneered by the Financial Times. Using an engagement metric empowers us in two significant ways:

• Firstly, we strive to objectively measure our audience’s preferences and gain a deep understanding of what drives user engagement. This entails identifying specific elements within our editorial content, product features, and user behaviors that lead to consistent engagement with our platform and offerings.

• Secondly, we use the engagement score to categorize our readers into distinct segments based on their levels of engagement. This segmentation allows us to provide personalized website interactions, such as targeted communication on pricing and content, tailored to the unique needs and interests of each reader group.

Before implementing these strategies, our approach to crafting an engagement metric consisted of two primary components: establishing a robust multidimensional methodology for measuring engagement and ensuring that engagement KPIs seamlessly aligned with business metrics, such as churn.

> You’ll also enjoy: From newsletters to predicting possible cancellation: tactics to reduce churn from The Audiencers’ Festival Madrid

Establish a robust multidimensional methodology for measuring engagement

While the traditional RFV factors have long been the standard for measuring user engagement, DER SPIEGEL has charted a distinctive course by introducing a novel element into the equation: Regularity. We go beyond the conventional practice of simply monitoring the frequency and volume of user visits to our site; we dig deeper, delving into the distribution of these visits over the past thirty days. In doing so, we recognize the influence of regularity in shaping user habits – a perspective that resonates with SPIEGEL’s overarching business objectives, embodied in the concept of our loyalty metric called Pay Day Rate. Moreover, this approach draws inspiration from contemporary research, underscoring the pivotal role that regularity plays in the formation of user habits.

The Pay Day Rate reflects our changing strategy. At first, we were mainly focused on getting new subscribers for our subscription business. However, as time passed, we started paying attention to cancellations and the idea of “shelf life”. By 2022, we took a significant step forward by developing this new loyalty metric that has become central to our mission and revolves around assessing how many of our subscribers engage with our site consistently on a daily basis over the course of seven days. A remarkable revelation surfaced from our analysis: fifty percent of our subscribers visit our outlet every single day of the past week. This transformative shift in our approach towards user engagement reflects our dedication to high-quality content and our proactive efforts to build a loyal readership.

A further fundamental shift from the classic RFV model is the way volume is measured. Instead of focusing solely on clicks on articles, DER SPIEGEL considers usage time across the entire site. This means that user engagement is evaluated by how deeply they immerse themselves in editorial content, spanning various formats such as text, video, stories and audio consumption. Frontpage scanning also contributes significantly to regular usage and habit formation and is incorporated into the calculation.

To make all these factors (frequency, volume, regularity) comparable, they are transformed into a value range between 0 and 100. This transformation involves the use of an empirical cumulative distribution function (ECDF). ECDF transforms numerical data, indicating the proportion of values that are less than or equal to a specific value. Given that most readers tend to have relatively low frequency and volume, the ECDF transformation focuses on making changes in the lower and middle value ranges more measurable. This is a deliberate choice to prioritize the development of readers with lower to medium engagement, as increasing the engagement of highly active users is less critical. In research on media use habits, this non-linear relationship is also evidenced (“The relationship between the number of repetitions and habit strength is not linear, but with the number of executions of the media behavior, the increase of habit strength decelerates.”- from The process of forming a mobile media habit: results of a longitudinal study in a real-world setting).

Regularity is quantified by the number of unique days a user visits spiegel.de in the last thirty days. 

These values are also standardized within a range of 0 to 100 and transformed using ECDF. Recency indicates when a user was last on our site. These values are also standardized between 0 and 100 without additional transformations. Thus, a user who visited spiegel.de yesterday gets a 100, while a user who hasn’t visited in more than 30 days receives a 0.

Rather than simply multiplying volume and frequency, as is done in the traditional RFV model, SPIEGEL’s engagement score assigns weights to the individual factors (frequency, volume, regularity). This allows us to flexibly determine the importance of regular versus deep usage. 

Furthermore, recency remains a key factor, much like in the RFV model. Each day without user engagement diminishes the weighted score from frequency, volume, and regularity by the same proportion, underscoring the importance of keeping users consistently engaged.

Finally, every spiegel.de user, whether they have an S+ subscription or not, is assigned an engagement score based on their activity in the last thirty days. The final score falls within a range of 0 to 100, with higher scores indicating greater engagement with our site. It is updated daily and integrated into Adobe Analytics & Target, enabling segmentation, analysis, and testing. In cases where a user access spiegel.de from multiple devices and is also registered, the score is calculated across all devices, ensuring a comprehensive view of the user’s engagement. The overall score is then transmitted to all end devices used by that user.

Alignment Between Engagement and Business Metrics

Boosting engagement is not an isolated mission. To ensure that we’re moving in the right direction, the engagement score must align with our core business metrics. One crucial aspect of this alignment is recognizing the impact of low engagement on user retention. When users exhibit low levels of engagement, it raises a red flag. It’s a warning sign that they may be on the verge of disengaging altogether, which, in turn, increases the likelihood of subscription terminations.

Looking into our data, the numbers paint a revealing picture of our subscriber base and their engagement levels: Only seven percent of our subscribers fall into the category of having an engagement score below twenty. Still, we observed a distinct trend where forty percent of cancellations were initiated by low engaged, full-paying subscribers. 

This data strongly signals the need for us to prioritize strategies aimed at elevating engagement among our subscribers, particularly those with low engagement. By doing so, we can actively work to reduce the churn rate and extend subscription duration.

On the flip side, high engagement, especially in the lead-up to and during subscription, is also a promising sign. It’s laying a strong foundation for a lasting relationship. We found that users who engage deeply with our content and platform before or while subscribing, tend to exhibit an extended subscription period.

A further look at the data showed, when comparing highly engaged readers to their low engaged counterparts, we observe a distinct divergence in subscription choices: High engagement readers are more inclined to opt for long-term subscription plans, spanning three and twelve months, as opposed to monthly cancellable subscriptions (not shown in graph). This signals that readers who actively engage with our content are not only more committed but also more willing to make a longer-term commitment to our platform. Building on this insight, one of our own survey results provide further validation: Readers with high engagement express a significantly greater interest in annual subscriptions.

Conclusion

The engagement score isn’t just a metric; it’s a strategic tool that bridges the gap between user engagement and business success. It empowers us to make informed decisions, refine our product and content strategies, and enhance user experiences – all with the ultimate goal of not only retaining our subscribers but also attracting new ones. Our next steps involve a deeper dive into understanding the causal factors that drive user engagement and use the engagement score to categorize our readers into distinct segments based on their levels of engagement. This segmentation allows us to provide personalized on-site interactions, such as targeted communication on pricing and content, tailored to the unique needs and interests of each reader group.

Architecture (Bonus content)

All visitors to spiegel.de within the past thirty days are assigned an engagement score, regardless of whether they are subscribers. To achieve this, we need to process large-scale behavioral data from thirty to forty million users daily, demanding the deployment of parallel processing pipelines in Spark. These pipelines run inside our Azure infrastructure, where raw data is sourced from the Adobe Data Feed and stored as binary parquet files within our Azure blob storage. Our compute infrastructure relies on a mid-sized virtual machine, while Azure DevOps is used for code repositories and continuous integration. Development is done within VS Code, ensuring seamless integration with our Azure ecosystem. This project’s scope includes using the engagement scores in Adobe Analytics and Target, to be able to personalize our product, conduct one-off analysis, and perform A/B testing. We used Adobe Customer Attributes, to be able to attach this high number of scores to our users daily.

Thanks to Henning Bumann, Simon Schwandner and Madeleine White for editing and proofreading.

The post How we measure audience engagement at DER SPIEGEL appeared first on Audiencers.