FAZ's Donika Lilova spoke at The Audiencers' Festival London in June 2024 on their digital subscription journey and, in particular, about how they're working on making their paywall dynamic with regular A/B testing to continuously optimize performance. In this article Donika dives deeper into this strategy.
Hi! I'm Donika and Head of Paid Content and Deputy Chief Subscription at Frankfurter Allgemeine Zeitung (FAZ). My responsibilities include the management of the Paywall Team.
Digital growth is a core objective of our company strategy, and we are on track to reach our subscription goal of 300,000 paying digital subscribers by 2025, in part thanks to our A/B testing and dynamic paywall.
Product + Sales
In 2021, we merged the two departments, Product and Sales, into one customer-focused matrix organisation with mission teams aligned to the customer lifecycle. This restructuring enables us to better understand the needs of our users, cater to their requirements more effectively, make a more efficient use of our internal resources and reach our set goals in a structured manner.
The Paywall Team
My team is responsible for generating new subscribers through the paywall on the website and in our apps. Our team is cross-functional, comprising colleagues from the editorial, data, online marketing and sales, and product management teams.
We believe that digital growth is generated from the product, and our paywall team plays a vital role in directly contributing to our company strategy on the website and in the apps.
FAZ+
Our paid content offering, FAZ+ represents the most significant contribution to our digital growth. It's available to all users on FAZ.NET, providing access to over 1,000 paid articles with a straightforward entry and exit process.
We have successfully driven consistent and sustained growth for this offering, reaching nearly 140,000 paying digital subscribers in six years. Our digital growth remains on a strong upward trajectory.
Our core team DNA is formed by our test and data-driven product development and digital marketing activities. A/B testing is a central component of our day-to-day operations. It took us several years to develop a streamlined testing process with standardized forms and to build up a knowledge database with all the conducted tests and learnings. Our Product + Sales department currently manages around 60-70 A/B tests per year and our goal is to be able to fit in even more parallel tests within our testing pipeline.
Our objective on the paywall is to increase clicks and conversions in order to optimize our standard and increase customer lifetime value. In 2018, we began with two to five paid articles using our freemium paywall model. Since then, we have increased the number of premium articles and tested more dynamic paid content approaches for different audiences and segments, based on their propensity and willingness to subscribe.
> An essential read: How to dynamic paywall (it's a lot more simple than you think)
Dynamic Paywall Test
One of our more prominent tests worth mentioning is our Dynamic Paywall Test, which was conducted in 2023 and examined different engagement segments.
Test Hypothesis
The following test hypothesis forms the basis of this experiment:
If we increase the number of Paid Articles on our desktop and mobile websites,
then we will generate more paid subscriptions,
because the users see more paid articles and thus, a higher buying pressure arises.
Test Setup
The test was conducted as an A/B test with an even audience split. Our objective was to focus on desktop users. This year, we conducted another re-test for mobile users only and were able to confirm the overall tendency. The test was deployed using our CMS and Adobe Target, with subsequent data analysis conducted using Adobe Analytics and Microsoft Power BI.
A total of 30% of the content used in the test was free, meaning that users would consistently view these articles as such.
Forty percent of the content was paid and displayed as premium content with the F+ icon and a paywall.
The remaining 30% were dynamic articles, which were displayed as either free or paid content, depending on the user segment.
Learnings
The key takeaways from our main test are as follows:
- The introduction of a greater number of premium paid content items has been found to result in an uplift of 48% for users who demonstrate a higher level of engagement. This equates to approximately 20 additional paywall orders per day.
- The A/B test presented here demonstrates the importance of addressing our users on a granular level.
- It's recommended that users with a low level of engagement continue to be offered the option of a lower paid content subscription, with the aim of increasing their engagement levels and subsequently transitioning them to a paid subscription model.
Test Results (Detail)
| Unique Visitors | Conversion Rate Orders Clicks | Conversion Rate Orders Overall | |||
| Control | 50% | 1,06% | 0,11% | ||
| Variation | 50% | 1,46% significant winner | Uplift: 38,05% confidence: 100,00% | 0,16% significant winner | Uplift: 48,08% confidence: 100,00% |
We rolled out this test successfully at the end of 2023/ beginning of 2024 with a subsequent follow-up test conducted exclusively for mobile users. The results of the subsequent test also corroborated the initial hypothesis. There was a notable increase of nearly 10% in orders when more paid content was distributed via mobile.
What are our next steps?
In June 2024, we successfully conducted our first A/B Test Dynamic Paywall Offer based on a propensity-to-buy score we internally developed with our data team. The results there were very promising, which is why we are continuing our paywall journey towards scoring models and dynamic offers, while also optimising our paid content.
