This article was originally published by Frank Bi on Hacks/Hackers, republished with permission.
> Journalism has to be right the first time. Product work learns more from what doesn't work. Frank Bi, AI Strategy and Product Advisor at Hacks/Hackers, argues that when products are held to the journalistic bar, nothing half-baked ever launches, so nothing ever teaches you anything > Build a public sandbox. A labelled subdomain (beta, labs) resets the expectations readers bring to the homepage while still producing real behaviour data. Bi launched the Minnesota Star Tribune's AI restaurant guide Culinary Compass on the main domain instead, where readers judged a four-week experiment as a Star Tribune guide > Set the end date before launch, ideally by picking something with a natural start & end. Bi's Fair Bot ran only the 12 days of the Minnesota State Fair, so nobody ever had to make the call to shut it down > Show your work. Newsrooms already have a practice for issuing corrections, so extend a version of it to experimental products and publish what the team believed and why, before that thinking disappears at launch
There’s a tension between journalism and product innovation, and it comes down to a clash of mental models. Journalism requires getting the facts right the first time. On the other hand, product innovation learns more from what doesn’t work than from what does, and sometimes that means putting something half-baked in the wild to find out what audiences will actually respond to.
Luke Williams, a former creative director at frog design who taught my innovation and design class at NYU Stern this spring, has a line for this: be wrong at the start to be right at the end. It resonated because I’ve been in news organizations where the journalistic mental model is applied, often for good reasons. But when the product is held to the journalistic bar, nothing half-baked gets launched to teach us anything.
Code is now accessible to anyone with a laptop and an internet connection. The old bottleneck, engineering resources, is gone. That should mean launching more ideas, iterating faster, and learning about our audiences and our mistakes along the way.
At the Minnesota Star Tribune, where I launched our first two audience-facing AI products, the most useful lessons came from how readers responded, including when the products missed the targets we’d set for them.
So how do you hold the line on credibility while building products you’re willing to let fail? A few suggestions:
Build a public sandbox
Separate the experiment from the main product. A subdomain with a clear label (beta, labs, sandbox) tells readers what they’ve found and resets the expectations they’d bring to the homepage. The readers who show up are opting in, and they still give you real behavior data. From there, the experiments that work can graduate to the main site.
At the Star Tribune, our first audience-facing AI product came out of a four-week experimental sprint. The result was Culinary Compass, a restaurant guide based on a reader’s in-the-moment vibe. We launched it on the primary domain, and readers who found it there expected a Star Tribune restaurant guide, not a rapid experiment. It told us a lot about what our audience expected out of us, and what they expected our use of AI to deliver for them. A sandbox would have taught us the same thing but for less.
Give it an end date before it starts
Decide when the experiment ends before it launches. Better yet, experiment with something that has a natural beginning and end. This helps remove the call of pulling something, so if the end date was always the plan, no one has to make the call to shut it down.
I launched Fair Bot in 2025 for the Minnesota State Fair knowing it would only be useful for the 12 days the fair ran. It let readers use semantic search to explore more than 300 food vendors by craving instead of by name, for example “something fresh” in a sea of fried food, or “sweet but not too salty.”
The tool logged more than 10,000 searches, but a surprising number of them asked where the bathrooms were. We had built a food finder, but our audience wanted a better fair guide. When the fair closed for the year, so did the bot, and the insights we gleaned went into planning for the next one.
Show your work
I miss the blogs newsroom product teams used to keep about how they built something. A lot of thinking goes into a product decision, and most of it disappears once it ships. Writing it down forces the team to say what they believed and why, and it gives the rest of the industry something to build on.
I worked at Vox Media from 2015 to 2022 in roles that straddled the newsroom and product teams, and I had the chance to contribute to a few posts for the Vox Product blog. I was reading the blog before I worked there, and it shaped how I thought about the job long before I was in the seat.
Product blogs are also where a version of a post-mortem can live. We already have a practice for when we need to issue a correction, and a flavor of this practice should extend to our experimental products.
We work in an industry that expects transparency from the people and institutions we cover. Holding the products we build to some version of that standard is a reasonable place to start, and it’s the part of the journalistic mental model that experimental products should look to keep.
Let’s compare notes
I’m the AI Strategy and Product Advisor at Hacks/Hackers, and I want to take the lessons above to independent newsrooms, the ones without a product team to spare. I work with news organizations as they think through where AI belongs in their work, from adoption and training to tooling and audience-facing products. Whatever the project, the work starts the same way: try it, watch what happens, and write down what you learned.
