Seongtaek Lim, creator of CoHeadline and PhD candidate in Cornell University’s Department of Information Science, talks to Youth Radio’s production department interns about A/B testing.
I have a bit of a one-track mind when it comes to storytelling. I’m always thinking about the best way to make the story logically unfold and stay true to the message I want to communicate. But I never consider what it takes to get people to click through to read my work.
Major news outlets rely heavily on captivating headlines to attract readers. And some media companies go so far as to use A/B testing, which turns headline writing into a numbers game.
According to ProPublica, sites like the the New York Times and the New York Post use A/B testing to compare multiple headlines for the same article and determine which headline is the most popular. For instance, the Post published a video in which a Wisconsin police officer was nearly flattened by her own vehicle. Their headline “Cop’s fast feet save her from getting crushed by her own car” beat out “Crazy dashcam footage of cop almost getting crushed.”
When news outlets employ A/B testing, readers are the unknowing test subjects, voting on which headline they prefer. “After testing headlines for whatever amount of time that a news company deems fit, the bot evaluates them. This helps news outlets gain more readership,” said Seongtaek Lim, a PhD candidate in Cornell University’s Department of Information Science.
Lim visited Youth Radio and spoke to teen reporters and coders from the production department. He showed us CoHeadline, a browser extension in Google Chrome that he created to allow users to see if news outlets are conducting an A/B test. With CoHeadline, the user can see different variations of headlines, when the publication is A/B testing.
Lim created CoHeadline so users can know when they are part of an experiment. “News outlets want to know more about us–their readers,” Lim said. “And that’s very important. But we should know what’s going on behind the scenes.”
There are a few limitations of CoHeadline, which is currently not released to the public. It only picks up A/B testing when it is happening live. If there is no live testing, then CoHeadline becomes inactive. Although it allows the user to see the various headlines as they are tested, it doesn’t allow users to actively choose which headline they prefer. Still, it gives readers some insight into what’s happening behind the screen. CoHeadline users can see for themselves when they’re playing a role in A/B testing.
As a budding journalist, headlines are usually an afterthought for me. But CoHeadline has opened my eyes to the importance that media companies attribute to a headline. By investing in strategies like A/B testing, publications demonstrate how much influence they believe headlines have over a reader’s decision to click on a story. And in turn, now that I’m more aware of a headline’s significance, I’m beginning to put more thought into the art–and science–of headline writing.