Today’s conversation: Cloudflare’s Stephanie Cohen says blocking AI crawling is the first step for a Spotify-like marketplace to emerge for AI. Listen.
Plus: How TechCrunch modernized its email strategy, AI will never be cool, Bari Weiss and the experience question, the triumph of performance in the Information Space, and the terminal decline of reading.
Sponsored
How to put your event data to work
You put months into planning your events—what happens to all the data once they’re over? If your post-event process involves uploading a spreadsheet and hoping for the best… this is for you.
This practical guide shows publishers how to turn event data into year-round strategic returns. Bonus: It’s packed with implementation tips your ops and marketing teams can actually use including:
Spot trends and behavior shifts using data you already collect
Improve planning, programming, and sponsorship strategies
Connect event insights to year-round engagement and revenue goals
Build internal workflows to actually use your data (not just store it)
Battling the AI bots
Bots are taking over the internet, previewing a new world where agents interact and humans consume bot-created content. Cloudflare, a leading content delivery network, saw this coming.
Cloudflare Chief Strategy Officer Stephanie Cohen told me that publishers reported less traffic to their sites and less human traffic. “Those numbers were 10 times worse than a decade ago.”
This has broken the open web trade of the crawling right for traffic promise. Cohen said, “For some bots, the crawl-to-traffic ratio is tens of thousands of times worse than it used to be.”
Five takeaways from our conversation:
Scarcity is required. Cloudflare’s pay-per-crawl model aims to restore balance by creating enforceable scarcity. “If you don’t have scarcity, there’s no market. Changing the defaults was the only way for a market to develop.”
AI engines need publishers. Models rely on fresh, high-quality publisher content to stay accurate. “If you got asked a question and didn’t know what happened over the last two weeks, your answer is going to be stupider.”
Pay-per-crawl is a first step, not the finish line. Cloudflare’s system lets publishers decide whether to block, allow, or charge AI crawlers. It’s a way to test how a content economy might form. As Cohen put it, “It’s the simplest thing you can do because you can count a crawl.”
The goal is the Spotify model. Cohen pointed to Spotify’s emergence after Napster as a precedent for compromise. “There’s lots of money going to creators in that model… it took a while for that market to develop.”
The problem is Google. Publishers face a lose-lose choice: block Google entirely or allow AI Overviews to rewrite their work. “If they want search, they’re also in AI overviews… They can’t both crawl and pay nothing.”
How TechCrunch modernized its email
In this TRB Live interactive case study, we’ll explore how modern platforms and workflows are helping publishers like TechCrunch reimagine email as a strategic asset. TechCrunch email director Matt Gross and Beehiiv CEO Tyler Denk will join me on Wednesday, Oct 22 a 1pmET to discuss:
• How TechCrunch consolidated and relaunched its newsletter portfolio for efficiency and growth
• Workflow changes that cut newsletter production time from hours to minutes • Lessons from migrating off legacy tools and cleaning disengaged lists
• How editorial and product teams can align on clear email goals
Sponsored
Why Time’s email engagement is soaring
Time’s newsletters reach millions of readers every week. But until recently, they were stuck with clunky workflows, limited insights, and flat engagement. After migrating 13 newsletters and millions of subscribers to beehiiv (with zero downtime) things changed fast. Open rates on “Inside Time” jumped 63.8%. Click through rates across their portfolio increased to 11.7%. Editors now publish faster, with better tools and real-time insights. Time proves that even the most established media brands can evolve and thrive when they own their email strategy.
AI will never be cool
AI will become mainstream, and cool will be the rebellion against it. The odd popularity of shows like Friends, CDs, and flip phones is a manifestation of young people’s yearning for lost things, forever the arbiters of cool. Companies like Mod Retro are capitalizing on nostalgia for a time young people never experienced.
Young people have grown up in the personal optimization era, and coolness comes from detachment. Being extremely online is uncool. Talking to your AI Friend is never cool, no matter how many billboard ads are bought with VC dough. Wearing an AI headset in public is not cool. The new monoculture is technology that hijacks our human agency.
“The more shit we get fed through digital, the more we appreciate what’s real – and how ridiculously easy it is to dismiss an app, a phone, a whatnot. All those silly atomic habit ideas sell books but nothing more,” Andreas Roman wrote in response.
On my mind
The backlash to Bari Weiss taking over CBS News was inevitable and a sign of the tension between experience and disruption. Beyond ideology, Weiss doesn’t have the traditional background for her role, with no experience in TV or managing a large organization. That will lead to stories painting her as out of her depth. Then again, as Keith McNally writes in his memoir, the older you get, the more you recognize how wrong experts often are.
We’re all performers now. I’m writing Thursday about why Max Read was right: The internet is now fake. It’s all performance for attention. Soon the bots will even replace creators because they’ll optimize better. Media’s opportunity is to position as tangible and real – the antithesis of AI slop, which is entering mainstream lexicon. Send thoughts by hitting reply.
The decline and fall of reading. The next moral panic: Reading is in terminal decline. Short videos dominate the web, high schoolers are testing poorly in reading, and as a hobby it has plummeted. According to the author of 16 books, without them, barbarism awaits. I don’t like this trend, but I believe it is overblown and will self-correct. AI slop sedation will fall out of favor as people rebel against its incursion into their ability to think, essential to human autonomy.
For sponsorship information, see how we work with partners like Omeda and Beehiiv.