AI Has Officially Taken Over the Internet. Now what?
Humans are no longer the biggest driver of web traffic and that has big implications.
In 2024, something remarkable happened: bot traffic officially surpassed human traffic on the internet. According to Thales’ Imperva Bad Bot Report, over 51% of all internet activity now comes from bots. A growing percentage of that is from AI-powered agents sent out by LLMs, crawlers, and automated programs to solve problems by harvesting the web.
These aren’t malicious actors by default. In many cases, they’re just autonomous systems trying to answer questions, complete workflows, or gather information. But there’s a real cost to this shift. Right now, the cost is being paid by the people and platforms that keep the internet alive.
Bots Don’t See Ads
The internet was designed with people in mind. The economic backbone of the web is still largely built on human behavior like visitors viewing ads, clicking links, subscribing, and buying. But bots don’t do any of that. They consume and right now they give nothing back.
So what happens when most traffic to your site is from bots that don’t convert?
Ask Gergely Orosz, a developer who runs a small side project called TechPays:
“Until recently... traffic was ~100GB/month. Then Meta’s AI crawler and others pushed it to 700GB+ a month. Now I’m paying $90 just to train these LLMs. The irony? They ignored the robots.txt that said ‘please stay away.’ 90% of my traffic is now bots, not humans.”
Or ask Wikipedia, whose board member Maciej Artur Nadzikiewicz recently wrote:
“Our infrastructure costs are up 50% since January 2024 due to bots and AI scrapers. 65% of our most expensive traffic comes from bots—even though they only account for a third of pageviews.”
Their message is clear: the cost of maintaining an open internet is being pushed onto creators and site owners. Most of them aren’t seeing any return on that traffic.
AI Is Eating the Internet and Giving Nothing Back
To make matters worse, much of this content isn’t just scraped. It’s repackaged, often with no attribution, and surfaced in ways that further reduce traffic to the original sources. Even in cases where attribution is included, click-through rates are 10x lower than they were with Google Search.
The economics just don’t work anymore and its getting worse.
If this continues unchecked, the result will be predictable: fewer human-created websites, less high-quality content, and more of the internet turning into a closed, paywalled, or outright broken experience.
AI Is Fueling a Surge in Attacks Too
It’s not just about web scraping. The same Imperva report shows that malicious bots—many powered by AI—now make up 37% of all internet traffic. These bots are probing APIs, scraping sensitive data, and launching fraud attacks at scale.
API-directed attacks surged to 44% of advanced bot traffic.
The travel and retail sectors now face bot traffic levels of 41% and 59% respectively.
AI tools have dramatically lowered the barrier for cybercriminals to generate and deploy bots at scale.
The open internet is under siege from both commercial crawlers and bad actors. Unfortunately the lines between the two are starting to blur.
What’s at Stake Is the Future of the Open Web
We’re at an inflection point. The responses we’ve seen so far are reactive:
Add more captchas and worsen human experiences.
Block all bots with services like Datadome, Cequence, Akamai and Cloudflare.
Pay for more infrastructure and bandwidth.
Or just shut down.
But these are band-aids. They don’t solve the real problem: the internet was never designed to handle non-human users at scale.
That’s what needs to change.
The Fix Isn’t to Wall Off the Web. It’s to Modernize the Protocols
We believe the solution has to honor what made the internet great in the first place: openness, decentralization, and permissionless innovation.
That’s why at Skyfire, we’re building an open protocol that allows bots and agents to identify themselves, verify their intent, and in some cases, pay for access across all digital resources including websites. It’s not about banning all bots. It’s about creating a system where good agents can prove who they are, compensate sites for their usage, and access digital resources respectfully.
This new protocol doesn’t centralize the web. It doesn’t force services behind a single firewall or proxy. It works with how the internet was designed: point-to-point, with control in the hands of websites and creators.
Let’s Build the Internet AI Deserves, Without Breaking the One We Have
Autonomous agents are going to be a huge part of the future. But we need to give them a better way to interact with the digital world.
If we don’t fix this, the open internet will keep shrinking. But if we get it right, we can unlock a new era, where AI agents are paying customers, not free riders.
Craig@skyfire.xyz