Additional commentary by Emily Lyons
Welcome back to AI Pulse, our blog series in which we share data insights on the state of AI bots.
If you missed any of the previous blog posts in the AI Pulse series, check out OpenAI’s Wild Bot Behavior After GPT-5, AI Bot Mitigation Is Increasing Everywhere, What Circadian Rhythms Reveal About AI Bot Behavior, and How Big Tech Impacts AI Bot Traffic now.
This post’s insights are all about AI bots’ favorite targets — with a breakdown by industry, AI bot types, and more. We also discuss mitigation and how the most targeted industries are responding.
AI bots definitely play favorites
Since we began tracking AI bot traffic, clear patterns have emerged, recognizing that some industries attract far more activity than others. With our newly formalized AI bot categorization, we’ve been able to dig deeper to understand which types of AI bots target specific sectors, and why.
Figure 1 shows the distribution of AI bot traffic across Akamai’s network. More than 45% of this traffic targets commerce customers, followed by publishing and digital media, and then high-tech.
Love at first sight: Commerce and AI platforms
Why do AI bots target commerce the most? Because AI bots love structured, high-value data, and online retail is full of it. Product listings, prices, reviews, and availability data are clean, consistent, and constantly updated, making them ideal fuel for AI models.
Across all AI bot categories, commerce leads the way, though AI agents are muted at present, and the distribution across AI fetchers is far more distributed (Figure 2).
AI platforms and agentic commerce are built on this foundation. Agentic shopping assistants, comparison engines, and market intelligence tools increasingly rely on retail data for training and operation.
We’re also seeing new partnerships form across the ecosystem with merchants and payment providers to support verified, transactional AI experiences in which agents can safely browse, compare, and even purchase on behalf of users.
Travel and hospitality brands are building further on this collaboration, experimenting with Model Context Protocol (MCP) servers and technology alliances to make their content easier for AI platforms to understand and reference safely.
Publishers and content creators, not so much
Publishing and digital media rank second among AI bot targets, accounting for more than 14% of total AI bot traffic across the Akamai network. But that surge in interest isn’t exactly friendly.
AI platforms increasingly scrape original articles, videos, and creative works to train models, power AI-driven search, and generate summaries, all without attribution or compensation. This behavior dilutes ad and subscription revenue, weakens brand integrity, and blurs ownership of digital content.
Akamai stands with publishers and original content creators. We’re partnering with the world’s largest publishers and digital media brands to actively manage how AI bots interact with their content by enforcing licensing, monetization, and intellectual property protections so content fuels AI responsibly, not exploitatively.
High tech: Training the next generation
High tech ranks close behind publishers in AI bot activity. This segment, which includes software as a service (SaaS) platforms, cloud providers, and hyperscalers, represents the infrastructure that powers much of the web.
AI bots target these sites for their rich technical content, public APIs, and product data, which are all valuable for model training and competitive intelligence. In effect, the builders of automation have become prime sources for training the next generation of it.
Akamai’s AI bot categories
In a previous AI Pulse, we introduced Akamai’s new AI bot categories to help better define which AI bots do what tasks.
At a high level:
AI training crawlers automatically scan and collect large amounts of data from websites to be used as training data for large language models (LLMs)
AI search crawlers automatically scan and index websites for AI-powered search experiences
AI fetchers grab specific web pages in real time to fulfill specific user requests on AI assistants, such as answering a question or summarizing a web page via an AI chatbot
The AI bot breakdown
Since we’ve started tracking AI bot traffic, we now have more than 40 AI bots categorized as part of our bot directory. Interestingly enough, we are seeing a substantial amount of traffic for only 9 of those 40.
We’ve found that the most active AI bots are ChatGPT-User, Bytespider, Meta-ExternalAgent, GPTBot, ClaudeBot, and OAI-SearchBot. Three additional AI bots account for a minor amount of traffic: Google-CloudVertexBot, PerplexityBot, and TikTokSpider. Everything else accounts for a minimal amount of traffic.
AI bots by vertical
Figure 3 demonstrates this bot traffic, broken out by vertical. Again, we see that commerce is the key target across nearly all the five key players; it is equally targeted, regardless of the specific AI bot.
What we found particularly interesting, however, is the recent rise in GPT-User traffic in the public sector. We hypothesize that because GPT-User is a fetcher that is focused on pulling content for user-driven requests, this may be associated with today’s political climate, particularly in the United States, where the public is increasingly concerned about and interested in sourcing government information.
To block or not to block?
So, what is everyone doing about AI bot traffic? We mentioned in our second post in the AI Pulse series that we were seeing increased mitigation, or control, of AI bots across industries and geographies.
Akamai provides our customers with the ability to monitor or mitigate their AI bot traffic. Our customers primarily use these top three mitigation action choices:
Deny (a total block)
Tarpit (an aggressive action that slows down the entire connection, often involving keeping the connection open for a long time, serving no response or a very slow, continuous stream of data, which effectively traps the bot in an endless loop or significantly ties up its resources until it times out)
Delay (a more subtle action that intentionally introduces a specific, short latency [a delay of a few seconds] before serving the response, which can slow down bots during peak traffic times)
Since June, we’ve observed double the amount of AI bot traffic being mitigated, primarily driven by the public sector (Figure 4).
Our AI bot traffic tracking has provided us with some insights:
We previously noted that AI bot traffic mitigation across commerce was rising, but it has since stabilized. This reflects that the commerce industry has taken a liking to AI bots and will probably continue to allow them to access their content.
Earlier in this post, we mentioned that AI bot traffic across the public sector accounted for a large amount of total ChatGPT-User traffic. Interestingly enough, we also found that another specific AI bot had been targeting the public sector — specifically, a single U.S. government agency.
Mitigating this particular bot activity accounted for nearly half of our total AI bot mitigation actions every day from the end of September and into October, which we were not expecting.
The so what
AI bots are here, and they’re not going anywhere. Their impact will keep evolving as we move deeper into the agentic era, reshaping how industries interact with automation.
In the commerce industry, the rise of agentic commerce will only strengthen the connection between businesses and the AI-driven ecosystems built around them.
In publishing, Akamai will continue to stand with content creators, helping them to defend against evasive and unauthorized scraping, protect their hard-earned revenue, and ensure fair compensation for their work.
Akamai is partnering with monetization alliances, payment providers, industry peers, developers, and AI platforms to make sure every industry has the tools and support needed to manage AI bots responsibly and effectively.
Stay tuned
As always, stay tuned for the next post in this series, in which we’ll present another AI bot insight.
Learn more
To gain visibility and control today in managing AI bots, contact an expert.
Tags
