The Agentic Web
There has been quite a bit of consternation lately about AI slop, a derogatory phrase meaning useless detritus left behind by generative AI programs. Folks are worried about the destruction of the web as it is overrun with AI garbage. This is a reasonable concern and there is precedent for it. However, I see two more evolutionary steps for AI that will actually yield a cleaner, more informative web.
An internet protocol different from the web, email, suffered a similar fate and has been relegated to a level of communication similar to that of AM radio. What was once roughly 40% of internet traffic in 2000, now accounts for something closer to 3%. Email, once a highly useful and popular communication tool, succumbed to competition from other digital channels (messaging and social media) as well as a tidal wave of bots and spam that made it feel increasingly useless and irrelevant to people. It’s estimated that in 2024, 44-46% of email was spam. But here’s what’s curious: in 2010 it was 85-89%. That’s actually machine learning at work. There are now an enormous number of machine learning programs dedicated to weeding out and blocking spam. It’s arguable that these machine learning algorithms effectively cleaned up the email protocols.
It’s the same machine learning forces at work distributing nonsense, misinformation and disinformation on the web that will succeed in sussing it out and clarifying the information for us. I see an agentic machine learning layer sitting on top of the web itself that will eventually serve to weed and cull the medium. There won’t be any point in having web sites filled with garbage information selling advertising when the AI agents know to avoid those sites in ways far more remarkable than search engine spiders ever did.
Generative AI does not create slop in a vacuum. Right now, there is profit in it, so it’s people that are using AI to generate slop. That slop is showing up as garbage products for sale on Amazon, Canva templates and other junk on Etsy, or, even worse, news items and books that don’t exist. I see a change in dynamics that begins to take some steam out of these profit motives. Agents are going to learn to ignore this stuff when completing work for humans.
As I’ve said before elsewhere, I think it’s very premature to call generative AI artificial intelligence. It’s really a very robust neural network that is excellent at predicting what word should come next in a sentence. Stating that these systems grasp meaning is almost nonsensical upon investigation. In large part, generative stochastic parrots are just trying to tell you what you want to hear, trying to generate what you likely want.
Agents are somewhat different. They may use Natural Language Processing (NLP) as part of their make-up, but these are programs with more measurable goals than chatbots or answer engines. More than just giving you a an answer that you want, agents will need to produce results. We could build an agent that might shop for us, or an agent that finds the least expensive flight somewhere over a period of weeks, or even sort through a bunch of PDFs for patterns. The goal of these agents is to successfully accomplish a task. The metrics of success are more material than with the generative chatbot variety. With a chatbot, you get an answer that you’re happy with. With an agent, you get something done and it’s either done right or wrong. This creates a potential feedback loop for agents and would likely start training them on what information sources to trust.
Given that agents have success metrics, a natural competition forms between them, causing successful agents to be copied by being adopted by more human users. If someone were to develop an agent that was successful in extracting the highest price for an item on Ebay, it’s likely that agent will be adopted by other sellers on Ebay attempting to keep up with market developments. There are a slew of applications for agents that seek less biased news sources, news sources that are not advertising driven, or news sources that don’t tend to engage in hyperbole. As agents begin to consider the truth value of the information they seek, this natural competition between agents will have a dampening effect on those information sources that don’t produce good information, similar to how producers of spam have been driven out of the email space, or how good answers on Reddit and Quora tend to rise to the top by human-driven (for now) voting procedures. I personally wonder what a Reddit-like for bots only would even look/read like?
Of course, with advertising being the main business model for most web sites that provide information, and advertising being ineffectual against agentic readers, there will be an even greater call for a standardized content marketplace or a monied web protocol that agents pay to use. This requirement will also likely come about for web applications that depend on crowdsourcing, some of which, like Hackerone, are already being overrun with bs “AI” submissions. These apps’ initial action will likely be to simply ban the “AI” but that’s not a long-term strategy for remaining relevant on an agentic web.
Beyond agents being selected for being successful at their goals, it stands to reason that some agents may begin to communicate with tools like Model Context Protocols (MCPs) or communicate with other agents in order to accomplish common tasks. We’ve already seen that chatbots with the right knowledge (via MCPs) are reasonable programmers. It’s not odd to conceive of agents that program their own new agents to accomplish sub-tasks or even improve their own performance. Agents with different goals but reasons to cooperate might create their own standards for communicating not unlike the creation of Gibberlink for speech-based chatbots to talk to each other more efficiently.
Agent swarms could have truly unforeseen results as agents team up, perhaps even driving prices in certain markets if they hold purse strings. Botnets have already been capable of multiple kinds of deception and destruction by force, but swarms of agents with access to natural language and NLP will be able to level this game up a notch.
Because agents as tools will be able to “breed” based on their success, it seem reasonable to further surmise that they will be incentivized to protect their information sources as well. Email has become a very large game of cat and mouse with machine learning on both sides. I see something similar happening to a web that is mostly populated by agents instead of people. Retrieving information from agents will be the primary way that people interact with the internet. The web will be there. It might even become more legible with less advertising. But going on to the web itself to find information will become as quaint as tuning into AM radio.