Rethinking Bots: From Threats to Trusted Customers in the AI Era
For the last thirty years, digital marketers have been focused on keeping bots out. The very term “bots” had a negative connotation, and the layers and layers of protection we’ve built are now a liability to brands’ web stacks.
This isn’t because security stopped mattering… It hasn’t. But all of the CAPTCHAs, rate limits, login walls, fingerprinting and internet fortresses we’ve tried to build must be re-examined in the modern context of AI. Brands may be doing themselves a disservice with the default position that automated traffic is hostile until proven otherwise.
My co-founder has been preaching about this for years, and I’ve nodded along for most of them. Only recently have I felt the upside-downness of it, and how much of what we built to protect brands now works against us.
Understanding the Scale of the Problem
Bots are now 53% of web traffic — that’s the stat everyone repeats on LinkedIn. However, a deeper look reveals agents are doing far more than before. They don’t just scan sites; they interact and execute workflows, blurring the lines between good and bad traffic and making it increasingly difficult to differentiate.
This challenge is critical: your defenses can’t easily distinguish between a bot scraping your catalog maliciously, an assistant trying to buy on behalf of a customer, or bad actors who remain threats.
Brands Are Starting to Adopt New Strategies
As agents increasingly act on behalf of consumers, websites occupy a transformed role in the customer journey. Imagine a customer telling an assistant, “Find me trail runners, men’s size 10.5, waterproof, under $150, at my door by Saturday.” The agent lands on your site, reads the page code, infers button functions, and makes selections.
If the agent encounters friction or access issues, it doesn’t email support or retry later. Instead, it moves on to a competitor’s site that is easier to navigate and reports back to your customer. No cart, no session, no trace of interaction on your analytics.
This shift is profound. Site friction no longer just reduces conversion rates—it silently erodes your consideration set before your analytics even register a visit.
Rather than forcing agents to reverse-engineer your site, brands need to implement new protocols and site structures that assist rather than defend against bots. Otherwise, agents will reroute consumers to marketplaces and resellers where data is accessible. Your product risks becoming a mere listing in a comparison table—lacking your brand story, bundles, or differentiation beyond price and ratings.
Bots must be recognized as a first-class audience when optimizing your website and digital experience. Here are key considerations for evolving your web presence for both humans and bots.
1. Find Out What You’re Blocking and Make It Easier for “Good Bots” to Consume Your Site
Your bot policy was likely set by a vendor default and may not have been reviewed since. There’s a real risk you’re blocking valuable demand under the guise of protection.
For example, many brands still block bots like ChatGPT, Claude, and Perplexity, which fetch pages when users ask questions. If these bots can’t reach your content, you can’t be cited when shoppers ask AI what to buy.
2. Make Sure That Your Site Structure Is Laid Out Well for Agents, Not Just Well Styled
When it comes to AI agents, structure beats style. Organized data—such as pricing, availability, FAQs, sizing, shipping windows, and return policies—is crucial for agents to understand and utilize your offerings effectively.
For instance, Ratings & Reviews hidden inside a visual carousel or image are invisible to agents and won’t influence their decisions.
3. Map Your Customer Journey and Make Sure the Actions Agents Need to Take Are Well Articulated
Identify the key tasks a customer expects an agent to perform on your site and optimize for those specifically.
A footwear brand, for example, should ensure agents can easily navigate to purchase products, read reviews, and compare them with competitors. Since these steps are integral to the shopper’s journey, brands must re-engineer their sites to make these actions seamless for both humans and agents.
4. Revamp Your Analytics Stack
Often, agents visit your site and relay information to consumers across other platforms rather than directly on your website. What truly matters is whether your brand appeared, was cited, and ultimately chosen during the AI search interaction with your customer.
This dynamic means traditional web analytics may no longer offer a complete picture. For example, a 35% drop in organic search traffic might not indicate less visibility but rather a shift in impressions from your website to interactions within large language models between agents and shoppers.
For many brands, this calls for new metrics and analytics to understand how bots and AI are reshaping organic discovery.
Conclusion
None of this means the last thirty years were a mistake. The gates were appropriate for the internet we had. They’re just pointed at the wrong threat now.
Evolving for an agentic internet is not a simple SEO tweak or checklist. It’s architectural—and marketing owns much more of this evolution than traditionally expected. Security teams won’t be the ones telling you in your Monday standup that your bot rules are costing you customers.
