There’s a misconception about bots – they’re all bad. And therefore, companies treat them as a security issue. Once they detect them, they block them and move on.

That strategy may have worked six years ago, but not today. Considering non-human traffic, such as legitimate AI crawlers indexing content for LLM training and agentic systems making decisions on behalf of customers, has surpassed human traffic, blocking everything can hurt more than help.