Press "Enter" to skip to content

That’s not even a new one, part II

This should have been a great day at work since I had all my sleep and was fully rested and Mum was handling my grandmother pretty well.

It was not a great day at work.

We all spent half the day fighting fires from the LLM bot attacks. Again. And trying to implement more mitigations and band-aids to weather the storm. Again.

Someone shared an article about “the inevitability of LLM crawlers” in a shared Library space, and I’m going to print my response here verbatim. I think it’s important to say this to as wide an audience as will listen to me.

The inevitability tone of the end of this article makes me roll my eyes so hard it could have given me eye strain.  I don’t believe that this is either how any of this should behave, nor do I think it will be how any of this will behave in the future.  Massive parallel queries like that, “millions” to answer a single user’s question (hyperbole more than likely, but still), are a huge waste of bandwidth and processing time.  Right now, most LLM companies are still highly subsidised, but that won’t be the case forever.  Any org that can make it to the other side [of the LLM bubble], whatever that may be, is going to regret wasting precious resources building the sort of infrastructure described in the article, IMO.

I explicitly reject the concept that regular botnet attacks on infrastructure from AI/LLM training or “agentic search” or whatever is something that will be with us very long. It is simply not sustainable and evolutionary pressures will end it. The market will force the subsidies to end eventually, and when it does, so do the millions of useless requests per day.

This is not the new normal to which we should all be adjusting. This is the war to endure before calm finally finds us, weary and shipwrecked but somehow alive on the shore, clinging to the sand that no longer thinks…

2 Comments

  1. Артем Грязнов
    Артем Грязнов 15 November 2026

    Some people say that glimpses of that calm are already starting to show. That’s because in areas where AI is used to create or maintain something, it’s beginning to cost more money than if humans were doing the job.

    • Spots
      Spots 15 November 2026

      Yes, agreed. I think that companies are realising it doesn’t make sense to pay for an AI to do what a human can do better. But also, the AI companies are realising their stuff could be less expensive if it worked more efficiently. I have used an LLM a few times to search for specific information that was hard to find with keyword searches, and it seemed to be much better at being precise, and following things like robots.txt. There is hope…

Leave a Reply

Your email address will not be published. Required fields are marked *