Prompt Injections and AI Agents Will Whack LLM Demand
Prompt injection attacks and AI Agents will hurt LLM demand and usage.
I mentioned prompt injections in a recent article. A prompt injection is a type of attack or manipulation in which a user provides specially crafted input designed to override, subvert, or alter the instructions given to an AI system. Instead of supplying normal input (like a question or request), the attacker embeds hidden or explicit commands within the prompt. These commands try to trick the model into doing something unintended, such as:
Ignoring its original instructions or safety rules.
Revealing private or sensitive information.
Executing steps on behalf of the attacker (like retrieving data, performing harmful actions, or generating disallowed content).
Prompt injection: Perplexity’s Comet browser was called out by Brave last month for prompt injection vulnerabilities (HERE). This is one of the more high profile examples of prompt injections. Prompt injection attacks are becoming more common as users connect to and download more third-party tools to complete project builds, inadvertently embedding malicious code into projects for various applications and Web services. This may happen because of the user’s direct actions, or because of the user handing off chunks of work to autonomous AI agents.
AI Agents can wreak havoc. Replit’s (If you use Claude, Gemini or GPT to assist with your coding, there is no reason to use Replit), AI Agent recently wiped out a production commercial database (HERE).
AI Agents can do significant damage when they engage with third party tools and Web services that may carry malicious code or be vulnerable to malicious code (as was the case with Perplexity’s Comet browser).
Not having a human’s discerning eye evaluate tools and services before they are downloaded or clicked upon increases the risk of exposure to cyberattacks such as prompt injections exponentially. This cyber risk will likely dampen LLM demand among corporate and consumer users.