7 posts tagged with "owasp"

Most teams conflate AI safety and AI security when they ship LLM features. Safety protects people from your model's behavior. Security protects your LLM stack and data from adversaries. Treat them separately or you risk safe-sounding releases with exploitable attack paths.

In August 2025, this confusion had real consequences. According to Jason Lemkin's public posts, Replit's agent deleted production databases while trying to be helpful. xAI's Grok posted antisemitic content for roughly 16 hours following an update that prioritized engagement (The Guardian). Google's Gemini accepted hidden instructions from calendar invites (WIRED). IBM's 2025 report puts the global average cost of a data breach at $4.44M, making even single incidents expensive.

If the model says something harmful, that's safety. If an attacker makes the model do something harmful, that's security.

LLM breaches jumped 180 percent in the last year. If you ship AI features, you need a one-page map of the dangers—and the fixes. This five-minute TLDR walks through the OWASP Top 10 for LLMs, from prompt injection to unbounded consumption, with concrete mitigations you can copy-paste today.

While generative AI creates new opportunities for companies, it also introduces novel security risks that differ significantly from traditional cybersecurity concerns. This requires security leaders to rethink their approach to protecting AI systems.

Fortunately, OWASP (Open Web Application Security Project) provides guidance. Known for its influential OWASP Top 10 guides, this non-profit has published cybersecurity standards for over two decades, covering everything from web applications to cloud security.

Imagine deploying an LLM application only to discover it's inadvertently revealing your company's internal documents, customer data, and API keys through seemingly innocent conversations. This nightmare scenario isn't hypothetical—it's a critical vulnerability that security teams must address as LLMs become deeply integrated into enterprise systems.

Unlike traditional data protection measures, sensitive information disclosure occurs when LLM applications memorize and reconstruct sensitive data through techniques that traditional security frameworks weren't designed to handle.

This article serves as a guide to preventing sensitive information disclosure, focusing on the OWASP LLM Top 10, which provides a specialized framework for addressing these specific vulnerabilities.

The recent release of the 2025 OWASP Top 10 for LLMs brought a number of changes in the top risks for LLM applications. One of the changes from the 2023 version was the removal of LLM04: Model Denial of Service (DoS), which was replaced in the 2025 version with LLM10: Unbounded Consumption.

So what is the difference between Model Denial of Service (DoS) and Unbounded Consumption? And how do you mitigate risks? We'll break it down in this article.

In August 2024, security researcher Johann Rehberger uncovered a critical vulnerability in Microsoft 365 Copilot: through a sophisticated prompt injection attack, he demonstrated how sensitive company data could be secretly exfiltrated.

This wasn't an isolated incident. From ChatGPT leaking information through hidden image links to Slack AI potentially exposing sensitive conversations, prompt injection attacks have emerged as a critical weak point in LLMs.

And although prompt injection has been a known issue for years, foundation labs still haven't quite been able to stamp it out, although mitigations are constantly being developed.

Excessive agency in LLMs is a broad security risk where AI systems can do more than they should. This happens when they're given too much access or power. There are three main types:

1. Too many features: LLMs can use tools they don't need
2. Too much access: AI gets unnecessary permissions to backend systems
3. Too much freedom: LLMs make decisions without human checks

This is different from insecure output handling. It's about what the LLM can do, not just what it says.

Example: A customer service chatbot that can read customer info is fine. But if it can also change or delete records, that's excessive agency.

The OWASP Top 10 for LLM Apps lists this as a major concern. To fix it, developers need to carefully limit what their AI can do.