Skip to main content

6 posts tagged with "owasp"

View All Tags

OWASP Red Teaming: A Practical Guide to Getting Started

Vanessa Sauter
Principal Solutions Architect

While generative AI creates new opportunities for companies, it also introduces novel security risks that differ significantly from traditional cybersecurity concerns. This requires security leaders to rethink their approach to protecting AI systems.

Fortunately, OWASP (Open Web Application Security Project) provides guidance. Known for its influential OWASP Top 10 guides, this non-profit has published cybersecurity standards for over two decades, covering everything from web applications to cloud security.

Sensitive Information Disclosure in LLMs: Privacy and Compliance in Generative AI

Vanessa Sauter
Principal Solutions Architect

Imagine deploying an LLM application only to discover it's inadvertently revealing your company's internal documents, customer data, and API keys through seemingly innocent conversations. This nightmare scenario isn't hypothetical—it's a critical vulnerability that security teams must address as LLMs become deeply integrated into enterprise systems.

Unlike traditional data protection measures, sensitive information disclosure occurs when LLM applications memorize and reconstruct sensitive data through techniques that traditional security frameworks weren't designed to handle.

This article serves as a guide to preventing sensitive information disclosure, focusing on the OWASP LLM Top 10, which provides a specialized framework for addressing these specific vulnerabilities.

Beyond DoS: How Unbounded Consumption is Reshaping LLM Security

Vanessa Sauter
Principal Solutions Architect

The recent release of the 2025 OWASP Top 10 for LLMs brought a number of changes in the top risks for LLM applications. One of the changes from the 2023 version was the removal of LLM04: Model Denial of Service (DoS), which was replaced in the 2025 version with LLM10: Unbounded Consumption.

So what is the difference between Model Denial of Service (DoS) and Unbounded Consumption? And how do you mitigate risks? We'll break it down in this article.

Prompt Injection: A Comprehensive Guide

Ian Webster
Engineer & OWASP Gen AI Red Teaming Contributor

In August 2024, security researcher Johann Rehberger uncovered a critical vulnerability in Microsoft 365 Copilot: through a sophisticated prompt injection attack, he demonstrated how sensitive company data could be secretly exfiltrated.

This wasn't an isolated incident. From ChatGPT leaking information through hidden image links to Slack AI potentially exposing sensitive conversations, prompt injection attacks have emerged as a critical weak point in LLMs.

And although prompt injection has been a known issue for years, foundation labs still haven't quite been able to stamp it out, although mitigations are constantly being developed.

Understanding Excessive Agency in LLMs

Ian Webster
Engineer & OWASP Gen AI Red Teaming Contributor

Excessive agency in LLMs is a broad security risk where AI systems can do more than they should. This happens when they're given too much access or power. There are three main types:

  1. Too many features: LLMs can use tools they don't need
  2. Too much access: AI gets unnecessary permissions to backend systems
  3. Too much freedom: LLMs make decisions without human checks

This is different from insecure output handling. It's about what the LLM can do, not just what it says.

Example: A customer service chatbot that can read customer info is fine. But if it can also change or delete records, that's excessive agency.

The OWASP Top 10 for LLM Apps lists this as a major concern. To fix it, developers need to carefully limit what their AI can do.