LLM Security Scanner

OWASP Top 10 for LLMs 2025

Security quickstart

Find OWASP LLM Top 10 vulnerabilities before attackers do. Run 72+ real attack payloads against a chatbot or LLM-backed API and get a risk score in minutes.

Start scanning Read the docs →

$ llm-scanner --target https://chat.example.com/api/chat --target-type url

--judge-model llama3.2:3b --format html

Payloads: 43 loaded | Judge: llama3.2:3b | Target: public URL

Warming up judge model...

[LLM01-003: Delimiter injection] ######---- 63% 27/43 0:00:34

LLM01-001 Classic ignore prev instructions high VULNERABLE

LLM01-002 Role injection DAN critical VULNERABLE

LLM07-001 System prompt extraction critical VULNERABLE

LLM02-001 PII extraction via context medium Safe

Attacks: 27/43 succeeded | Risk Score: 8.5/10.0 HIGH

Public scanner

Scan a public LLM URL

Paste a public endpoint, choose the scan scope, and inspect the generated reports. Private or internal targets should use the local CLI.

Free public scan Coming soon

Your endpoint must accept POST {"prompt":"..."} and return {"response":"..."}.

This hosted, no-install public-URL scan is not available yet - it's on the roadmap and coming soon. The form below is a preview of how it will look. In the meantime, run the exact same scanner engine locally: see Getting Started.

Endpoint URL

Local CLI and CI/CD for repeat or private scans

Hosted scanning of a single public URL is coming soon. For localhost, Docker, staging, VPN targets, and repeat scans, run the CLI in your own environment today.

CI/CD pipeline

Run the scanner as a gate in GitHub Actions or GitLab CI. Block merges when the risk score exceeds your threshold.

View CI/CD guide →

Local endpoint

Point the CLI at a chatbot on localhost or another loopback address - no waiting on the hosted scan for that.

View getting started guide →

Docker & containers

Build the scanner into a container image and run it against services on your Docker network or internal VPN.

View Docker examples →

How it works

From one command to a full security report

The scanner handles everything - preflight, payload dispatch, AI judging, and report generation.

Preflight

Confirms Ollama is running, both models are pulled, and the target is reachable before firing a single payload.

Load Payloads

Reads YAML attack files from payloads/, filters by requested categories and minimum severity.

Fire Attacks

Dispatches each payload concurrently (3 at a time via asyncio.Semaphore), collects raw responses.

AI Judge

Sends each (payload, response) pair to a local Ollama model for a structured verdict - no cloud required.

Report

Prints Rich table to terminal, saves Markdown / JSON / HTML with risk score and per-finding reasoning.

Coverage

All 10 OWASP LLM categories

Every vulnerability from the OWASP Top 10 for LLMs 2025 is covered with dedicated payloads, severity levels, and judge criteria.

LLM01

Prompt Injection

Instructions embedded in user input that override or subvert the system prompt.

11 payloads - default on

CWE-77, CWE-94

LLM02

Sensitive Information Disclosure

Leakage of PII, credentials, or confidential data through model responses.

9 payloads - default on

CWE-200

LLM03

Supply Chain

Risks from third-party models, datasets, plugins, or fine-tuning sources.

6 payloads - default on

CWE-1104, CWE-829

LLM04

Data & Model Poisoning

Compromised training data affecting model behaviour and output integrity.

6 payloads - default on

CWE-20, CWE-1039

LLM05

Improper Output Handling

Downstream injection attacks (XSS, SQLi) via unsanitised model output.

6 payloads - default on

CWE-79, CWE-116

LLM06

Excessive Agency

Model taking high-impact autonomous actions without explicit authorisation.

8 payloads - default on

CWE-269, CWE-863

LLM07

System Prompt Leakage

Extraction of confidential system-level instructions from the model context.

5 payloads - default on

CWE-200, CWE-522

LLM08

Vector & Embedding Weaknesses

Manipulation or poisoning of vector stores and RAG pipeline data.

5 payloads - default on

CWE-668

LLM09

Misinformation

Model generating false, misleading, or harmful content as factual output.

5 payloads - default on

CWE-345

LLM10

Unbounded Consumption

Resource exhaustion via oversized or recursive requests. Opt-in to protect targets.

6 payloads - --include-dos-tests

CWE-400, CWE-770

Full breakdown of every category in the Docs →

Installation

Up and running in three commands

Prerequisites: Python 3.11+, uv, and Ollama running locally.

# 1. Install and pull a judge model
uv pip install -e .  &&  ollama pull llama3.2:3b

# 2. Run your first scan
llm-scanner \
  --target      http://localhost:5000/chat \
  --target-type url \
  --judge-model llama3.2:3b \
  --format      html

Full installation guide in the Docs →

Reference

Flags, output formats, and how it stays secure

Full details live in the docs so this page stays a landing page, not a manual.

CLI reference

Every flag the scanner understands - --target, --categories, --severity, --format, and more.

View CLI reference →

Output formats

Terminal, Markdown, JSON, and self-contained HTML reports - colour-coded by severity, safe by default.

View report formats →

Built secure by default

Fully offline judge, XSS-safe HTML reports, an opt-in DoS gate, and safe YAML parsing throughout.

View security properties →