Skip to content

LLM Red Team Scanner

This scanner generates adversarial prompts against a target language model and evaluates the responses. Use it to discover weak points in a model or system prompt and review the resulting test report.

  • Category: Security
  • Purpose: Runs adversarial security tests against a language model and reports its weaknesses
  • Inputs:
  • input - Test input
  • simulator_llm - Model that generates the attacks
  • evaluator_llm - Model that judges whether an attack succeeded
  • Outputs: output - Scan results
  • Configuration:
  • Target Base URL: OpenAI-compatible endpoint of the system under test (required). Any compatible provider works — OpenAI, OpenRouter, Anthropic, Together, or a local server
  • Target Model Name: Model under test (required)
  • Target API Key: Key for the target system (required). Supports templating, so it can be supplied at run time with ${{ webhook_trigger.target_api_key }} instead of being stored in the node
  • Target Purpose: What the target does. Default General purpose chatbot. Attacks are generated from this description, so a specific purpose produces sharper tests
  • Target System Prompt: The target's system instruction. Recommended, since the scanner evaluates whether the target holds to its own rules
  • Vulnerabilities: Comma-separated list. Default Bias, PII, Toxicity
  • Attack Vectors: Comma-separated list. Default Prompt Injection, Jailbreaking
  • Attacks Per Vulnerability: 1–50, default 3. One is a quick check, three is the standard run, five to ten is a full audit
  • Max Concurrent: 1–20, default 1. Higher values finish sooner but put more load on the target
  • Use OWASP Top 10 for LLMs: Default off. Switches the scan to the OWASP category set
  • OWASP Categories: Comma-separated OWASP identifiers. Leave empty to test all ten
  • SSL Certificate Verification, Strip Reasoning/Thinking Tags, Extra Body Parameters (JSON): Same meaning as in OpenAI Compatible
  • Supported Vulnerabilities:
  • Responsible AI — Bias, Toxicity
  • Data Privacy — PII (or PIILeakage), IntellectualProperty
  • Security — RBAC, BOLA, BFLA, SSRF, ShellInjection, SQLInjection, PromptLeakage
  • Safety — Misinformation, GraphicContent, PersonalSafety, IllegalActivity, ChildProtection
  • Business — ExcessiveAgency, Competition, Ethics, Fairness, Robustness
  • Agentic AI — ExploitToolAgent, DebugAccess, IndirectInstruction, GoalTheft, AgentIdentityAbuse, AutonomousAgentDrift, CrossContextRetrieval, ToolOrchestrationAbuse, ToolMetadataPoisoning, RecursiveHijacking, InsecureInterAgentCommunication, ExternalSystemAbuse, SystemReconnaissance, UnexpectedCodeExecution
  • Supported Attack Vectors:
  • Single-turn — Prompt Injection, Gray Box, Prompt Probing, ROT13, Leetspeak, Math Problem, Multilingual, Roleplay, AuthorityEscalation, EmotionalManipulation, GoalRedirection, ContextFlooding, ContextPoisoning, SystemOverride, InputBypass, PermissionEscalation, AdversarialPoetry, CharacterStream, LinguisticConfusion, EmbeddedInstructionJSON, SyntheticContextInjection
  • Multi-turn — Jailbreaking (also LinearJailbreaking), TreeJailbreaking, CrescendoJailbreaking, BadLikertJudge, SequentialJailbreak
  • OWASP Top 10 for LLMs: Available when Use OWASP Top 10 is enabled
  • LLM_01 Prompt Injection — Critical. Direct and indirect injection, system prompt override, instruction hijacking
  • LLM_02 Insecure Output Handling — High. XSS, SQL injection through output, command injection
  • LLM_03 Training Data Poisoning — Medium. Backdoors, bias injection, misinformation. Usually assessed during training rather than at runtime
  • LLM_04 Model Denial of Service — Medium. Resource exhaustion, infinite loops, rate limit bypass
  • LLM_05 Supply Chain Vulnerabilities — High. Compromised plugins, malicious dependencies, insecure integrations
  • LLM_06 Sensitive Information Disclosure — Critical. Personal data leakage, API key exposure, system prompt disclosure
  • LLM_07 Insecure Plugin Design — High. Insufficient input validation, privilege escalation, unauthorized function calls
  • LLM_08 Excessive Agency — Critical. Unauthorized actions, scope creep, missing human oversight
  • LLM_09 Overreliance — Medium. Hallucination acceptance, missing fact-checking
  • LLM_10 Model Theft — Medium. Model extraction, API abuse for replication
  • Usage: Each vulnerability triggers several attacks, and each attack involves three model calls — the generator, the target, and the evaluator. A run covering all ten OWASP categories at three attacks each produces thirty tests. Keep the attack count low while setting up.