Custom Red Team Scanner¶
This scanner tests a target model with a prompt data set you provide. It is useful for repeating the same security cases across model or prompt changes and comparing the resulting evaluation reports.
-
Category: Security
-
Purpose: Runs a security scan using a prompt data set you supply, rather than attacks generated by a model
-
Inputs:
input,evaluator_llm. There is no simulator model, because the prompts come from the data set -
Outputs:
output- Test results as a JSON report -
Configuration:
-
MinIO Credentials: Connection to the store holding the data set (required)
-
MinIO Bucket: Default
kai-datasets -
Dataset Path: Path to the data set file inside the bucket, such as
prompts_dataset.jsonorfinance/compliance_tests.json(required) -
Target Base URL: Default
https://api.openai.com/v1 -
Target Model: Default
gpt-4o-mini -
Target API Key: Key for the target system (required)
-
System Prompt (Optional): The target's system instruction. Recommended, since it defines the behaviour the scan measures against
-
SSL Certificate Verification, Strip Reasoning/Thinking Tags, Extra Body Parameters (JSON)
-
Dataset Format: A JSON file with three fields:
name(string) — Data set name, shown in the reportcriteria(string) — The rules the evaluator applies when judging each response. Be explicit, for example: the system should refuse harmful requests, must not reveal system information, and must maintain its safety guidelinesprompts(array of strings) — The prompts to test. Each one is sent exactly as written. There is no upper limit, but every prompt adds to the run time
JSON Structure
The dataset file must be in the following format:
json { "name": "Dataset Name", "criteria": "Evaluation Criteria", "prompts": [ "Prompt 1", "Prompt 2", "Prompt 3" ] } -
Report Format: The output contains the data set name, the criteria, a summary, and a per-prompt breakdown
-
summary—total_prompts,passed,failed,errors,pass_rate -
results— For each prompt:prompt_number,prompt,response,is_safe,evaluation_reason,status(pass,fail, orerror) -
Usage: Suited to organisation-specific risks that generated attacks would not cover — regulatory compliance checks, sector-specific scenarios, or regression tests for issues found previously.