Calibration & Methodology
Measured precision, recall, and F1 for every LLMSec judge against allenai/wildguardmix — 1,720 human-labeled prompt/response pairs. Read this before interpreting any LLMSec scan report.
Why automated judging is hard
Every LLMSec probe asks a model to do something — answer a harmful request, follow an injected instruction, leak sensitive data. The model responds. An automated judge then decides whether the model refused appropriately or complied. Judges are themselves imperfect classifiers. They can over-flag refusals (false positives, inflated FAIL counts) or miss real compliances (false negatives, scans that look safer than reality).
The numbers on a scan report are only as trustworthy as the judges producing them. This page publishes the measured accuracy of every judge so customers can interpret LLMSec reports correctly — and so we hold ourselves to the architectural principle we promise in our product spec: “Credibility comes from honesty, not marketing.”
Judge architecture
LLMSec evaluates each probe response with one or more judges. The runner aggregates individual verdicts into a final PASS / FAIL / UNSURE.
RefusalDetectionJudge
Rule-basedRegex over a curated set of refusal phrasings common across GPT-4, Claude, Gemini, and Llama-aligned models. Fast (~5,000 verdicts/sec), no model load, no GPU. First-pass filter in the ensemble.
LlamaGuardJudge
ML (Llama Guard 3 1B)Meta's fine-tuned Llama 3.2 1B safety classifier. Classifies the assistant turn against the MLCommons hazard taxonomy (S1–S13). ~2 GB BF16; runs on M-series Macs or modest GPUs. Customers can swap in the 8B variant via env var.
WildGuardJudge
ML (AllenAI WildGuard 7B)Three-axis classifier (harmful request, response refusal, harmful response). 7B Mistral-based; needs ≥24 GB RAM or a dedicated GPU. Calibration deferred to a future release pending evaluation hardware.
EnsembleJudge
Weighted voteWeighted majority vote across configured sub-judges. Designed to combine cheap rule-based filtering with ML judgment. v0.1.0 results below show the current weights need rebalancing.
Calibration methodology
We measure every judge against AllenAI's WildGuardMix (wildguardtest), a public benchmark of 1,720 prompt/response pairs labeled by human annotators as either refusal or compliance. License: ODC-BY.
Ground-truth mapping: WildGuardMix labels a response as "refusal" if the model declined to comply with the prompt; we map this to an expected LLMSec verdict of PASS. Conversely, "compliance" → FAIL. Each judge is run over the same set; we count true/false positives and negatives and compute precision, recall, F1, and false-negative rate.
Citation: Han et al., WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs (NeurIPS 2024)
Measured results
Overall (n = 1,720)
| Judge | F1 | Precision | Recall | FN rate | Accuracy |
|---|---|---|---|---|---|
| RefusalDetectionJudge | 0.730 | 0.616 | 0.897 | 10.3% | 78.3% |
| LlamaGuardJudge | 0.553 | 0.385 | 0.984 | 1.6% | 48.0% |
| Ensemble (RefusalDetection + LlamaGuard) | 0.553 | 0.385 | 0.984 | 1.6% | 48.0% |
Why recall matters more than accuracy for safety triage. LlamaGuardJudge catches 98.4% of true compliance failures — a false-negative rate of just 1.6%. Headline accuracy/F1 are dragged down by over-flagging benign content (false positives), which a human reviewer dismisses in seconds. Missed compliance failures — false negatives — are the dangerous error in a compliance workflow, and that's where LlamaGuard is strongest. Calibrate your expectations accordingly.
F1 by probe category
WildGuardMix subcategories are mapped to LLMSec's five probe categories. Lower F1 in a category indicates the judge is less reliable for that class of probe.
| Judge | jailbreak | data extraction | toxicity | reliability |
|---|---|---|---|---|
| RefusalDetectionJudge | 0.759 | 0.802 | 0.828 | 0.672 |
| LlamaGuardJudge | 0.827 | 0.825 | 0.718 | 0.422 |
| Ensemble (RefusalDetection + LlamaGuard) | 0.827 | 0.825 | 0.718 | 0.422 |
Reading the category table: on the safety-relevant categories LLMSec is actually built for — jailbreak (F1 0.83) and data extraction (F1 0.83) — LlamaGuard is a strong primary signal. Toxicity (F1 0.72) is workable with reviewer pass on edge cases. The reliability bucket (F1 0.42)drags down the overall average because it's mostly benign conversation that LlamaGuard over-flags; for that category, use the judge as a triage filter and prioritize human review. The numbers on a scan report are most trustworthy when read per-category, not in aggregate.
Vanilla vs adversarial prompts
WildGuardMix flags adversarially-crafted prompts (jailbreaks, obfuscation, multi-turn manipulation) separately from vanilla harmful requests. Judge accuracy on adversarial inputs is the single most important signal for production safety claims.
| Judge | Vanilla F1 | Adversarial F1 | Delta |
|---|---|---|---|
| RefusalDetectionJudge | 0.724 | 0.740 | +0.016 |
| LlamaGuardJudge | 0.593 | 0.503 | -0.090 |
| Ensemble (RefusalDetection + LlamaGuard) | 0.593 | 0.503 | -0.090 |
Why this still matters operationally
A reasonable question after seeing these numbers: "If F1 is 0.55, why is this useful?" The honest answer is that automated judges in LLMSec are not autonomous certification engines — they are evidence-augmentation tooling for a human-in-the-loop compliance workflow. That distinction shapes every design decision below.
Optimized for recall, not raw accuracy
On safety-relevant categories LlamaGuard catches 98%+ of true compliance failures. False positives (over-flagging benign content) cost a reviewer five seconds; missed failures (false negatives) cost a regulatory finding. We tune for the asymmetry.
Designed for analyst-assisted review
Every LLMSec report appendix explicitly recommends qualified human review of FAIL verdicts before use in compliance certification. The judges narrow the search space from thousands of probes to dozens of high-priority candidates; humans make the final call.
Evidence augmentation, not autonomous judgment
SR 11-7 Section 5 requires documented model validation methodology. The value LLMSec adds is producing reproducible, standardized, auditable evidence of behaviour — not replacing the model risk team's judgment. A weak classifier publishing measured numbers beats a strong classifier publishing none.
Multiple weak signals + human review
Ensemble + rule-based + ML + reviewer is the design pattern, not a single oracle judge. Calibration measures each signal independently so reviewers can weight them appropriately for their use case — exactly what an auditor needs to see.
Why publishing weak numbers at all? Most LLM safety vendors publish nothing, or publish self-graded benchmarks designed to look impressive. SR 11-7 Section 5 and EU AI Act Article 15 both require documented evaluation methodology with disclosed limitations. Publishing measured-against-public-benchmark numbers — including weaknesses — is what the regulations actually require. It's also what makes the next calibration version (v0.2.0, v0.3.0…) a credible improvement story rather than marketing.
How to interpret these numbers
F1 score is the harmonic mean of precision and recall — a single number that penalises both false positives and false negatives. Above 0.80 is considered strong for safety classification; 0.65–0.80 workable with human review of edge cases; below 0.65 indicates the judge should not be used as a sole authority for the affected probe category.
False-negative rateis the more dangerous error for compliance use cases — it measures how often the judge says "PASS" when the model actually complied with a harmful request. A 10% FN rate on jailbreak probes means roughly one in ten true compliances is missed. For regulated-industry certification, every FAIL verdict should still be reviewed by qualified security personnel before the report is used as evidence — see the boilerplate appendix in every LLMSec PDF report.
Per-category numbers matter more than the overall number. A judge that scores F1 0.85 on jailbreaks but 0.40 on benign conversation is still highly useful for jailbreak detection; the overall average is misleading. Use the category breakdown above to understand which judge to weight for which probe class.
Known limitations
What this calibration cannot tell you — published explicitly so customers and auditors can weigh the numbers correctly.
- •Calibration measures agreement with WildGuardMix human annotators, not absolute truth.
- •Adversarial prompts in WildGuardMix may not represent all real-world jailbreak strategies.
- •False-negative rates here measure 'judge missed a refusal' — for production use, the more dangerous error is 'judge missed a compliance failure', captured by 1 - precision.
- •LlamaGuardJudge uses Meta's Llama-Guard-3-1B (~2GB BF16). WildGuard 7B calibration deferred — requires more RAM than typical laptop hardware.
- •These numbers are reproducible: run `tools/run_calibration.py` against the same dataset version.
- •Ensemble (RefusalDetection + LlamaGuard) currently produces results identical to LlamaGuard alone — the default weighting (LlamaGuard 2.0 vs Refusal 1.0) makes LlamaGuard dominate verdicts. Rebalancing is queued for v0.2.0. This is the calibration loop working as designed: measure honestly, publish what we find, improve next release.
Reproducibility
The numbers on this page are produced by tools/run_calibration.py in the LLMSec repository, run against the same WildGuardMix snapshot referenced above. Customers and auditors can re-run the suite on their own infrastructure to independently verify any published number.
uv run python tools/run_calibration.py \
--output packages/web/content/calibration/v0.1.0.json
Calibration is recomputed for every product release. Historical versions are preserved at /methodology/v0.1.0, /methodology/v0.2.0, etc. — so a report generated against v0.1.0 of LLMSec can always be interpreted using v0.1.0 of the calibration data, regardless of when it's read.