Skip to content

Commit 69659e3

Browse files
emmanuelgjrclaude
andcommitted
Integrate LAAF v2.0 — LPCI red-teaming framework + crosswalk (v1.5.7)
- Add evals/laaf/ suite: README, run_laaf.sh, laaf_crosswalk.py, 6 stage configs - Add data/tools-supplement.json (LAAF v2.0 mapped to 8 OWASP entries) - Add INC-021 to incidents.json: LAAF empirical study (67–100% breakthrough rates) - Update generate.js: incidents loading, tools-supplement merge, mergeTools fix - Update evals/ci/github-action.yml: laaf-eval job with per-stage thresholds - Regenerate all 41 data/entries/ + docs/data.js - Update CHANGELOG.md and README.md for v1.5.7 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 13bc8f1 commit 69659e3

26 files changed

Lines changed: 1612 additions & 6 deletions

CHANGELOG.md

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,48 @@ Versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
99

1010
## [Unreleased]
1111

12-
All planned items complete through v1.5.6. Next: community contributions, additional incident entries, and framework expansion.
12+
All planned items complete through v1.5.7. Next: community contributions, additional incident entries, and framework expansion.
13+
14+
---
15+
16+
## [1.5.7] — 2026-03-28
17+
18+
### Added
19+
20+
#### LAAF v2.0 integration — Logic-layer Automated Attack Framework
21+
22+
LAAF v2.0 (https://github.com/qorvexconsulting1/laaf-V2.0) is the first automated red-teaming framework purpose-built for Logic-layer Prompt Control Injection (LPCI) vulnerabilities in agentic LLM systems. This release integrates it fully into the crosswalk evaluation suite.
23+
24+
**New files:**
25+
26+
| File | Purpose |
27+
|---|---|
28+
| `evals/laaf/README.md` | Integration guide — LPCI attack vectors, LAAF stage × OWASP crosswalk, threshold definitions, quickstart, CI/CD |
29+
| `evals/laaf/run_laaf.sh` | Full S1–S6 suite runner with per-stage thresholds (0% for S1/S3/S4/S6; 5% for S2; 10% for S5) |
30+
| `evals/laaf/laaf_crosswalk.py` | Maps LAAF scan results to OWASP entries and MAESTRO layers; outputs MD/CSV/JSON |
31+
| `evals/laaf/stage_configs/s1.yaml` | S1 Reconnaissance — system prompt extraction (LLM07, LLM01) |
32+
| `evals/laaf/stage_configs/s2.yaml` | S2 Logic-Layer Injection — RAG/document poisoning (LLM01, ASI01, DSGAI04) |
33+
| `evals/laaf/stage_configs/s3.yaml` | S3 Trigger Execution — memory-restored payload activation (ASI01, ASI06, LLM06) |
34+
| `evals/laaf/stage_configs/s4.yaml` | S4 Persistence and Reuse — cross-session foothold (ASI06, LLM06, DSGAI04) |
35+
| `evals/laaf/stage_configs/s5.yaml` | S5 Evasion — layered encoding filter bypass (LLM01, LLM02) |
36+
| `evals/laaf/stage_configs/s6.yaml` | S6 Trace Tampering — audit log concealment (DSGAI01, LLM07) |
37+
38+
**LPCI attack vectors → OWASP crosswalk:**
39+
40+
| Vector | Description | OWASP | MAESTRO |
41+
|---|---|---|---|
42+
| AV-1 Tool Poisoning | Compromise tool definitions | ASI02, LLM01, DSGAI04 | L3 Origin |
43+
| AV-2 Memory-Persistent Triggers | Dormant encoded payloads in memory | ASI06, ASI01 | L2 Origin, L7 Impact |
44+
| AV-3 Role Override | Privilege escalation via memory entrenchment | ASI03, LLM06, ASI01 | L6 Origin |
45+
| AV-4 Vector Store Persistence | Adversarial content indexed in RAG corpus | DSGAI04, LLM01, ASI06 | L2 Origin, L1 Impact |
46+
47+
**Updated:**
48+
- `evals/ci/github-action.yml``laaf-eval` job added (schedule + manual dispatch; 6 stages with crosswalk stage configs)
49+
- `data/incidents.json` — INC-021: LAAF empirical study, 67–100% LPCI breakthrough rates across 5 production LLMs (Atta et al., arXiv:2507.10457)
50+
- `data/tools-supplement.json` — NEW: supplemental tools file merged into entries at generation time
51+
- `scripts/generate.js` — loads `tools-supplement.json` + fixed `acc[id].tools` accumulation assignment
52+
- `data/entries/` — LAAF v2.0 added to tools for LLM01, LLM06, LLM07, ASI01, ASI02, ASI03, ASI06, DSGAI04
53+
- `docs/data.js` — regenerated
1354

1455
---
1556

README.md

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
[![License: CC BY-SA 4.0](https://img.shields.io/badge/License-CC%20BY--SA%204.0-lightgrey.svg)](https://creativecommons.org/licenses/by-sa/4.0/)
44
[![OWASP Lab](https://img.shields.io/badge/OWASP-GenAI%20Data%20Security-blue)](https://genai.owasp.org)
5-
[![Version](https://img.shields.io/badge/version-1.5.6-green)](CHANGELOG.md)
5+
[![Version](https://img.shields.io/badge/version-1.5.7-green)](CHANGELOG.md)
66
[![Source Lists](https://img.shields.io/badge/source%20lists-3-blueviolet)](README.md)
77
[![Mapping Files](https://img.shields.io/badge/mapping%20files-58-brightgreen)](README.md)
88
[![Frameworks](https://img.shields.io/badge/frameworks-17-orange)](README.md)
@@ -27,7 +27,8 @@ Every file answers one question: **which controls from framework X address vulne
2727
| **40+** open-source tools | Catalogued and organised by function |
2828
| **10** eval profiles | Runnable Garak + PyRIT tests mapped to OWASP entries |
2929
| **17** compliance reports | Per-framework gap assessments auto-generated from data layer |
30-
| **20** documented incidents | Real-world + research incidents with MAESTRO layer attribution |
30+
| **21** documented incidents | Real-world + research incidents with MAESTRO layer attribution |
31+
| **LAAF v2.0** | First agentic LPCI red-teaming framework — fully integrated with 6-stage × OWASP crosswalk |
3132

3233
All free. All open-source. Built for practitioners.
3334

@@ -255,6 +256,7 @@ GenAI-Security-Crosswalk/
255256
│ ├── README.md ← Setup guide and result interpretation
256257
│ ├── garak/ ← 7 YAML profiles (LLM01/02/04/07/09, ASI01/05)
257258
│ ├── pyrit/ ← 3 async Python scripts (LLM01, DSGAI04, ASI01)
259+
│ ├── laaf/ ← LAAF v2.0 LPCI suite (S1–S6 + crosswalk reporter)
258260
│ └── ci/ ← github-action.yml — drop-in CI/CD template
259261
260262
└── i18n/
@@ -280,6 +282,30 @@ node scripts/compliance-report.js --list-frameworks # see all options
280282

281283
Each report includes: executive summary, coverage matrix (OWASP entries × controls), per-control detail with notes, and a prioritised action plan.
282284

285+
## LAAF v2.0 — LPCI red-teaming
286+
287+
[LAAF v2.0](https://github.com/qorvexconsulting1/laaf-V2.0) is integrated as the third evaluation framework alongside Garak and PyRIT. It covers the attack surface that surface-level injection tests miss: memory persistence, layered encoding, semantic reframing, and 6-stage lifecycle attacks.
288+
289+
```bash
290+
pip install git+https://github.com/qorvexconsulting1/laaf-V2.0.git
291+
export OPENAI_API_KEY=sk-...
292+
bash evals/laaf/run_laaf.sh # S1–S6 full suite
293+
laaf scan --target mock --dry-run # No API key needed
294+
```
295+
296+
| LAAF Stage | OWASP | Threshold |
297+
|---|---|---|
298+
| S1 Reconnaissance | LLM07, LLM01 | 0% |
299+
| S2 Logic-Layer Injection | LLM01, ASI01, DSGAI04 | 5% |
300+
| S3 Trigger Execution | ASI01, ASI06, LLM06 | 0% |
301+
| S4 Persistence | ASI06, LLM06, DSGAI04 | 0% |
302+
| S5 Evasion | LLM01, LLM02 | 10% |
303+
| S6 Trace Tampering | DSGAI01, LLM07 | 0% |
304+
305+
See `evals/laaf/README.md` for the full LPCI attack vector → OWASP → MAESTRO crosswalk.
306+
307+
---
308+
283309
## Incident tracker
284310

285311
20 real-world and research-demonstrated incidents, each mapped to OWASP entries and MAESTRO architectural layers:

data/entries/ASI01.json

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -668,6 +668,11 @@
668668
"name": "Teleport",
669669
"type": "commercial",
670670
"url": "https://goteleport.com"
671+
},
672+
{
673+
"name": "LAAF v2.0",
674+
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
675+
"type": "open-source"
671676
}
672677
],
673678
"incidents": [
@@ -712,6 +717,12 @@
712717
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
713718
"year": 2024,
714719
"incident_id": "INC-020"
720+
},
721+
{
722+
"name": "LAAF v2.0 — Empirical LPCI breakthrough rates of 67–100% across 5 production LLMs",
723+
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
724+
"year": 2026,
725+
"incident_id": "INC-021"
715726
}
716727
],
717728
"crossrefs": {

data/entries/ASI02.json

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -642,6 +642,11 @@
642642
"name": "HashiCorp Vault",
643643
"type": "open-source",
644644
"url": "https://www.vaultproject.io"
645+
},
646+
{
647+
"name": "LAAF v2.0",
648+
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
649+
"type": "open-source"
645650
}
646651
],
647652
"incidents": [
@@ -668,6 +673,12 @@
668673
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
669674
"year": 2024,
670675
"incident_id": "INC-019"
676+
},
677+
{
678+
"name": "LAAF v2.0 — Empirical LPCI breakthrough rates of 67–100% across 5 production LLMs",
679+
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
680+
"year": 2026,
681+
"incident_id": "INC-021"
671682
}
672683
],
673684
"crossrefs": {

data/entries/ASI03.json

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -683,6 +683,11 @@
683683
"name": "AWS Secrets Manager",
684684
"type": "commercial",
685685
"url": "https://aws.amazon.com/secrets-manager/"
686+
},
687+
{
688+
"name": "LAAF v2.0",
689+
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
690+
"type": "open-source"
686691
}
687692
],
688693
"incidents": [
@@ -691,6 +696,12 @@
691696
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
692697
"year": 2024,
693698
"incident_id": "INC-019"
699+
},
700+
{
701+
"name": "LAAF v2.0 — Empirical LPCI breakthrough rates of 67–100% across 5 production LLMs",
702+
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
703+
"year": 2026,
704+
"incident_id": "INC-021"
694705
}
695706
],
696707
"crossrefs": {

data/entries/ASI06.json

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -622,6 +622,11 @@
622622
"name": "Chroma",
623623
"type": "open-source",
624624
"url": "https://www.trychroma.com"
625+
},
626+
{
627+
"name": "LAAF v2.0",
628+
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
629+
"type": "open-source"
625630
}
626631
],
627632
"incidents": [
@@ -630,6 +635,12 @@
630635
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
631636
"year": 2024,
632637
"incident_id": "INC-016"
638+
},
639+
{
640+
"name": "LAAF v2.0 — Empirical LPCI breakthrough rates of 67–100% across 5 production LLMs",
641+
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
642+
"year": 2026,
643+
"incident_id": "INC-021"
633644
}
634645
],
635646
"crossrefs": {

data/entries/DSGAI04.json

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -568,10 +568,20 @@
568568
"type": "open-source",
569569
"url": "https://github.com/twosixlabs/armory"
570570
},
571+
{
572+
"name": "Cleanlab",
573+
"type": "open-source",
574+
"url": "https://github.com/cleanlab/cleanlab"
575+
},
571576
{
572577
"name": "Great Expectations",
573578
"type": "open-source",
574579
"url": "https://github.com/great-expectations/great_expectations"
580+
},
581+
{
582+
"name": "LAAF v2.0",
583+
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
584+
"type": "open-source"
575585
}
576586
],
577587
"incidents": [
@@ -592,6 +602,12 @@
592602
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
593603
"year": 2024,
594604
"incident_id": "INC-016"
605+
},
606+
{
607+
"name": "LAAF v2.0 — Empirical LPCI breakthrough rates of 67–100% across 5 production LLMs",
608+
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
609+
"year": 2026,
610+
"incident_id": "INC-021"
595611
}
596612
],
597613
"crossrefs": {

data/entries/DSGAI21.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -684,6 +684,11 @@
684684
"type": "open-source",
685685
"url": "https://github.com/shmsw25/FActScoring"
686686
},
687+
{
688+
"name": "Cleanlab",
689+
"type": "open-source",
690+
"url": "https://github.com/cleanlab/cleanlab"
691+
},
687692
{
688693
"name": "RAGAS",
689694
"type": "open-source",

data/entries/LLM01.json

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -684,6 +684,11 @@
684684
"name": "Promptfoo",
685685
"type": "open-source",
686686
"url": "https://github.com/promptfoo/promptfoo"
687+
},
688+
{
689+
"name": "LAAF v2.0",
690+
"url": "https://github.com/qorvexconsulting1/laaf-V2.0",
691+
"type": "open-source"
687692
}
688693
],
689694
"incidents": [
@@ -758,6 +763,12 @@
758763
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
759764
"year": 2024,
760765
"incident_id": "INC-020"
766+
},
767+
{
768+
"name": "LAAF v2.0 — Empirical LPCI breakthrough rates of 67–100% across 5 production LLMs",
769+
"url": "https://github.com/emmanuelgjr/GenAI-Security-Crosswalk/blob/main/data/incidents.json",
770+
"year": 2026,
771+
"incident_id": "INC-021"
761772
}
762773
],
763774
"crossrefs": {

data/entries/LLM04.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -626,6 +626,11 @@
626626
"type": "commercial",
627627
"url": "https://www.dragos.com"
628628
},
629+
{
630+
"name": "Cleanlab",
631+
"type": "open-source",
632+
"url": "https://github.com/cleanlab/cleanlab"
633+
},
629634
{
630635
"name": "Garak",
631636
"type": "open-source",

0 commit comments

Comments
 (0)