Skip to content

[codex] Add scientific validation and live concordance panel#26

Merged
senseibelbi merged 1 commit intomainfrom
codex/live-concordance-scientific-validation
Apr 12, 2026
Merged

[codex] Add scientific validation and live concordance panel#26
senseibelbi merged 1 commit intomainfrom
codex/live-concordance-scientific-validation

Conversation

@senseibelbi
Copy link
Copy Markdown
Collaborator

What changed

  • added an offline scientific-validation harness that runs orchestrator scenarios and emits JSON/Markdown scorecards with assessment rollups and interop attachment coverage
  • added a live CTX concordance reference panel that exercises the real evidence synthesizer against curated ToxVal cases with expected robust/limited outcomes
  • extended evidence concordance matching so it can compare against real ToxVal-style fields such as toxicologicalEffect and toxvalNumeric
  • documented the new validation layers in the workflow/testing docs and exported the new APIs from the package surface

Why this changed

The orchestrator validation story was still mostly structural. We needed a reproducible way to detect scientific agreement drift in the actual evidence synthesis path, especially when upstream CTX payloads evolve.

Impact

  • maintainers can now run scripts/scientific_validation_report.py for offline structural validation
  • maintainers can now run scripts/live_concordance_panel.py for live CTX-backed concordance drift detection
  • evidence synthesis is materially more defensible against real ToxVal payload shapes

Validation

  • pytest -q tests/workflows/test_live_concordance_panel.py tests/workflows/test_scientific_validation_report.py tests/workflows/test_offline_workflows.py
  • python3 scripts/live_concordance_panel.py --json
  • python3 scripts/scientific_validation_report.py --scenario acute_toxicity --json
  • pytest -q

@senseibelbi senseibelbi marked this pull request as ready for review April 12, 2026 16:08
@senseibelbi senseibelbi merged commit f1e459e into main Apr 12, 2026
7 checks passed
@senseibelbi senseibelbi deleted the codex/live-concordance-scientific-validation branch April 12, 2026 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant