Skip to content

Implement per-step validation and replanning#1769

Draft
philippjfr wants to merge 6 commits intomainfrom
validation_replan
Draft

Implement per-step validation and replanning#1769
philippjfr wants to merge 6 commits intomainfrom
validation_replan

Conversation

@philippjfr
Copy link
Copy Markdown
Member

Description

  • Add a post-step validation hook in the plan execution loop (after each primary actor task output is available).
  • If step validation fails/incomplete, gather compact replanning context (original user goal, completed steps, failed/current step, validation findings, available agents/tools, current execution context).
  • Invoke Planner in partial replan mode using a new prompt template and update only the remaining plan segment.
  • Continue execution from the newly spliced plan tail.

AI Disclosure

  • This PR contains AI-generated content.
    • I have tested all AI-generated content in my PR.
    • I take responsibility for all AI-generated content in my PR.

Wrote detailed specification of what I wanted the validation / replan loop to look like and reviewed the generated code in detail.

Tools and Models: 5.3 Codex

Checklist

  • Tests added and are passing
  • Added documentation

@codecov
Copy link
Copy Markdown

codecov bot commented Mar 18, 2026

Codecov Report

❌ Patch coverage is 90.70796% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.95%. Comparing base (43c9c5d) to head (e299589).
⚠️ Report is 17 commits behind head on main.

Files with missing lines Patch % Lines
lumen/ai/coordinator/base.py 82.17% 18 Missing ⚠️
lumen/tests/ai/test_coordinator.py 97.16% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1769      +/-   ##
==========================================
+ Coverage   68.75%   68.95%   +0.19%     
==========================================
  Files         171      171              
  Lines       28979    29189     +210     
==========================================
+ Hits        19924    20126     +202     
- Misses       9055     9063       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ahuang11 ahuang11 marked this pull request as draft March 24, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant