Skip to content

[Agents] [Agent Skills] Add PR review agent and skills#2643

Open
bhapas wants to merge 2 commits intoelastic:mainfrom
bhapas:pr_review_agent
Open

[Agents] [Agent Skills] Add PR review agent and skills#2643
bhapas wants to merge 2 commits intoelastic:mainfrom
bhapas:pr_review_agent

Conversation

@bhapas
Copy link
Copy Markdown
Contributor

@bhapas bhapas commented May 6, 2026

Summary: ECS PR quality review agent

What this PR does

Adds an automated PR quality review for ECS schema contributions, complementary to PR Triage (routing). It focuses on field-level quality: naming, descriptions, types, examples vs pattern / expected_values, OTel hints, and overlap/conflicts against a base-branch schema inventory. Findings use High / Medium / Low severity and are posted as a PR comment with a summary and collapsible <details> sections.

Changes

New workflow

  • .github/workflows/pr-review.yml
    • Triggers: pull_request_target (opened, synchronize, ready_for_review); workflow_dispatch (PR number, optional model).
    • Phase A: Checkout base SHA; generate schema-inventory.tsv from schemas/*.yml; gh pr diff; OpenCode + LiteLLM (same secrets pattern as pr-triage.yml); write pr-review-report.md.
    • Phase B: Post PR comment from artifact.
    • Concurrency: pr-review-${{ pr_number }}, cancel in-progress.

New skill — .agents/skills/ecs-pr-review/

  • SKILL.md — Execution steps for agents (scope, diff, rules, inventory, report).
  • quality-rules.md — Rule IDs, severities, detection, remediation.
  • report-template.md — Comment layout (title, Overall counts, ### Summary details, severity <details>).

Rule update

  • .agents/rules/ecs-schema-standards.mdc — Links to ecs-pr-review skill, quality-rules, and pr-review.yml.

Configuration

  • LITELLM_API_KEY required (same as PR Triage). Optional: LITELLM_MODEL, LITELLM_BASE_URL, repo var LITELLM_MODEL_DEFAULT.

Notes

  • Comment-only; does not approve or block merge.
  • Does not replace routing/triage (RFC vs direct PR).

@bhapas bhapas self-assigned this May 6, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 6, 2026

🤖 GitHub comments

Just comment with:

  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@bhapas bhapas marked this pull request as ready for review May 8, 2026 11:56
@bhapas bhapas requested a review from a team as a code owner May 8, 2026 11:56
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 8, 2026

ECS PR Triage (automated)

PR Triage Report

PR: #2643 — [Agents] [Agent Skills] Add PR review agent and skills
Classification: Direct PR
Change type: Tooling
Scope: Moderate

Summary

This PR adds a new automated PR quality review agent alongside the existing PR triage agent. It introduces a new GitHub Actions workflow (.github/workflows/pr-review.yml), a new agent skill (ecs-pr-review) with quality rules and a report template, and a small cross-reference update to the existing ecs-schema-standards.mdc rule. No schema YAML files are modified, no fields are added or changed, and no breaking changes are introduced. The change is purely tooling/CI infrastructure, making it appropriate for a direct PR.

Files changed

  • Schemas: none
  • Generated: none
  • Tooling/scripts/tests: none (no changes under scripts/)
  • Docs (hand-authored): none
  • CI / GitHub: .github/workflows/pr-review.yml (new, +439 lines)
  • RFCs: none
  • Agent infrastructure (.agents/):
    • .agents/rules/ecs-schema-standards.mdc (modified, +5 lines — added cross-reference to new skill)
    • .agents/skills/ecs-pr-review/SKILL.md (new, +75 lines)
    • .agents/skills/ecs-pr-review/quality-rules.md (new, +112 lines)
    • .agents/skills/ecs-pr-review/report-template.md (new, +81 lines)

Routing decision

Direct PR is appropriate. This change is entirely tooling and CI automation — it adds a new GitHub Actions workflow and agent skill definitions. No schemas/*.yml files are touched, no fields are introduced or modified, and no ECS semantics are altered. Per classification-rules §2, "CI / automation" and "Tooling only" changes are low-risk direct PR patterns. No RFC triggers from §1 apply (no new field sets, no breaking changes, no new reuse topology, no novel use case, no ECS-wide scope changes, no new fields).

Risk notes

  • Breaking / deprecation: No. The workflow and skill are purely additive. The workflow uses pull_request_target which runs in the base-repo context — this is an intentional security pattern to access secrets for fork PRs, consistent with the existing pr-triage.yml workflow. Reviewers should verify that the workflow does not check out or execute untrusted PR code (it checks out the base SHA and reads PR diff via gh, which appears correct).
  • OTel / semconv: N/A — no schema fields involved.
  • Scope / reuse: N/A — no field sets, reuse, or categorization fields affected.

Completeness checklist

  • PR description (all sections) — The PR body does not follow the standard 7-section PR template (.github/PULL_REQUEST_TEMPLATE.md). However, it provides thorough documentation of what changed, why, and configuration details using a custom format. The "what", "why", and "notes" are well covered. Sections 4–6 (documentation, make build, tests) are not explicitly answered, though they are less relevant for a tooling-only PR with no schema changes.
  • CHANGELOG.next.md — Not required. No schemas/ or scripts/ files are changed. Agent infrastructure under .agents/ and .github/workflows/ does not require a changelog entry per repo conventions.
  • make + committed generated outputs — Not applicable. No schema changes; no regeneration needed.
  • OTel otel: on new/changed semconv-related fields — Not applicable. No schema fields changed.
  • Tests / make check — Not explicitly confirmed in the PR description. For a CI workflow change, manual testing of the workflow (e.g., via workflow_dispatch) would be the appropriate verification. Reviewers may want to confirm the workflow was tested.
  • CLA (contributor) — Author (@bhapas) appears to be an Elastic employee based on contribution history.

Recommended next actions

  1. Reviewer: Verify the pull_request_target workflow security model — confirm that no untrusted code from the PR branch is checked out or executed (the workflow checks out github.event.pull_request.base.sha, which is correct).
  2. Reviewer: Confirm the workflow was tested via workflow_dispatch on a test PR to validate the end-to-end flow (schema inventory generation, OpenCode invocation, comment posting).
  3. Reviewer: Consider whether the PR description should be updated to use the standard PR template format for consistency, or whether the custom format is acceptable for agent/tooling PRs.
  4. Contributor: No blocking issues identified. The PR is well-documented and appropriately scoped as a direct PR.

Posted by PR Triage workflow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant