[CI][Backports] Avoid triggering all package tests on the first commit of a backport branch

[mrodm]

Summary

When a new backport branch is created via the backport pipeline, a commit is pushed that updates CI infrastructure files (.buildkite/, .github/workflows/, go.mod, go.sum, magefile.go, etc.). This commit triggers a Buildkite build that ends up testing all packages in the repository, even though no package code has changed.

Testing all packages in this scenario is unnecessary: the packages are identical to the commit in main from which the branch was created, and those were already tested there.

Root cause

The function get_from_changeset() in .buildkite/scripts/common.sh#L626 falls back to origin/${BUILDKITE_BRANCH}^ when no previous successful build exists for the branch. The diff from that point to HEAD includes the CI infrastructure files committed by the backport script, which are not covered by the non_package_patterns exclusion list in is_pr_affected(), causing all packages to be marked as affected.

Suggested approaches

Option A — Reuse the existing get_previous_successful_commit signal

get_from_changeset() already calls get_previous_successful_commit() for backport-* branches (common.sh#L642-L649). When it returns "null", no successful build has ever run on that branch — for a freshly created backport branch this means it is the initial push.

The fix would change the fallback behaviour in that function:

if [[ "${previous_successful_commit}" == "null" ]]; then
    # First push of a new backport branch: only CI infrastructure files changed,
    # no package code was modified — skip package testing.
    from="${BUILDKITE_COMMIT}"  # same as $to → empty diff
fi

This is a minimal change that reuses already-computed data (Buildkite API call already in place).

Option B — Count commits since the branching point from main

commit_count=$(git rev-list --count "$(git merge-base origin/main HEAD)..HEAD")

If commit_count == 1, only the CI infrastructure commit was added on top of the main branching point, confirming this is the first push of the backport branch. This approach is independent of the Buildkite API and does not rely on build history being available.

Both options can be combined as a guard: branch name matches ^backport- and one of the above conditions is true.

[infra-vault-gh-plugin-prod]

Pinging @elastic/ecosystem (Team:Ecosystem)

[github-actions]

tl;dr: The issue is valid, and the most reliable fix is to handle the backport-* bootstrap build in .buildkite/scripts/common.sh:get_from_changeset() so the first branch-init commit produces an empty diff only for that bootstrap case.

Recommendation

Update get_from_changeset() in .buildkite/scripts/common.sh to add a backport bootstrap guard instead of expanding file allowlists:

1. Keep current behavior when previous_successful_commit exists.
2. If previous_successful_commit == "null", try get_previous_commit (failed/passed) before any skip logic.
3. Only when there is no prior build history, detect bootstrap (backport-* + one commit since merge-base with origin/main) and set from="$BUILDKITE_COMMIT" (empty diff).
4. Otherwise preserve current fallback.

This avoids full fan-out on initial backport branch creation while not masking later real changes.

Findings

• .buildkite/scripts/common.sh:626-650 currently does this for main/backport-*:
  • use get_previous_successful_commit
  • if null, fallback to origin/${BUILDKITE_BRANCH}^
• .buildkite/scripts/common.sh:655-663 sets to="origin/${BUILDKITE_BRANCH}" for main/backport-*.
• .buildkite/scripts/common.sh:759-785 computes git diff from merge-base(from,to) and marks package affected when non-excluded files are present.
• .buildkite/scripts/common.sh:765-780 non-package exclusions do not include go.mod, go.sum, magefile.go, tools.go, or dev/**.
• .buildkite/scripts/backport_branch.sh:150-223,241-255 bootstrap commit updates infra paths (.buildkite, dev/*, .github/workflows, go.mod, go.sum, magefile.go, .go-version, tools.go) and pushes the new backport-* branch.
• docs/ci_pipelines.md:71-73 documents expected branch behavior as “diff against latest successful build in branch,” which matches this fix direction.

Why not solve this in non_package_patterns:

• It would require broad exclusions for infra/tooling paths and can easily hide real CI/tooling changes that should intentionally fan out.

Verification

I ran focused local checks to validate the current behavior:

$ cd /home/runner/work/integrations/integrations && regex='^(packages/|\.github/(CODEOWNERS|ISSUE_TEMPLATE|PULL_REQUEST_TEMPLATE|workflows/)|CODE_OF_CONDUCT\.md|README\.md|docs/|catalog-info\.yaml|\.buildkite/pull-requests\.json|\.buildkite/pipeline\.schedule-daily\.yml|\.buildkite/pipeline\.schedule-weekly\.yml|\.buildkite/pipeline\.backport\.yml|\.buildkite/pipeline\.publish\.yml|\.buildkite/scripts/packages/.+\.sh|\.buildkite/scripts/backport_branch\.sh|\.buildkite/scripts/build_packages\.sh)' && printf '%s
' '.buildkite/scripts/common.sh' '.github/workflows/ci.yml' 'go.mod' 'go.sum' 'magefile.go' 'tools.go' '.buildkite/pipeline.backport.yml' | tee /tmp/gh-aw/agent/changed_files_sample.txt && echo '--- Files considered non-package-impacting (matched by regex):' && grep -E "$regex" /tmp/gh-aw/agent/changed_files_sample.txt && echo '--- Files considered affecting all packages (NOT matched by regex):' && grep -E -v "$regex" /tmp/gh-aw/agent/changed_files_sample.txt
.buildkite/scripts/common.sh
.github/workflows/ci.yml
go.mod
go.sum
magefile.go
tools.go
.buildkite/pipeline.backport.yml
--- Files considered non-package-impacting (matched by regex):
.github/workflows/ci.yml
.buildkite/pipeline.backport.yml
--- Files considered affecting all packages (NOT matched by regex):
.buildkite/scripts/common.sh
go.mod
go.sum
magefile.go
tools.go

$ cd /tmp/gh-aw/agent && rm -rf backport-sim && mkdir backport-sim && cd backport-sim && git init -q && git config user.email 'triage@example.com' && git config user.name 'triage' && mkdir -p packages/pkg && echo base > packages/pkg/file.txt && git add . && git commit -q -m 'base main' && git branch -M main && git checkout -q -b backport-pkg-1.2 && echo infra > go.mod && git add go.mod && git commit -q -m 'infra bootstrap commit' && TO=$(git rev-parse HEAD) && FROM_OLD='backport-pkg-1.2^' && FROM_PROPOSED="$TO" && MB_OLD=$(git merge-base "$FROM_OLD" "$TO") && MB_PROPOSED=$(git merge-base "$FROM_PROPOSED" "$TO") && echo "to=$TO" && echo "old_from=$FROM_OLD merge_base=$MB_OLD" && echo "proposed_from=$FROM_PROPOSED merge_base=$MB_PROPOSED" && echo '--- old diff names:' && git diff --name-only "$MB_OLD" "$TO" && echo '--- proposed diff names:' && git diff --name-only "$MB_PROPOSED" "$TO"
to=b9ac8c1d89be24605f765f5686a8f7b0f6e748e5
old_from=backport-pkg-1.2^ merge_base=c1753e5a299eccff99b9659f0e74e734241e1ffa
proposed_from=b9ac8c1d89be24605f765f5686a8f7b0f6e748e5 merge_base=b9ac8c1d89be24605f765f5686a8f7b0f6e748e5
--- old diff names:
go.mod
--- proposed diff names:

Detailed Action Plan

1. In .buildkite/scripts/common.sh:get_from_changeset() (~626-650), keep existing PR path unchanged.
2. In main/backport-* branch path:
   • Read previous_successful_commit first (current behavior).
   • If null, read previous_commit (failed/passed).
   • If that is also null and branch matches ^backport-:
     • Compute merge-base with origin/main and count commits from that base to origin/${BUILDKITE_BRANCH}.
     • If count is 1, set from="$BUILDKITE_COMMIT" so from==to and package diff is empty.
   • Keep existing fallback for non-bootstrap cases.
3. Verify with three scenarios in Buildkite:
   • First push of a newly created backport-* branch => “No packages to be tested”.
   • Next commit on same branch => selective package testing resumes.
   • Failed first build followed by second commit => second run diffs against previous commit, not empty diff.
4. Optional hardening: add a short debug line in trigger_integrations_in_parallel.sh showing resolved from and to to ease future triage.

Related Items

Type	Link	Relevance
Issue	#18869	Current triage target
Issue	#17403	Earlier CI/backport script refactors; context for recent behavior changes
PR	#18577	Modified backport + CI scripts and bootstrap content paths
PR	#8463	Introduced/reworked from/to reuse for main/backport-* diff logic
File	.buildkite/scripts/common.sh:626-650	get_from_changeset() fallback causing first-backport fan-out
File	.buildkite/scripts/common.sh:765-785	Non-package filtering and package-affect decision
File	.buildkite/scripts/backport_branch.sh:150-223,241-255	Bootstrap commit adds infra files that trigger fan-out
File	docs/ci_pipelines.md:71-73	Documented branch diff contract (latest successful build)

Note

🔒 Integrity filter blocked 8 items

The following items were blocked because they don't meet the GitHub integrity level.

• #1115 search_issues: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #16732 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #15829 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #14725 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #10895 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #7144 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #5197 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• #2715 search_pull_requests: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: Issue Triage

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.