fix(ci): unblock benchmark baseline (jq ARG_MAX exit 126)#42947
Open
MajorLift wants to merge 2 commits into
Open
fix(ci): unblock benchmark baseline (jq ARG_MAX exit 126)#42947MajorLift wants to merge 2 commits into
MajorLift wants to merge 2 commits into
Conversation
… 126)
`benchmark-stats-commit.sh` passed the aggregated benchmark payload to jq as a
single `--argjson presets` command-line argument. As the payload grew (web
vitals, long-task/TBT metrics, more steps, user-journey presets), it crossed the
kernel per-argument limit (`MAX_ARG_STRLEN`, ~128KB), so jq could not be exec'd
("Argument list too long") and the commit step exited 126.
Because `store-benchmark-stats` is `continue-on-error: true`, this surfaced as a
green check while the `main` benchmark baseline silently stopped updating
(no user-journey data since 2026-04-02, nothing at all since 2026-05-02).
Pass the large JSON via stdin, and read `page_load_json` from a temp file with
`--slurpfile`, so neither blob lands on argv.
MajorLift
requested review from
a team,
HowardBraham and
itsyoboieltr
as code owners
Contributor
|
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
Contributor
Builds ready [e21ea4e] [reused from 89d089f]
⚡ Performance Benchmarks (Total: 🟢 18 pass · 🟡 5 warn · 🔴 2 fail)
Bundle size diffs
|
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit e21ea4e. Configure here.
The first fix moved `presets_json` off argv inside
`assemble_performance_data`, but `COMMIT_DATA` (the
`{ timestamp, presets: ... }` wrapper, strictly larger than the inner
presets blob) was still passed as `--argjson data "${COMMIT_DATA}"` on
the final write. That command hits the same `MAX_ARG_STRLEN` limit and
silently fails under `continue-on-error`, keeping the baseline frozen.
Read `COMMIT_DATA` from a temp file via `--slurpfile` so nothing large
lands on argv.
Caught by Cursor Bugbot on #42947.
Contributor
Builds ready [61b250b] [reused from 31e5270]
⚡ Performance Benchmarks (Total: 🟢 18 pass · 🟡 3 warn · 🔴 3 fail)
Bundle size diffs
|
davidmurdoch
approved these changes
May 26, 2026
Contributor
davidmurdoch
left a comment
davidmurdoch
left a comment
There was a problem hiding this comment.
ultimately we should switch to a proper DB, only store an aggregate, or using some sort of rolling log files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
benchmark-stats-commit.shpassed the aggregated benchmark payload tojqas a single--argjson presetsargument. As the payload grew (web vitals, long-task/TBT, more steps, user-journey presets), it crossed the kernel per-arg limit (MAX_ARG_STRLEN, ~128KB), sojqcould not be exec'd ("Argument list too long") and the step exited 126.continue-on-error: true, so the failure surfaced as a green check while the rolling baseline behind PR benchmark comparisons silently stopped updating (no user-journey data since 2026-04-02, nothing at all since 2026-05-02).presets_jsonvia stdin andpage_load_jsonvia--slurpfileso neither blob lands on argv. After this lands, the next main push commits a freshperformance_data.jsonand the rolling baseline self-corrects (no manual reset).Fixes MetaMask/MetaMask-planning#7279.
CHANGELOG entry: null
Test plan
mainpush adds a fresh commit tostats/main/performance_data.jsonwith all presets includinguserJourneyOnboarding*Note
Low Risk
CI-only shell/jq plumbing for benchmark stats commits; no extension runtime, auth, or user data paths.
Overview
Fixes CI benchmark baseline commits that could fail silently when aggregated JSON grew past the kernel argv limit (~128KB), so
jqnever ran (exit 126, "Argument list too long").benchmark-stats-commit.shno longer passes large blobs asjq--argjsonarguments. It pipespresets_jsonon stdin, loadspage_load_jsonandCOMMIT_DATAvia temp files and--slurpfile, including when merging startup data intopageLoadand when appending a commit entry toperformance_data.json.After merge, the next
mainrun should resume updatingextension_benchmark_statsand PR benchmark comparisons should use fresh baselines again.Reviewed by Cursor Bugbot for commit 61b250b. Bugbot is set up for automated code reviews on this repo. Configure here.