Skip to content

fix: serialize generated pack regeneration#3231

Open
justin808 wants to merge 2 commits intomainfrom
codex/lock-generated-packs
Open

fix: serialize generated pack regeneration#3231
justin808 wants to merge 2 commits intomainfrom
codex/lock-generated-packs

Conversation

@justin808
Copy link
Copy Markdown
Member

@justin808 justin808 commented Apr 30, 2026

Summary

  • guard generate_packs_if_stale with a Rails tmp lock file so concurrent callers cannot clean and regenerate packs at the same time
  • re-check staleness after waiting for the lock so the second caller can skip work when the first already regenerated files
  • clean stale unlocked lock files after the short TTL

Fixes #1627

Test plan

  • bundle exec rubocop react_on_rails/lib/react_on_rails/packs_generator.rb react_on_rails/spec/dummy/spec/packs_generator_spec.rb
  • bundle exec rspec -I spec/dummy/spec spec/dummy/spec/packs_generator_spec.rb:414 (from react_on_rails/)
  • git diff --check

Note: bundle install inside react_on_rails/spec/dummy failed in this fresh worktree while compiling Nokogiri (nokogiri_gumbo.h missing), so the targeted dummy spec was run through the root bundle with the dummy spec load path.


Note

Medium Risk
Touches autobundling pack generation and introduces cross-process locking; misbehavior could cause hangs or skipped regeneration if the lock/TTL logic is wrong. Scope is contained and has dedicated concurrency test coverage.

Overview
Prevents concurrent callers of generate_packs_if_stale from racing by serializing generated pack cleanup/regeneration with an exclusive Rails tmp/react_on_rails_generate_packs.lock file.

After waiting for the lock, the generator now re-checks whether packs are already up to date to avoid redundant regeneration, and it cleans up stale lock files after a short TTL. Specs were updated to cover the concurrency behavior and to properly restore auto_load_bundle, and the CHANGELOG documents the fix.

Reviewed by Cursor Bugbot for commit 136def3. Bugbot is set up for automated code reviews on this repo. Configure here.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 30, 2026

Warning

Rate limit exceeded

@justin808 has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 37 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 57ac56f9-11d2-4be7-8362-9cbacb6c6130

📥 Commits

Reviewing files that changed from the base of the PR and between f25cde3 and 136def3.

📒 Files selected for processing (3)
  • CHANGELOG.md
  • react_on_rails/lib/react_on_rails/packs_generator.rb
  • react_on_rails/spec/dummy/spec/packs_generator_spec.rb
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/lock-generated-packs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
Review rate limit: 0/8 reviews remaining, refill in 37 seconds.

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80b3545cdf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

File.open(lock_path, File::RDWR) do |lock_file|
next unless lock_file.flock(File::LOCK_EX | File::LOCK_NB)

FileUtils.rm_f(lock_path)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid unlinking lock file while other callers may have opened it

Removing the lock path in remove_stale_generated_packs_lock can break mutual exclusion when multiple callers arrive during stale-lock recovery: one caller can rm_f the stale path and then lock a newly created inode, while another caller that already opened the old inode (before unlink) later acquires a lock on that old inode. In that case both callers run generate_packs_if_stale concurrently, which defeats the serialization guarantee and can reintroduce concurrent clean/regenerate races after a crash leaves a stale lock file.

Useful? React with 👍 / 👎.

@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented Apr 30, 2026

Greptile Summary

This PR serializes concurrent calls to generate_packs_if_stale using a Rails tmp/ file lock (flock), adding a stale-lock cleanup mechanism with a 120 s TTL and a re-check of staleness after the lock is acquired so that the second caller skips redundant regeneration. The locking approach is sound — flock semantics, ensure-based unlock, and Errno::ENOENT rescue are all handled correctly — but every caller currently serializes through the lock even for add_generated_pack_to_server_bundle and clean_non_generated_files_with_feedback, which run before the staleness re-check; adding a lightweight pre-lock staleness guard would restore fast-path throughput for the steady-state case.

Confidence Score: 4/5

Safe to merge — the locking logic is correct and the race condition it fixes is real; all findings are P2.

No P0 or P1 issues found. flock usage, ensure-based unlock, LOCK_NB stale cleanup, and return-inside-block semantics are all correct. Two P2 concerns: a timing-sensitive test assertion that may be flaky on slow CI, and the lack of a pre-lock fast path causing all callers to serialize even when packs are current.

react_on_rails/lib/react_on_rails/packs_generator.rb — consider a pre-lock staleness guard for throughput

Important Files Changed

Filename Overview
react_on_rails/lib/react_on_rails/packs_generator.rb Adds file-based exclusive locking around pack generation with stale-lock cleanup; correct use of flock and ensure for release, but all requests serialize through the lock even when packs are already current
react_on_rails/spec/dummy/spec/packs_generator_spec.rb Adds after-block restoration of auto_load_bundle (correct fix for test isolation) and a new concurrency spec; the new test has a timing-sensitive sleep assertion that may be flaky on slow CI

Sequence Diagram

sequenceDiagram
    participant A as Caller A (first)
    participant B as Caller B (concurrent)
    participant LF as Lock File (tmp/)
    participant FS as Pack Files

    A->>LF: remove_stale_generated_packs_lock (LOCK_NB check)
    A->>LF: File.open + flock(LOCK_EX) → acquired
    A->>FS: add_generated_pack_to_server_bundle
    A->>FS: clean_non_generated_files_with_feedback
    A->>FS: generated_files_present_and_up_to_date? → false
    A->>FS: clean_generated_directories
    B-->>LF: flock(LOCK_EX) → blocks (A holds lock)
    A->>FS: generate_packs
    A->>LF: flock(LOCK_UN) → lock released
    B->>LF: flock(LOCK_EX) → acquired
    B->>FS: add_generated_pack_to_server_bundle
    B->>FS: clean_non_generated_files_with_feedback
    B->>FS: generated_files_present_and_up_to_date? → true (re-check)
    Note over B: skips regeneration
    B->>LF: flock(LOCK_UN) → lock released
Loading

Reviews (1): Last reviewed commit: "Add changelog for generated pack locking" | Re-trigger Greptile

Comment on lines +439 to +444
sleep 0.1
expect { second_completed.pop(true) }.to raise_error(ThreadError)

release_generation << true
first_thread.join
second_thread.join
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Flaky timing assertion

The sleep 0.1 before the non-blocking pop is not a reliable way to assert that the second thread has reached and is blocked on flock. On a slow CI runner (or under load), the second thread may not have even entered generate_packs_if_stale yet after 100 ms, making second_completed.pop(true) raise ThreadError for the wrong reason — the thread simply hasn't run yet, not because it's blocked on the lock. A more robust pattern uses a second queue/latch that the thread posts to immediately before blocking.

Comment on lines +52 to 65
with_generated_packs_lock(verbose: verbose) do
add_generated_pack_to_server_bundle

# Clean any non-generated files from directories
clean_non_generated_files_with_feedback(verbose: verbose)
# Clean any non-generated files from directories
clean_non_generated_files_with_feedback(verbose: verbose)

are_generated_files_present_and_up_to_date = Dir.exist?(generated_packs_directory_path) &&
File.exist?(generated_server_bundle_file_path) &&
!stale_or_missing_packs?
if generated_files_present_and_up_to_date?
puts Rainbow("✅ Generated packs are up to date, no regeneration needed").green if verbose
return
end

if are_generated_files_present_and_up_to_date
puts Rainbow("✅ Generated packs are up to date, no regeneration needed").green if verbose
return
clean_generated_directories_with_feedback(verbose: verbose)
generate_packs(verbose: verbose)
end
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All callers serialize through the lock even when packs are up to date

add_generated_pack_to_server_bundle and clean_non_generated_files_with_feedback run inside the exclusive lock on every request, before the staleness re-check. When packs are already current (the common case after warm-up), every concurrent request still serializes through these two I/O operations and the lock acquire/release cycle. A lightweight pre-lock generated_files_present_and_up_to_date? guard would let subsequent callers exit immediately without contending on the lock.

Comment on lines +99 to +106
File.open(lock_path, File::RDWR) do |lock_file|
next unless lock_file.flock(File::LOCK_EX | File::LOCK_NB)

FileUtils.rm_f(lock_path)
puts Rainbow("🧹 Removed stale generated packs lock at #{lock_path}").yellow if verbose
ensure
lock_file&.flock(File::LOCK_UN)
end
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 LOCK_UN called even when LOCK_NB was never acquired

When flock(LOCK_EX | LOCK_NB) returns false, the next skips the deletion but the ensure still calls flock(LOCK_UN) on a file description that never held the lock. While benign on Linux/macOS, guarding the unlock with an acquired flag improves clarity and correctness on non-POSIX platforms.

remove_stale_generated_packs_lock(lock_path, verbose: verbose)

File.open(lock_path, File::RDWR | File::CREAT, 0o644) do |lock_file|
puts Rainbow("🔒 Waiting for generated packs lock at #{lock_path}").yellow if verbose
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Waiting for generated packs lock" message is printed before flock(LOCK_EX) is called, so it fires even when the lock is acquired immediately (no contention, no wait). Consider moving it after the flock call, or split into two messages — one before ("acquiring...") and one after ("acquired, proceeding...").

Suggested change
puts Rainbow("🔒 Waiting for generated packs lock at #{lock_path}").yellow if verbose
lock_file.flock(File::LOCK_EX)
puts Rainbow("🔒 Acquired generated packs lock at #{lock_path}").yellow if verbose


File.open(lock_path, File::RDWR | File::CREAT, 0o644) do |lock_file|
puts Rainbow("🔒 Waiting for generated packs lock at #{lock_path}").yellow if verbose
lock_file.flock(File::LOCK_EX)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flock(File::LOCK_EX) blocks indefinitely. The 120-second TTL only removes a stale lock file when no holder is present (the LOCK_NB attempt fails while the holder is still alive). If pack generation hangs (process alive but stuck), every subsequent caller will hang forever with no escape.

Consider using a timeout approach, e.g. a background thread that sends a signal after N seconds, or polling with LOCK_NB in a loop:

deadline = Time.now + GENERATED_PACKS_LOCK_TTL_SECONDS
loop do
  break if lock_file.flock(File::LOCK_EX | File::LOCK_NB)
  raise "Timed out waiting for generated packs lock" if Time.now > deadline
  sleep 0.5
end


yield
ensure
lock_file&.flock(File::LOCK_UN)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lock_file is the block parameter of File.open and is never nil, so &. (safe navigation) is unnecessary here and in the other ensure below. Minor, but it reads as if lock_file could be nil, which could mislead future readers.

Suggested change
lock_file&.flock(File::LOCK_UN)
lock_file.flock(File::LOCK_UN)

Comment on lines +58 to +61
if generated_files_present_and_up_to_date?
puts Rainbow("✅ Generated packs are up to date, no regeneration needed").green if verbose
return
end
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return inside a yield-based block works correctly here — Ruby propagates it as a method-return from generate_packs_if_stale, and the ensure in the File.open block still runs, properly releasing the lock. However, this is a subtle Ruby behavior that could surprise contributors who refactor this code (e.g., wrapping the block in a Proc instead of yield would turn it into a LocalJumpError).

A less surprising alternative is to use a boolean guard:

unless generated_files_present_and_up_to_date?
  clean_generated_directories_with_feedback(verbose: verbose)
  generate_packs(verbose: verbose)
end

This avoids relying on return-from-block semantics entirely.

Comment on lines +439 to +440
sleep 0.1
expect { second_completed.pop(true) }.to raise_error(ThreadError)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sleep 0.1 + non-blocking pop(true) pattern is a timing-based assertion: it assumes that 100 ms is long enough for thread 2 to reach the flock call and block, but not so long that CI load causes a false pass. This is a common source of flakiness in CI environments under load.

A more robust approach is to use a Queue (or Mutex + ConditionVariable) to know when thread 2 is actually blocked on the lock, rather than sleeping and hoping. For example, you could stub with_generated_packs_lock to signal a queue when it is entered by the second caller, before the block is executed.

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented Apr 30, 2026

Code Review

Overall: The approach is sound — using flock to serialize pack regeneration and re-checking staleness after acquiring the lock is the right pattern for this problem. The fix correctly addresses the race condition from issue #1627. A few items worth addressing before merging:

Issues

Medium — indefinite lock wait (no timeout)
flock(File::LOCK_EX) blocks forever. The 120 s TTL only removes a stale file; it does not bound how long a caller waits. If a process hangs during generation (alive but stuck), all subsequent callers block permanently. See inline for a suggested polling loop.

Medium — return inside yield block
return inside the block passed to with_generated_packs_lock returns from generate_packs_if_stale via Ruby's block-return semantics. The ensure in the File.open block does still run and releases the lock correctly, but this is a non-obvious coupling. Replacing the early return with a conditional (unless generated_files_present_and_up_to_date?) removes the reliance on this behaviour. See inline.

Minor — verbose message fires before the lock is acquired
The "Waiting for generated packs lock" message is printed before flock is called, so it appears even when the lock is free and acquired immediately. See inline.

Minor — test relies on wall-clock timing
The concurrent spec uses sleep 0.1 + a non-blocking Queue#pop to assert that the second thread is blocked. This passes on a fast machine but can become flaky under CI load. A queue or condition variable that thread 2 signals when it has entered the lock method would make the assertion deterministic.

Nit — unnecessary safe navigation in ensure
lock_file&.flock(File::LOCK_UN) — the block parameter is never nil, so &. is misleading. See inline.

Looks good

  • Re-checking staleness inside the lock (second caller skips regeneration) is the correct pattern.
  • remove_stale_generated_packs_lock correctly uses LOCK_EX | LOCK_NB so it never deletes a file held by an active process.
  • rescue Errno::ENOENT handles the TOCTOU window between File.exist? and File.open.
  • FileUtils.rm_f + flock(LOCK_UN) in ensure is safe even after deletion (the fd remains valid).
  • The old_auto_load_bundle teardown fix is a good catch.
  • CHANGELOG entry is clear and references both the issue and PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add lock file for creating packs

1 participant