Skip to content

Tee output so we can see it in journald logs#346

Merged
pantherman594 merged 1 commit intomasterfrom
post-startup-output
Mar 25, 2026
Merged

Tee output so we can see it in journald logs#346
pantherman594 merged 1 commit intomasterfrom
post-startup-output

Conversation

@pantherman594
Copy link
Copy Markdown
Contributor

Output is getting redirected to log files, which means they don’t show up in the app logs. We should send it to both places

PHP-138078

@pantherman594 pantherman594 requested review from a team as code owners March 25, 2026 19:21
@pantherman594 pantherman594 merged commit 9bd46a1 into master Mar 25, 2026
27 of 30 checks passed
@pantherman594 pantherman594 deleted the post-startup-output branch March 25, 2026 21:44
june-hua added a commit that referenced this pull request Mar 31, 2026
…proc

This partially reverts commit 9bd46a1.

Root Cause:
1. The exec > >(tee -a "${POST_STARTUP_OUTPUT_FILE}") creates a tee
subprocess that remains alive
2. The background process that waits for Jupyter inherits file
descriptors from tee
3. When the main startup script exits, tee subprocess stays alive
because the background process still has the pipes open
4. google-startup-scripts.service remains stuck in "activating" state
indefinitely, waiting for all child processes
5. google-dataproc-agent.service has
After=google-startup-scripts.service, so it never starts
6. This creates a race condition where the background Jupyter
configuration runs while Dataproc is still writing its config

Symptoms:
- google-startup-scripts.service stuck in "activating" for hours
  - google-dataproc-agent.service never starts (ConditionResult=no)
june-hua added a commit that referenced this pull request Mar 31, 2026
…proc (#354)

This partially reverts commit 9bd46a1.

Root Cause:
1. The exec > >(tee -a "${POST_STARTUP_OUTPUT_FILE}") creates a tee
subprocess that remains alive
2. The background process that waits for Jupyter inherits file
descriptors from tee
3. When the main startup script exits, tee subprocess stays alive
because the background process still has the pipes open
4. google-startup-scripts.service remains stuck in "activating" state
indefinitely, waiting for all child processes
5. google-dataproc-agent.service has
After=google-startup-scripts.service, so it never starts
6. This creates a race condition where the background Jupyter
configuration runs while Dataproc is still writing its config

Symptoms:
- google-startup-scripts.service stuck in "activating" for hours
  - google-dataproc-agent.service never starts (ConditionResult=no)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants