feat(ghciwatch): add new experimental progress bar option to report compilation output on 1 line by kallevmercury · Pull Request #380 · MercuryTechnologies/ghciwatch

kallevmercury · 2026-03-09T15:21:55Z

Proposal implementation for: #21

Labeled the PR with patch, minor, or major to request a version bump when it's merged.
Updated the user manual in docs/.
Added integration / regression tests in tests/.

I've been testing this one locally by pointing the local compilation to MWB via GHCIWATCH_BIN.

…tput on 1 line

9999years

Hi! I think that getting this over the line as the default would be very challenging (see below), but I would be willing to merge this as an experimental feature (to help clarify to users that it's unsupported). If you'd like to put the work in, here's what I'd like to do:

First, write a PR which adds a new --experimental-features flag; we could use the existing --tui flag as a proof-of-concept here.
Next, modify this PR to require e.g. --experimental-features progress.
Replace the regex in ProgressWriter with the parser in ghci/parse/ghc_message/compiling.rs. (Is there a reason you didn't use the parser in the first place? I couldn't figure it out.)

Then I think we could merge this. If you'd like, feel free to put ~30mins on my calendar and we can go over details.

Making this the new default behavior or shipping it as a non-experimental feature would require (I think) 1-2 months of work, and we just don't have the bandwidth for that right now. I know that's frustrating (it's frustrating for us as well) but we've been burned before and as a result we have a very high bar for changes to ghciwatch.

Some details, so that hopefully you can understand where we're coming from:

We're not set up to test output rendering directly like this. I had a plan for implementing a test framework for this (#202), but I believe implementing it correctly and with good UX would take 2-4 weeks on its own.
- As a result, we have essentially zero tests for this feature. Testing is extremely important for ghciwatch; about a third of the code in the repo is test code, and a feature like this which stream-edits GHC output has a high blast radius and a high bar for testing.
- Because we're modifying the way GHC output is rendering, we need to be very confident that all GHC output renders correctly; this is very difficult, especially because GHCi sessions have user code and GHC itself share the same stdout handle. Some examples of ways this gets tricky:
  - It is possible for users to spawn threads which write to stdout interleaved with compilation messages, and we do this in practice to run the backend server in the REPL.
  - Similarly, error messages and compilation output can be interleaved. The tests sort of hint at this but don't feature (e.g.) an error message between two compilation messages.
  - Another big source of complexity is that users often want to write and flush output without writing a newline (for example our test suite in CI which prints a single . and no newline for each test case); I believe that this mechanism would break use-cases like this without any indication to the user that output is being hidden.
Ghciwatch runs on everyone's computers, under dozens of setups. The variance is pretty huge, and when we ship new features we're consistently surprised by issues we weren't able to foresee or catch ahead of time. I expect that shipping this feature would add 1-2 weeks of reactive work (where we receive an unexpected issue report and have to drop our tasks to fix it immediately). Reactive work like this tends to be spread over time, so there's a good chance we will need to fix these bug reports when we're no longer feeling very familiar with the code.
- As a result of this, we're simply afraid of shipping changes like this to ghciwatch — they tend to incur significant cost that's difficult for us to bear. It's very frustrating for us when our users experience mysteriously unreliable software, so with a limited budget for maintenance, we've mostly settled on not shipping changes at all. I know this is frustrating to hear as a contributor as well, and I wish we could push this over the line quickly.

kallevmercury · 2026-03-10T20:33:10Z

Hi! I think that getting this over the line as the default would be very challenging (see below), but I would be willing to merge this as an experimental feature (to help clarify to users that it's unsupported). If you'd like to put the work in, here's what I'd like to do:

First, write a PR which adds a new --experimental-features flag; we could use the existing --tui flag as a proof-of-concept here.

Next, modify this PR to require e.g. --experimental-features progress.

Replace the regex in ProgressWriter with the parser in ghci/parse/ghc_message/compiling.rs. (Is there a reason you didn't use the parser in the first place? I couldn't figure it out.)

Then I think we could merge this. If you'd like, feel free to put ~30mins on my calendar and we can go over details.

Making this the new default behavior or shipping it as a non-experimental feature would require (I think) 1-2 months of work, and we just don't have the bandwidth for that right now. I know that's frustrating (it's frustrating for us as well) but we've been burned before and as a result we have a very high bar for changes to ghciwatch.

Some details, so that hopefully you can understand where we're coming from:

We're not set up to test output rendering directly like this. I had a plan for implementing a test framework for this ([DUX-1985] Test the TUI #202), but I believe implementing it correctly and with good UX would take 2-4 weeks on its own.

As a result, we have essentially zero tests for this feature. Testing is extremely important for ghciwatch; about a third of the code in the repo is test code, and a feature like this which stream-edits GHC output has a high blast radius and a high bar for testing.

Because we're modifying the way GHC output is rendering, we need to be very confident that all GHC output renders correctly; this is very difficult, especially because GHCi sessions have user code and GHC itself share the same stdout handle. Some examples of ways this gets tricky:

It is possible for users to spawn threads which write to stdout interleaved with compilation messages, and we do this in practice to run the backend server in the REPL.

Similarly, error messages and compilation output can be interleaved. The tests sort of hint at this but don't feature (e.g.) an error message between two compilation messages.

Another big source of complexity is that users often want to write and flush output without writing a newline (for example our test suite in CI which prints a single . and no newline for each test case); I believe that this mechanism would break use-cases like this without any indication to the user that output is being hidden.

Ghciwatch runs on everyone's computers, under dozens of setups. The variance is pretty huge, and when we ship new features we're consistently surprised by issues we weren't able to foresee or catch ahead of time. I expect that shipping this feature would add 1-2 weeks of reactive work (where we receive an unexpected issue report and have to drop our tasks to fix it immediately). Reactive work like this tends to be spread over time, so there's a good chance we will need to fix these bug reports when we're no longer feeling very familiar with the code.

As a result of this, we're simply afraid of shipping changes like this to ghciwatch — they tend to incur significant cost that's difficult for us to bear. It's very frustrating for us when our users experience mysteriously unreliable software, so with a limited budget for maintenance, we've mostly settled on not shipping changes at all. I know this is frustrating to hear as a contributor as well, and I wish we could push this over the line quickly.

@9999years Thank you for the awesome PR review and for sharing so much context! I started on the first step to add a new experimental-features flag: #385

In `[N of M] Compiling ...` messages, parse `N` and `M` out for further inspection. Split off of #380

9999years

thanks for your work on this! I should be able to review this (early?) tomorrow

9999years

Left some small comments and a few questions, but I think this is almost ready to go!

kallevmercury · 2026-03-11T21:46:30Z

😢

9999years

lgtm, thanks for this! we may have to force merge because i don't want to write a ci pipeline

9999years · 2026-03-11T23:05:58Z

+use crate::ghci::parse::compiling;
+use crate::ghci::parse::CompilingProgress;
+
+/// Equivalent to `s[..s.floor_char_boundary(max_cols)]` (stable in Rust 1.82).


Oh, that's nice. We could potentially update the toolchain in this repo, just haven't bothered. We actually have a fairly recent Rust toolchain in mwb.

9999years · 2026-03-11T23:13:34Z

CI moment I'm afraid. nix flake check passed on my mac and Linux boxes, so I think we can merge this whenever.

feat(ghciwatch): add new progress bar option to report compilation ou…

9bf337c

…tput on 1 line

kallevmercury added the minor New functionality, non-breaking label Mar 9, 2026

kallevmercury added 3 commits March 9, 2026 11:31

style: clippy

a7b256e

chore: formatting

f6f80d0

docs: missing doc update

b238171

kallevmercury marked this pull request as ready for review March 9, 2026 18:47

kallevmercury requested a review from 9999years March 9, 2026 18:48

9999years requested changes Mar 10, 2026

View reviewed changes

refactor: consolidate to winnow parser

c9aae53

kallevmercury requested a review from a team as a code owner March 10, 2026 19:25

kallevmercury requested review from csamak and dtpowl March 10, 2026 19:25

kallevmercury mentioned this pull request Mar 10, 2026

fix(flags): add a new experimental-features flag to gate new features starting with the existing "tui" flag #385

Merged

3 tasks

9999years reviewed Mar 10, 2026

View reviewed changes

Comment thread docs/cli.md

Merge branch 'main' into kallev/progress-bar

f6703f6

9999years pushed a commit that referenced this pull request Mar 10, 2026

compilation_log: Parse current, total module numbers

99c81e6

In `[N of M] Compiling ...` messages, parse `N` and `M` out for further inspection. Split off of #380

9999years mentioned this pull request Mar 10, 2026

compilation_log: Parse current, total module numbers #387

Merged

9999years pushed a commit that referenced this pull request Mar 10, 2026

compilation_log: Parse current, total module numbers

b1b95e5

In `[N of M] Compiling ...` messages, parse `N` and `M` out for further inspection. Split off of #380

kallevmercury added 5 commits March 10, 2026 19:43

refactor: put new progress bar under experimental-features

c4c1733

chore: add warning if both tui and progress are enabled at the same time

4735602

test: for interleaved error and compilation output

c843a6e

test: better name

addc622

refactor: output mode

336248d

kallevmercury changed the title ~~feat(ghciwatch): add new progress bar option to report compilation output on 1 line~~ feat(ghciwatch): add new experimental progress bar option to report compilation output on 1 line Mar 11, 2026

9999years reviewed Mar 11, 2026

View reviewed changes

Comment thread src/ghci/parse/ghc_message/compiling.rs

Merge branch 'main' into kallev/progress-bar

fe3d644

9999years requested changes Mar 11, 2026

View reviewed changes

kallevmercury added 2 commits March 11, 2026 14:45

refactor: use pretty assertions and crossterm

a4fe25b

perf: vec -> dequeue

f50e807

kallevmercury added 5 commits March 11, 2026 15:14

refactor: resolve potential panic for non-ASCII characters

1d1b55e

doc: improve comment

bdf3f63

refactor: debug assertion

6b58a29

refactor: try switching out the sync IO in async context

02220d8

fix: add recompilation reason

c696322

kallevmercury added 2 commits March 11, 2026 17:54

refactor: simplify

da98f66

chore: fmt

73c0867

kallevmercury requested a review from 9999years March 11, 2026 22:06

9999years approved these changes Mar 11, 2026

View reviewed changes

kallevmercury enabled auto-merge (squash) March 11, 2026 23:15

9999years disabled auto-merge March 11, 2026 23:23

9999years merged commit a08fa77 into main Mar 11, 2026
0 of 2 checks passed

9999years deleted the kallev/progress-bar branch March 11, 2026 23:23

kallevmercury mentioned this pull request Mar 12, 2026

chore(deps): remove some unused dependencies #391

Merged

3 tasks

Conversation

kallevmercury commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

9999years left a comment

Choose a reason for hiding this comment

Uh oh!

kallevmercury commented Mar 10, 2026

Uh oh!

Uh oh!

9999years left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

9999years left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kallevmercury commented Mar 11, 2026

Uh oh!

9999years left a comment

Choose a reason for hiding this comment

Uh oh!

9999years Mar 11, 2026

Choose a reason for hiding this comment

Uh oh!

9999years commented Mar 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kallevmercury commented Mar 9, 2026 •

edited

Loading