feat: add pretty run report by kaeun97 · Pull Request #416 · egraphs-good/egglog-python

kaeun97 · 2026-05-05T23:07:41Z

Resolves #398.

Here is an example code:

from __future__ import annotations
from egglog import *

egraph = EGraph()

class Num(Expr):
    def __init__(self, n: i64Like) -> None: ...
    def __add__(self, other: Num) -> Num: ...
    def __mul__(self, other: Num) -> Num: ...

x, y = vars_("x y", Num)
egraph.register(rewrite(x + y).to(y + x))
egraph.register(Num(1) + Num(2))
report = egraph.run(10)
print(report)

Output before:

RunReport { iterations: [IterationReport { rule_set_report: RuleSetReport { changed: true, rule_reports: {"(rewrite (__main___Num___add__ _x _y) (__main___Num___add__ _y _x))": [RuleReport { plan: None, search_and_apply_time: 2.625µs, num_matches: 1 }]}, search_and_apply_time: 5.375µs, merge_time: 583ns }, rebuild_time: 1.125µs }, IterationReport { rule_set_report: RuleSetReport { changed: false, rule_reports: {"(rewrite (__main___Num___add__ _x _y) (__main___Num___add__ _y _x))": [RuleReport { plan: None, search_and_apply_time: 1.125µs, num_matches: 1 }]}, search_and_apply_time: 2.75µs, merge_time: 1.041µs }, rebuild_time: 0ns }], updated: true, search_and_apply_time_per_rule: {"(rewrite (__main___Num___add__ _x _y) (__main___Num___add__ _y _x))": 3.75µs}, num_matches_per_rule: {"(rewrite (__main___Num___add__ _x _y) (__main___Num___add__ _y _x))": 2}, search_and_apply_time_per_ruleset: {"": 8.125µs}, merge_time_per_ruleset: {"": 1.624µs}, rebuild_time_per_ruleset: {"": 1.125µs} }

Output after:

PrettyRunReport(iterations=[PrettyIterationReport(rule_set_report=PrettyRuleSetReport(changed=True, rule_reports={'rewrite(x + y).to(y + x)': [PrettyRuleReport(plan=None, search_and_apply_time=datetime.timedelta(0), num_matches=1)]}, search_and_apply_time=datetime.timedelta(0), merge_time=datetime.timedelta(0)), rebuild_time=datetime.timedelta(0)), PrettyIterationReport(rule_set_report=PrettyRuleSetReport(changed=False, rule_reports={'rewrite(x + y).to(y + x)': [PrettyRuleReport(plan=None, search_and_apply_time=datetime.timedelta(0), num_matches=1)]}, search_and_apply_time=datetime.timedelta(0), merge_time=datetime.timedelta(0)), rebuild_time=datetime.timedelta(0))], updated=True, search_and_apply_time_per_rule={'rewrite(x + y).to(y + x)': datetime.timedelta(0)}, num_matches_per_rule={'rewrite(x + y).to(y + x)': 2}, search_and_apply_time_per_ruleset={'': datetime.timedelta(0)}, merge_time_per_ruleset={'': datetime.timedelta(0)}, rebuild_time_per_ruleset={'': datetime.timedelta(0)})

saulshanabrook

Thank you for this! Added a few comments. Could you also add this to the changelog file with a link to this PR?

codspeed-hq · 2026-05-07T01:49:04Z

Merging this PR will improve performance by 67.74%

⚡ 1 improved benchmark
✅ 5 untouched benchmarks
⏩ 8 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	Simulation	`test_jit[lda]`	11.6 s	6.9 s	+67.74%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing kaeun97:kaeun97/pretty-report (01802ec) with main (8812ec9)}

8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

saulshanabrook

Thanks for the fixes, I left a few small comments. There are also some mypy and formatting issues I think.

There is a bigger question about performance, if the codspeed is correct it looks like this slows things down by a ton!

Taking almost 40% of the time in a bigger benchmark just to translate bindings.

It makes me wonder about a different approach, where we set each rewrite and rule with a manual name like 1, 2, 3, ... and then we don't have to do the name searching and mangling and can just parse the name as an int then look it up? And if it's a birewrite just take off the <= or >=?

It would make the egglog file a bit more verbose, but makes parsing the reports more straightforward and more performant which seems like a good tradeoff?

I was also going back and forth on whether the RunReport should store a RewriteOrRule or the decl? If we just store the RewriteOrRule it's easier to pretty print, can just use the builtin one, and it's easier for users to grab that off and compare it or use it... But most of the other exposed objects just store the decls, so I will leave it up to you!

EDIT: It looks like the docs failures also highlight some other exceptions from this. I imagine also if we name the rules here that might also help since it seems like it's hitting on looking up the string?

kaeun97 · 2026-05-07T18:21:32Z

@saulshanabrook Thanks for the thorough review! I do agree that the performance looks concerning. The numeric name approach you mentioned would work for bindings with a "name" field - so not for, RewriteDecl nor BiRewriteDecl. That would require the rust side change. Happy to prioritize that before continuing on with this PR. Also, we can do lazy loading (translate when the user needs it) to have minimal impact to performance.

saulshanabrook · 2026-05-07T23:05:09Z

The numeric name approach you mentioned would work for bindings with a "name" field - so not for, RewriteDecl nor BiRewriteDecl. That would require the rust side change. Happy to prioritize that before continuing on with this PR.

Ah yeah I kept forgetting about this! I just talked to some other folks on the egglog team and they said that sounds like a great feature to add, just something we hadn't gotten around to yet. It should also I think be relatively straightforward so a good first PR to egglog core if you don't mind doing that...

Then once that is merged hopefully should just be able to update the pin here and can use that feature. I believe the version of egglog we depend on here is pretty recent, so hopefully won't be other changes we have to adapt to.

kaeun97 · 2026-05-20T01:23:45Z

Hey @saulshanabrook , thank you again for your feedback. The benchmark seems much better now. Let me know how it looks!

saulshanabrook

Thanks again for your continued updates on this!

I have some additional cleanup feedback, to try and keep the data structures a bit more minimal and specific, raise any errors earlier, and make sure bi-rewrite preserves both times.

…ar type

kaeun97 · 2026-05-22T17:15:54Z

@saulshanabrook , thank you for the clean up suggestion. It does indeed made it much nicer to read and use! Let me know how this looks!

saulshanabrook

This looks good! I just need to figure out why the checks are failing, they seem to all get cancelled and I am not sure why... I dont think its due to this PR?

saulshanabrook · 2026-06-01T13:38:41Z

+def _format_rule_key(decls: Declarations, key: RewriteOrRuleDecl) -> str:
+    return pretty_decl(decls, key)


This could be removed, but could be done in a follow up

saulshanabrook · 2026-06-01T13:40:38Z

+    def _setup_simple_egraph(self):
+        egraph = EGraph()
+
+        class Num(Expr):
+            def __init__(self, n: i64Like) -> None: ...
+            def __add__(self, other: Num) -> Num: ...
+
+        x, y = vars_("x y", Num)
+        egraph.register(rewrite(x + y).to(y + x))
+        egraph.register(Num(1) + Num(2))
+        return egraph
+
+    def test_run_returns_report(self):
+        egraph = self._setup_simple_egraph()


We usually use pytest style functional tests, unless we use classes for grouping, where _setup_simple_egraph would just be a function instead of a method

saulshanabrook · 2026-06-01T13:43:30Z

@coderabbitai full review

coderabbitai · 2026-06-01T13:43:39Z

✅ Actions performed

Full review triggered.

coderabbitai · 2026-06-01T13:50:49Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: b752b7f2-ba6d-419b-91b3-ad55d1e14ab0

📥 Commits

Reviewing files that changed from the base of the PR and between 8812ec9 and 3e45d9c.

⛔ Files ignored due to path filters (1)

Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (8)

Cargo.toml
docs/changelog.md
python/egglog/bindings.pyi
python/egglog/egraph.py
python/egglog/egraph_state.py
python/egglog/run_report.py
python/tests/test_run_report.py
src/conversions.rs

📜 Recent review details

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)

GitHub Check: CodSpeed Macro Runners
GitHub Check: docs
GitHub Check: test 3.13
GitHub Check: benchmark (codspeed-macro)
GitHub Check: test 3.12
GitHub Check: benchmark (ubuntu-latest)
GitHub Check: mypy
GitHub Check: test 3.11

🧰 Additional context used

📓 Path-based instructions (1)

python/egglog/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Prefer relative imports inside python/egglog

Files:

python/egglog/run_report.py
python/egglog/egraph.py
python/egglog/egraph_state.py

🪛 Ruff (0.15.14)

python/tests/test_run_report.py

[error] 6-6: from egglog import * used; unable to detect undefined names

(F403)

[warning] 11-11: Missing return type annotation for private function _setup_simple_egraph

(ANN202)

🔇 Additional comments (20)

Cargo.toml (3)

26-30: LGTM!

53-57: LGTM!

26-30: Confirm 2e5657b exists for both repos and patch override should apply

Cargo.toml (lines 26-30, 53-57): commit 2e5657b exists in both egraphs-good/egglog and saulshanabrook/egg-smol. Cargo canonicalizes git URLs, so the [patch.'https://github.com/egraphs-good/egglog'] override should match the https://github.com/egraphs-good/egglog.git dependencies and apply the egg-smol sources at the same rev = 2e5657b.

docs/changelog.md (1)

9-9: LGTM!

python/egglog/bindings.pyi (1)

406-410: LGTM!

src/conversions.rs (2)

161-163: LGTM!

499-503: LGTM!

python/egglog/egraph_state.py (4)

89-92: LGTM!

110-111: LGTM!

291-310: LGTM!

312-315: LGTM!

python/egglog/run_report.py (5)

14-15: LGTM!

18-31: LGTM!

33-69: LGTM!

71-84: LGTM!

86-143: LGTM!

python/egglog/egraph.py (3)

45-45: LGTM!

Also applies to: 74-74

958-977: LGTM!

979-985: LGTM!

python/tests/test_run_report.py (1)

1-184: LGTM!

📝 Walkthrough

Summary by CodeRabbit

Release Notes

New Features
- Python's RunReport now displays rule identifiers as command declarations instead of raw s-expression strings, improving readability
- Enhanced RunReport string formatting with better output organization
- Rewrite objects now support optional name parameter for better rule identification
- EGraph.stats() now returns structured report objects matching run() output
Documentation
- Updated changelog with RunReport improvements

Walkthrough

This PR implements a Python-friendly wrapper layer for e-graph execution reports. It adds deterministic rule naming to the state, introduces dataclass models that translate low-level rule identifiers to their originating command declarations, and exposes these structured reports through the public EGraph API instead of raw bindings payloads.

Changes

Run Report Wrapper and Integration

Layer / File(s)	Summary
Rewrite name support in types and conversions `python/egglog/bindings.pyi`, `src/conversions.rs`	`Rewrite` type stubs now declare a `name: str` field and `__new__` accepts optional `name` parameter. Python↔Rust conversion for `GenericRewrite` includes and passes through the `name` value. `Command::Function` conversion also updates to include explicit `term_constructor` and `unextractable` fields.
Rule naming infrastructure in EGraphState `python/egglog/egraph_state.py`	`EGraphState` adds `rule_name_counter` and `rule_name_to_command_decl` fields to track and map generated rule names to their originating declarations. `copy()` preserves these fields. `command_to_egg` assigns standardized names to unnamed rules, populates the mapping for both forward and directional bi-rewrite names, and stores the mapping for later lookup.
Run report wrapper dataclasses and translation `python/egglog/run_report.py`	New module introduces `RuleReport`, `RuleSetReport`, `IterationReport`, and `RunReport` dataclasses with `_from_bindings()` constructors that translate rule keys through `state.rule_name_to_command_decl`, accumulate per-rule timing and match metrics, and provide structured `__repr__` formatting without raw s-expression content.
EGraph API integration of RunReport `python/egglog/egraph.py`	`EGraph.run()` and `EGraph.stats()` now return `RunReport` objects instead of raw bindings payloads. Import and export `RunReport`, delegate to `_run_schedule`, and construct `RunReport` via `_from_bindings()` to translate rule keys and expose Python-friendly reporting.
Run report test coverage `python/tests/test_run_report.py`	Comprehensive test suite validates `RunReport` return types, rule key translation to command declarations, timing/match metrics, nested iteration/ruleset/rule structure, string formatting without egglog s-expressions, and coverage of named/unnamed rules and birewrite variants.
Dependency updates and documentation `Cargo.toml`, `docs/changelog.md`	Pin egglog-family crates to egraphs-good/egglog revision `2e5657b` with matching patch configuration. Update changelog to document the Python-friendly `RunReport` wrapper that returns command declarations as rule keys instead of raw s-expression strings and improves `str()` output.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 8.89% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'feat: add pretty run report' directly and clearly describes the main change—adding a pretty-printed run report wrapper for better Python readability.
Description check	✅ Passed	The description is directly related to the changeset, providing before/after output examples and explaining the transformation from raw Rust struct representation to Python-friendly dataclass-like format.
Linked Issues check	✅ Passed	The PR fully addresses issue `#398`'s objectives by implementing a high-level wrapper (RunReport dataclasses) that translates rules into Python rule strings and formats output as Python dataclass instead of Rust struct.
Out of Scope Changes check	✅ Passed	The Cargo.toml dependency update and bindings.pyi Rewrite name field addition directly support the pretty run report feature; all other changes enable or implement the core pretty-reporting functionality with no unrelated scope creep.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

saulshanabrook · 2026-06-01T13:56:22Z

@kaeun97 Could you add eggcc-2mm to the SKIP_TESTS:

egglog-python/python/tests/test_bindings.py

Line 38 in 8812ec9

SKIP_TESTS = {

I think the CI is getting stuck on it somehow.

You could also just add a pytest skip mark to the whole thing here:

egglog-python/python/tests/test_bindings.py

Lines 48 to 56 in 8812ec9

    
           @pytest.mark.parametrize( 
        
               "example_file", 
        
               [ 
        
                   pytest.param(path, id=path.stem, marks=pytest.mark.slow if path.stem in SLOW_TESTS else []) 
        
                   for path in (EGG_SMOL_FOLDER / "tests").glob("*.egg") 
        
                   if path.stem not in SKIP_TESTS 
        
               ], 
        
           ) 
        
           def test_example(example_file: pathlib.Path):

There were new examples that got added in the egglog commits that I think are breaking CI for some reason... They do pass locally.

kaeun97 added 2 commits May 6, 2026 00:04

feat: add pretty run report

2007b1a

feat: add test for pretty run report

86133fa

kaeun97 marked this pull request as ready for review May 5, 2026 23:09

kaeun97 mentioned this pull request May 5, 2026

Pretty Run Report #398

Open

saulshanabrook reviewed May 5, 2026

View reviewed changes

Comment thread python/egglog/egraph.py Outdated

Comment thread python/egglog/egraph_state.py Outdated

Comment thread python/egglog/run_report.py Outdated

Comment thread python/egglog/run_report.py Outdated

Comment thread python/egglog/run_report.py Outdated

kaeun97 added 4 commits May 7, 2026 01:07

chore: rename to runreport

f554bb2

chore: pre-commit

c554d1a

fix: store CommandDecl

810bd95

fix: make methods private

c09c7de

kaeun97 requested a review from saulshanabrook May 7, 2026 01:11

feat: update changelog

3d7bec7

saulshanabrook reviewed May 7, 2026

View reviewed changes

Comment thread python/egglog/egraph_state.py Outdated

Comment thread python/egglog/run_report.py Outdated

Comment thread python/egglog/egraph_state.py Outdated

Comment thread python/egglog/run_report.py Outdated

Comment thread python/tests/test_run_report.py Outdated

saulshanabrook reviewed May 7, 2026

View reviewed changes

Comment thread python/egglog/run_report.py Outdated

This was referenced May 11, 2026

Add numeric name field to RewriteDecl and BiRewriteDecl egraphs-good/egglog#870

Closed

Add name to rewrite and birewrite commands egraphs-good/egglog#871

Merged

kaeun97 added 4 commits May 20, 2026 01:26

chore: use 2e5657b egglog

ff7f688

fix: numberic name approach for better performance

01802ec

chore: better types

8ca9d10

fix: add assertion for specific text in test

e217a3f

kaeun97 requested a review from saulshanabrook May 20, 2026 01:23

saulshanabrook reviewed May 20, 2026

View reviewed changes

Comment thread python/egglog/egraph_state.py

Comment thread python/egglog/egraph_state.py Outdated

Comment thread python/egglog/egraph_state.py Outdated

Comment thread python/egglog/run_report.py Outdated

Comment thread python/egglog/run_report.py Outdated

Comment thread python/egglog/egraph.py

chore: clean up by removing egg_rule_to_command_decl fallback and cle…

3e45d9c

…ar type

kaeun97 requested a review from saulshanabrook May 22, 2026 18:32

saulshanabrook moved this to This Week in Saul Shanabrook May 26, 2026

saulshanabrook added this to Saul Shanabrook May 26, 2026

saulshanabrook approved these changes Jun 1, 2026

View reviewed changes

saulshanabrook moved this from This Week to Waiting for Review in Saul Shanabrook Jun 1, 2026

kaeun97 added 3 commits June 3, 2026 00:19

chore: test_run_report is functional style

7ecc467

chore: remove _format_rule_key

3603683

chore: skip eggcc-2mm

5182add

		def _format_rule_key(decls: Declarations, key: RewriteOrRuleDecl) -> str:
		return pretty_decl(decls, key)

Conversation

kaeun97 commented May 5, 2026

Uh oh!

saulshanabrook left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codspeed-hq Bot commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 67.74%

Performance Changes

Footnotes

Uh oh!

saulshanabrook left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kaeun97 commented May 7, 2026

Uh oh!

saulshanabrook commented May 7, 2026

Uh oh!

kaeun97 commented May 20, 2026

Uh oh!

saulshanabrook left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kaeun97 commented May 22, 2026

Uh oh!

saulshanabrook left a comment

Choose a reason for hiding this comment

Uh oh!

saulshanabrook Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

saulshanabrook Jun 1, 2026

Choose a reason for hiding this comment

Uh oh!

saulshanabrook commented Jun 1, 2026

Uh oh!

coderabbitai Bot commented Jun 1, 2026

Uh oh!

coderabbitai Bot commented Jun 1, 2026

Summary by CodeRabbit

Release Notes

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning)

Uh oh!

saulshanabrook commented Jun 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codspeed-hq Bot commented May 7, 2026 •

edited

Loading

saulshanabrook left a comment •

edited

Loading