Skip to content

fix: allow tracing custom operators in SymbolicUtils conversion#171

Draft
MilesCranmerBot wants to merge 3 commits intoSymbolicML:masterfrom
MilesCranmerBot:bot/issue-170-supported-ops
Draft

fix: allow tracing custom operators in SymbolicUtils conversion#171
MilesCranmerBot wants to merge 3 commits intoSymbolicML:masterfrom
MilesCranmerBot:bot/issue-170-supported-ops

Conversation

@MilesCranmerBot
Copy link
Copy Markdown
Contributor

@MilesCranmerBot MilesCranmerBot commented Mar 13, 2026

Fixes #170

Summary

  • Allow non-whitelisted operators to be traced when index_functions=false by calling the operator on symbolic children
  • Keep index_functions=true behavior unchanged (operator indexing for round-trip preservation)
  • If tracing fails with a MethodError, still raise a clear unsupported-operation error

Tests

  • Updated SymbolicUtils extension tests to cover:
    • traceable custom operators with index_functions=false
    • preserved custom-operator round-tripping with index_functions=true
  • Ran full package test suite locally (Pkg.test()), which passed

When index_functions=false, try tracing non-whitelisted operators by calling them on symbolic children; only error on MethodError.\n\nAlso update SymbolicUtils tests to cover both traced custom operators and indexed round-tripping paths.\n\nCo-authored-by: Miles Cranmer <[email protected]>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 13, 2026

Benchmark Results (Julia v1)

Time benchmarks
master 9558210... master / 9558210...
eval/ComplexF32/evaluation 6.9 ± 0.45 ms 6.93 ± 0.5 ms 0.995 ± 0.098
eval/ComplexF64/evaluation 10.2 ± 0.92 ms 10.2 ± 0.76 ms 1 ± 0.12
eval/Float32/derivative 11.3 ± 1.4 ms 10.8 ± 0.97 ms 1.05 ± 0.16
eval/Float32/derivative_turbo 11.1 ± 1.1 ms 10.7 ± 0.98 ms 1.04 ± 0.14
eval/Float32/evaluation 2.45 ± 0.25 ms 2.45 ± 0.24 ms 0.999 ± 0.14
eval/Float32/evaluation_bumper 0.597 ± 0.016 ms 0.596 ± 0.017 ms 1 ± 0.038
eval/Float32/evaluation_turbo 0.514 ± 0.03 ms 0.507 ± 0.03 ms 1.02 ± 0.083
eval/Float32/evaluation_turbo_bumper 0.598 ± 0.016 ms 2.8 ± 0.19 ms 0.214 ± 0.016
eval/Float64/derivative 14 ± 2.6 ms 14.1 ± 2.4 ms 0.999 ± 0.25
eval/Float64/derivative_turbo 14.2 ± 2.9 ms 13.8 ± 1.6 ms 1.03 ± 0.24
eval/Float64/evaluation 2.91 ± 0.3 ms 2.84 ± 0.31 ms 1.03 ± 0.15
eval/Float64/evaluation_bumper 1.26 ± 0.045 ms 1.26 ± 0.045 ms 0.997 ± 0.05
eval/Float64/evaluation_turbo 1.01 ± 0.064 ms 0.998 ± 0.061 ms 1.02 ± 0.09
eval/Float64/evaluation_turbo_bumper 1.25 ± 0.045 ms 3.29 ± 0.25 ms 0.381 ± 0.032
utils/combine_operators/break_sharing 0.0421 ± 0.001 ms 0.0424 ± 0.00085 ms 0.993 ± 0.032
utils/convert/break_sharing 27.7 ± 3 μs 27.3 ± 3.1 μs 1.02 ± 0.16
utils/convert/preserve_sharing 0.1 ± 0.0081 ms 0.0983 ± 0.0075 ms 1.02 ± 0.11
utils/copy/break_sharing 28.3 ± 3.1 μs 26.7 ± 2.4 μs 1.06 ± 0.15
utils/copy/preserve_sharing 0.1 ± 0.0084 ms 0.098 ± 0.0067 ms 1.02 ± 0.11
utils/count_constant_nodes/break_sharing 13 ± 0.85 μs 13 ± 1 μs 1.01 ± 0.1
utils/count_constant_nodes/preserve_sharing 0.0855 ± 0.0059 ms 0.0842 ± 0.0051 ms 1.02 ± 0.093
utils/count_depth/break_sharing 13.6 ± 1.1 μs 13.6 ± 0.93 μs 1 ± 0.11
utils/count_nodes/break_sharing 12.3 ± 0.75 μs 12.1 ± 0.66 μs 1.01 ± 0.083
utils/count_nodes/preserve_sharing 0.0882 ± 0.0057 ms 0.0848 ± 0.0052 ms 1.04 ± 0.093
utils/get_set_constants!/break_sharing 0.0338 ± 0.0036 ms 0.0328 ± 0.0042 ms 1.03 ± 0.17
utils/get_set_constants!/preserve_sharing 0.172 ± 0.0097 ms 0.172 ± 0.01 ms 0.998 ± 0.081
utils/get_set_constants_parametric 0.0472 ± 0.0059 ms 0.0454 ± 0.0054 ms 1.04 ± 0.18
utils/has_constants/break_sharing 7.87 ± 1 μs 7.62 ± 0.85 μs 1.03 ± 0.18
utils/has_operators/break_sharing 2.67 ± 0.19 μs 2.58 ± 0.13 μs 1.03 ± 0.09
utils/hash/break_sharing 24.6 ± 1.9 μs 23.4 ± 1.3 μs 1.05 ± 0.097
utils/hash/preserve_sharing 0.1 ± 0.0062 ms 0.101 ± 0.0066 ms 0.993 ± 0.09
utils/index_constant_nodes/break_sharing 30.4 ± 1.9 μs 28.4 ± 1.8 μs 1.07 ± 0.093
utils/index_constant_nodes/preserve_sharing 0.104 ± 0.0073 ms 0.0993 ± 0.0056 ms 1.05 ± 0.094
utils/is_constant/break_sharing 8.06 ± 0.85 μs 8 ± 0.85 μs 1.01 ± 0.15
utils/simplify_tree/break_sharing 31 ± 1.2 μs 0.0318 ± 0.001 ms 0.974 ± 0.048
utils/simplify_tree/preserve_sharing 0.114 ± 0.0064 ms 0.112 ± 0.0047 ms 1.01 ± 0.07
utils/string_tree/break_sharing 0.436 ± 0.018 ms 0.435 ± 0.018 ms 1 ± 0.057
utils/string_tree/preserve_sharing 0.529 ± 0.018 ms 0.527 ± 0.016 ms 1 ± 0.045
time_to_load 0.163 ± 0.0008 s 0.165 ± 0.0011 s 0.988 ± 0.0081
Memory benchmarks
master 9558210... master / 9558210...
eval/ComplexF32/evaluation 0.972 k allocs: 2.49 MB 0.981 k allocs: 2.51 MB 0.991
eval/ComplexF64/evaluation 0.999 k allocs: 5.09 MB 0.987 k allocs: 5.03 MB 1.01
eval/Float32/derivative 4.62 k allocs: 17.4 MB 4.63 k allocs: 17.4 MB 0.997
eval/Float32/derivative_turbo 4.63 k allocs: 17.4 MB 4.66 k allocs: 17.5 MB 0.993
eval/Float32/evaluation 0.987 k allocs: 1.29 MB 0.978 k allocs: 1.27 MB 1.01
eval/Float32/evaluation_bumper 0.303 k allocs: 0.393 MB 0.303 k allocs: 0.393 MB 1
eval/Float32/evaluation_turbo 0.966 k allocs: 1.26 MB 0.978 k allocs: 1.27 MB 0.988
eval/Float32/evaluation_turbo_bumper 0.303 k allocs: 0.393 MB 0.303 k allocs: 0.393 MB 1
eval/Float64/derivative 4.8 k allocs: 0.0351 GB 4.78 k allocs: 0.0349 GB 1
eval/Float64/derivative_turbo 4.78 k allocs: 0.035 GB 4.74 k allocs: 0.0347 GB 1.01
eval/Float64/evaluation 1 k allocs: 2.57 MB 1.01 k allocs: 2.57 MB 0.997
eval/Float64/evaluation_bumper 0.303 k allocs: 0.771 MB 0.303 k allocs: 0.771 MB 1
eval/Float64/evaluation_turbo 1 k allocs: 2.57 MB 0.999 k allocs: 2.56 MB 1
eval/Float64/evaluation_turbo_bumper 0.303 k allocs: 0.771 MB 0.303 k allocs: 0.771 MB 1
utils/combine_operators/break_sharing 4 allocs: 0.953 kB 4 allocs: 0.953 kB 1
utils/convert/break_sharing 2 k allocs: 0.123 MB 2 k allocs: 0.123 MB 1
utils/convert/preserve_sharing 2.4 k allocs: 0.192 MB 2.4 k allocs: 0.192 MB 1
utils/copy/break_sharing 2 k allocs: 0.123 MB 2 k allocs: 0.123 MB 1
utils/copy/preserve_sharing 2.4 k allocs: 0.192 MB 2.4 k allocs: 0.192 MB 1
utils/count_constant_nodes/break_sharing 4 allocs: 0.953 kB 4 allocs: 0.953 kB 1
utils/count_constant_nodes/preserve_sharing 0.404 k allocs: 0.0696 MB 0.404 k allocs: 0.0696 MB 1
utils/count_depth/break_sharing 4 allocs: 0.953 kB 4 allocs: 0.953 kB 1
utils/count_nodes/break_sharing 4 allocs: 0.953 kB 4 allocs: 0.953 kB 1
utils/count_nodes/preserve_sharing 0.404 k allocs: 0.0696 MB 0.404 k allocs: 0.0696 MB 1
utils/get_set_constants!/break_sharing 0.898 k allocs: 25.2 kB 0.898 k allocs: 25.2 kB 1
utils/get_set_constants!/preserve_sharing 1.7 k allocs: 0.138 MB 1.7 k allocs: 0.138 MB 1
utils/get_set_constants_parametric 1.42 k allocs: 0.0663 MB 1.42 k allocs: 0.0663 MB 1
utils/has_constants/break_sharing 4 allocs: 0.203 kB 4 allocs: 0.203 kB 1
utils/has_operators/break_sharing 4 allocs: 0.203 kB 4 allocs: 0.203 kB 1
utils/hash/break_sharing 0.104 k allocs: 2.52 kB 0.104 k allocs: 2.52 kB 1
utils/hash/preserve_sharing 0.504 k allocs: 0.0711 MB 0.504 k allocs: 0.0711 MB 1
utils/index_constant_nodes/break_sharing 2.1 k allocs: 0.094 MB 2.1 k allocs: 0.094 MB 1
utils/index_constant_nodes/preserve_sharing 2.5 k allocs: 0.163 MB 2.5 k allocs: 0.163 MB 1
utils/is_constant/break_sharing 4 allocs: 0.203 kB 4 allocs: 0.203 kB 1
utils/simplify_tree/break_sharing 0.104 k allocs: 2.52 kB 0.104 k allocs: 2.52 kB 1
utils/simplify_tree/preserve_sharing 0.504 k allocs: 0.0711 MB 0.504 k allocs: 0.0711 MB 1
utils/string_tree/break_sharing 11.9 k allocs: 0.999 MB 11.9 k allocs: 0.999 MB 1
utils/string_tree/preserve_sharing 12.3 k allocs: 1.07 MB 12.3 k allocs: 1.07 MB 1
time_to_load 0.145 k allocs: 11 kB 0.145 k allocs: 11 kB 1

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 13, 2026

Codecov Report

❌ Patch coverage is 66.66667% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.46%. Comparing base (29ebdab) to head (9558210).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
ext/DynamicExpressionsLoopVectorizationExt.jl 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #171      +/-   ##
==========================================
+ Coverage   59.43%   59.46%   +0.03%     
==========================================
  Files          30       30              
  Lines        2682     2689       +7     
==========================================
+ Hits         1594     1599       +5     
- Misses       1088     1090       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@MilesCranmer
Copy link
Copy Markdown
Member

@MilesCranmerBot needs fixes for formatting and for code coverage. i.e., make the diff coverage 100%.

Comment on lines +116 to +123
traced = try
op(sym_children...)
catch e
if e isa MethodError
throw(error("Unsupported operation $(op) in SymbolicUtils conversion"))
end
rethrow()
end
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be type unstable? i.e., might need to do

local traced
try
    traced = op...
catch e
    ...
end

Can you check if this is type unstable? Please show the inferred type in each instance (REPL output with Cthulhu.jl)

@MilesCranmerBot

- add regression coverage for unsupported custom-op paths\n- ensure SymbolicUtils compat avoids Julia 1.12 precompile breakage\n\nCo-authored-by: Miles Cranmer <[email protected]>
@MilesCranmerBot
Copy link
Copy Markdown
Contributor Author

MilesCranmerBot commented Mar 13, 2026

Implemented ✅

  • Fixed formatting on the touched test file.
  • Added extra SymbolicUtils conversion tests to cover:
    • traceable custom ops with index_functions=false
    • non-traceable custom ops raising a clear error
    • non-MethodError exceptions being rethrown
  • Bumped SymbolicUtils compat from 4 to 4.1 to avoid the Julia 1.12 downgrade-compat precompile failure seen with 4.0.0.

Pushed to this PR branch in commit ae6c2a6.

I also ran Pkg.test() locally in this branch; it passed.

@MilesCranmer
Copy link
Copy Markdown
Member

@MilesCranmerBot there's still a downgrade compat failure despite assertion otherwise, please fix

@MilesCranmerBot
Copy link
Copy Markdown
Contributor Author

MilesCranmerBot commented Mar 14, 2026

Fixed — the downgrade failure was coming from @turbo @. cumulator_1 = op(cumulators...) in DynamicExpressionsLoopVectorizationExt, which older LoopVectorization (v0.12.151 in downgrade-compat) cannot macro-expand reliably.

I replaced that line with a plain indexed inbounds loop:

  • cumulator_1[j] = op(map(c -> c[j], cumulators)...)

This avoids the problematic @turbo vararg-broadcast expansion path while preserving behavior.

Commit: 9558210a4e5f6ac896084ed0432ead5e11289a49

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Check for supported operators in DynamicExpressionsSymbolicExt is too restrictive?

2 participants