Skip to content

[BUG] Generated SQL lexers fail to compile against dbt-antlr4 1.1+ — Cargo.toml needs =1.0.5 pin #1639

@thejens

Description

@thejens

Hi! While integrating dbt-fusion as a library in a Rust project, Claude and I ran into another build-time issue. Filing this for tracking — the report below was drafted by Claude, but I've sanity-checked the diagnosis and it does look like a real one.


Describe the bug

dbt-fusion's generated SQL lexers (Snowflake, Redshift, BigQuery, Databricks, DuckDB, Trino) are pinned against the API of dbt-antlr4 1.0.5, but the workspace Cargo.toml requires the dependency loosely (dbt-antlr4 = "1.0.5", which is a compatibility range, not an exact pin). Cargo resolves this to the latest compatible version on crates.io — 1.2.1 — and the build then fails with multiple E0425 / E0061 / E0277 errors because dbt-antlr4 1.1+ changed its public API in incompatible ways.

dbt-fusion's own builds aren't affected because the workspace Cargo.lock pins dbt-antlr4 to 1.0.5. Downstream Rust consumers that depend on dbt-fusion crates as git/path dependencies (without the lockfile) see Cargo resolve to 1.2.1 and break.

API changes between 1.0.5 and 1.2.1 that the generated code still depends on:

Generated code call dbt-antlr4 1.0.5 dbt-antlr4 1.2.1
dbt_antlr4::recognizer::check_version("1","0") pub fn check_version(major: &str, minor: &str) in recognizer.rs:15 macro_rules! check_version in lib.rs:136 (function removed)
DFA::new(&_ATN, _ATN.get_decision_state(i), i) (3 args) pub fn new(atn: &'static ATN, atn_start_state: ATNStateRef, decision: i32) (3 args) pub fn new(atn, arena: &'sim bumpalo::Bump, atn_start_state, decision) (4 args, added arena)
LexerATNSimulator::new_lexer_atnsimulator(&_decision_to_DFA, &_shared_context_cache, ...) (3 args) (matching 3-arg signature) takes 2 args

Compile errors:

error[E0425]: cannot find function \`check_version\` in module \`dbt_antlr4::recognizer\`
   --> crates/dbt-sql/dbt-lexer-snowflake/src/generated/snowflake/snowflakelexer.rs:834:33
    |
834 |         dbt_antlr4::recognizer::check_version("1","0");
    |                                 ^^^^^^^^^^^^^ not found in \`dbt_antlr4::recognizer\`

error[E0061]: this function takes 4 arguments but 3 arguments were supplied
   --> crates/dbt-sql/dbt-lexer-snowflake/src/generated/snowflake/snowflakelexer.rs:906:18
    |
906 |         dfa.push(DFA::new(
    |                  ^^^^^^^^

error[E0061]: this function takes 2 arguments but 3 arguments were supplied
   --> crates/dbt-sql/dbt-lexer-snowflake/src/generated/snowflake/snowflakelexer.rs:892:9
    |
892 |         LexerATNSimulator::new_lexer_atnsimulator(
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

These are reproduced for every dialect with a generated lexer (dbt-lexer-snowflake, dbt-lexer-redshift, dbt-lexer-bigquery, dbt-lexer-databricks, dbt-lexer-duckdb, dbt-lexer-trino).

What version of dbt Fusion is this bug in?

2.0.0-preview.174 (git rev 00969f911ae9a43e008a16a65c9208ebdf9c041a). The same issue exists on every published version since dbt-antlr4 1.1.0 was released to crates.io.

Is this a discrepancy between the dbt Fusion Engine and dbt Core?

  • YES
  • NO (this is a Rust dependency-resolution issue — not relevant to dbt Core)

To Reproduce

Any Rust project that depends on a dbt-fusion crate without inheriting dbt-fusion's Cargo.lock will reproduce the build failures. A minimal Cargo.toml:

[package]
name = "dbt-antlr4-mismatch"
version = "0.1.0"
edition = "2024"

[dependencies]
dbt-loader  = { git = "https://github.com/dbt-labs/dbt-fusion", rev = "00969f911ae9a43e008a16a65c9208ebdf9c041a" }
dbt-parser  = { git = "https://github.com/dbt-labs/dbt-fusion", rev = "00969f911ae9a43e008a16a65c9208ebdf9c041a" }
dbt-adapter = { git = "https://github.com/dbt-labs/dbt-fusion", rev = "00969f911ae9a43e008a16a65c9208ebdf9c041a" }
# ... plus the [patch.crates-io] entries dbt-fusion's own Cargo.toml uses
# (forked arrow-rs, ring) so the rest of the workspace builds.

cargo build resolves dbt-antlr4 = "1.0.5" to the latest compatible 1.2.1 and fails to compile the generated lexers with the errors quoted above.

Workaround (in the consuming project's Cargo.toml): add an exact-version pin to override resolution.

[dependencies]
dbt-antlr4 = "=1.0.5"

This forces Cargo to resolve to exactly 1.0.5, matching what the generated code expects.

Expected behavior

Downstream consumers of dbt-fusion crates should not have to know about the dbt-antlr4 version coupling. The build should succeed against whatever version Cargo resolves.

Suggested fix

Two options, in order of preference:

  1. Pin exactly in Cargo.toml. Change every dbt-antlr4 = "1.0.5" reference in dbt-fusion's workspace to dbt-antlr4 = "=1.0.5". This propagates the constraint to downstream consumers via the Cargo.toml itself (not just the lockfile). Smallest possible change; preserves current behavior.

  2. Regenerate lexers against current dbt-antlr4. Re-run the antlr4 codegen tool with a newer version that targets dbt-antlr4 1.2.x (or whatever the current public API is), and bump the dependency accordingly. Lets dbt-fusion track the latest runtime crate. Larger change; affects the codegen toolchain.

Either fix unblocks downstream consumers.

Operating System and CPU Type

  • macOS (Darwin 25.4.0)
  • ARM (Apple Silicon)

The bug is platform-independent — it's purely a Cargo-resolution issue.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions