Revisit CJS→ESM transform: replace Babel with deno_ast CJS parser

[bartlomieju]

Context

In #3767 we attempted to replace the 960-line Babel CJS→ESM transform with a simpler approach: a content-based heuristic for CJS detection + a lightweight ESM shim (export default module.exports). This was reverted in #3798 because of two fundamental issues.

Problems with the simplified approach

1. CJS detection heuristic is fragile

The heuristic used code.includes("export ") to detect ESM files and skip CJS wrapping. This is fooled by comments — e.g. @opentelemetry/api's CJS entry has:

// TODO: Remove ProxyTracerProvider export in the next major version.

This caused the CJS shim to be skipped entirely, resulting in exports is not defined at runtime.

In #3793 we tried fixing this with a two-layer approach:

1. Check nearest package.json "type" field (Node.js semantics)
2. Fall back to a regex detecting actual ESM statements (export\s*[{*], import\s*[{*"'], etc.)

This fixed detection but revealed problem 2.

2. The shim doesn't support named exports

The CJS shim wraps modules as:

var module = { exports: {} };
var exports = module.exports;
// ... CJS code ...
export default module.exports;

This only provides a default export. Named imports like import { trace } from "@opentelemetry/api" get undefined, causing #3797:

Cannot read properties of undefined (reading 'getTracer')

The old Babel transform properly converted exports.foo = ... and Object.defineProperty(exports, "foo", ...) patterns into ESM named exports. The simplified shim cannot do this without analyzing the CJS code structure.

Proposed solution

Replace the Babel CJS transform with deno_ast's CJS parser (or cjs-module-lexer which Node.js uses internally). This would:

1. Correctly detect CJS vs ESM — no heuristics needed, the parser knows
2. Extract named exports — the parser identifies all exports.xxx and Object.defineProperty(exports, "xxx", ...) patterns
3. Be much smaller than the Babel transform — purpose-built for this exact task
4. Match Node.js behavior — same approach Node.js uses for import { foo } from "cjs-package" interop

What didn't work

Approach	Detection	Named exports	Notes
code.includes("export ")	Fooled by comments	N/A	Original #3767 approach
package.json type + ESM regex	Works for formatted & minified	No	#3793 approach
Babel AST transform	Works	Works	Current — 960 lines, heavyweight

Test coverage gaps found

• The CJS test fixtures (commonjs_mod.js, maxmind.js) were changed to ESM in refactor: remove CJS and env var Babel transforms, let Vite handle natively #3767, so the "CJS tests" weren't testing CJS at all
• No test for named imports from CJS packages (import { foo } vs import foo)
• No test using a real CJS npm package from node_modules (fixtures were local files that bypass the node_modules detection path)
• The monorepo's Deno loader resolves @opentelemetry/api to its ESM entry, masking the CJS issue in CI

[jeveloper]

Props to you working on this :)