Skip to content

fix: json decoder fixes for utf-16 corner cases#1170

Merged
jaromil merged 9 commits intomasterfrom
fix/json-decode
Feb 16, 2026
Merged

fix: json decoder fixes for utf-16 corner cases#1170
jaromil merged 9 commits intomasterfrom
fix/json-decode

Conversation

@jaromil
Copy link
Copy Markdown
Member

@jaromil jaromil commented Feb 14, 2026

  • Escape handling kept self-contained and compatible with existing escape_char_map_inv usage in parse_string (zenroom-json.lua:265).
    • Added strict surrogate-pair validation:
      • valid pair required for high surrogates
      • lone low/high surrogates rejected
      • implemented via parse_surrogate_unicode_escape + guarded state in parse_string (zenroom-json.lua:249, zenroom-json.lua:286).
    • Optimized hot decode paths with byte-based scanning:
      • create_byte_set + byte sets for space/delim/escape (zenroom-json.lua:186)
      • next_char, decode_error newline scan (zenroom-json.lua:207, zenroom-json.lua:218)
      • array/object token checks and parser dispatch map switched to byte keys (zenroom-json.lua:359, zenroom-json.lua:386, zenroom-
        json.lua:423).

@jaromil
Copy link
Copy Markdown
Member Author

jaromil commented Feb 15, 2026

Asking your review here @matteo-cristino especially about this potential regression:
[fix(json): reject duplicate keys in objects

two tests needed mending because of it, so maybe any code in production does? DID or sd-jwt related.

the fix is motivated by a security issue: duplicating keys may confuse a reviewer, hiding bad values in input data.

@matteo-cristino
Copy link
Copy Markdown
Collaborator

I think it was just a typo in the tests, all keys must always be unique

@jaromil jaromil merged commit 4961db3 into master Feb 16, 2026
33 checks passed
@jaromil jaromil deleted the fix/json-decode branch February 16, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants