Alternation fails matching the same token on either side

[zoffixznet]

Moved from RT#131092

use Grammar::Tracer;
grammar G {
    token TOP { <first-fail> || <second-succeed> }
    token first-fail { <thing> '?' }
    token second-succeed { <thing> '!' }
    token thing { "foo" }
}
note G.parse("foo!")

#grammar tracer output:

TOP
|  first-fail
|  |  thing
|  |  * MATCH "foo"
|  * FAIL
|  second-succeed
|  |  thing
|  |  * MATCH "" # it actually matches it but it's an empty string...
|  * FAIL
* FAIL
Nil

If you just create an identical second token and use that instead it works:

grammar G {
    token TOP { <first-fail> || <second-succeed> }
    token first-fail { <thing> '?' }
    token second-succeed { <thing2> '!' }
    token thing { "foo" }
    token thing2 { "foo" }
}

note G.parse("foo!")

｢foo!｣
 second-succeed => ｢foo!｣
  thing2 => ｢foo｣