ANTLR grun option (TestRig) does not recognize non-ASCII characters

I am using grun.bat (java org.antlr.v4.runtime.misc.TestRig% *).

Antlr version - 4.5

Running grun.bat with the -tokens option on the input text results in 0 errors, but non-ASCII characters are not recognized, so their token is '????'

Add encoding option to grun.bat file:

java -Dfile.encoding=UTF-8 org.antlr.v4.runtime.misc.TestRig %*

      

results in an error for a non-ASCII character, for example.

line 5: 1 token recognition error in: 'ื'

Non-ASCII characters of the input text are included in utf-8.

How can I properly display non-ASCII characters in tokens?

+3


source to share





All Articles