ANTLR grun option (TestRig) does not recognize non-ASCII characters

I am using grun.bat (java org.antlr.v4.runtime.misc.TestRig% *).

Antlr version - 4.5

Running grun.bat with the -tokens option on the input text results in 0 errors, but non-ASCII characters are not recognized, so their token is '????'

Add encoding option to grun.bat file:

java -Dfile.encoding=UTF-8 org.antlr.v4.runtime.misc.TestRig %*

      

results in an error for a non-ASCII character, for example.

line 5: 1 token recognition error in: 'ื'

Non-ASCII characters of the input text are included in utf-8.

How can I properly display non-ASCII characters in tokens?

+3
utf-8 antlr4


source to share


No one has answered this question yet

Check out similar questions:

287
Detect encoding and do everything UTF-8
12
Running Ascii regex on top of non-ASCII characters with UTF-8
3
Non ascii characters in link_to cause application to crash in Internet Explorer 9 in rails application
2
PHP filter for non-standard characters
2
Getting "incompatible character encodings: UTF-8 and ASCII-8BIT" when displaying a user-entered ASCII-8BIT character
2
Displaying non-ASCII characters in URLs with Safari
2
Antlr4 does not recognize Unicode characters correctly
1
How do I configure the Eclipse Java compiler to give an error for a non-UTF-8 character?
0
Swedish C ++ UTF-8 characters read as ASCII
0
Does Antlr lexer have syntax errors as tokens



All Articles
Loading...
X
Show
Funny
Dev
Pics