| Author |
Quick question on StreamTokenizer
|
|
| overbored 2004-09-20, 9:04 am |
| How do I prevent StreamTokenizer from returning any TT_NUMBER items?
Basically all I want is to have a steady stream of plain TT_WORD items,
even if they're all digits. I tried wordChars('0', '9') but to no avail.
Thanks in advance.
| |
| Boudewijn Dijkstra 2004-09-20, 4:03 pm |
| "overbored" <overboredNO@SPAMoverbored.net> schreef in bericht
news:Xns956A1896DF4Eyangstaoverbored@127
.0.0.1...
> How do I prevent StreamTokenizer from returning any TT_NUMBER items?
> Basically all I want is to have a steady stream of plain TT_WORD items,
> even if they're all digits. I tried wordChars('0', '9') but to no avail.
This will just assign a group of chars to two types. Try calling
resetSyntax() first.
| |
| overbored 2004-09-20, 9:03 pm |
| But won't that clear out *everything*? Then I'd have to rebuild the entire
table, and I would have to do research into locales and character sets and
whatnot.... Is there no alternative?
"Boudewijn Dijkstra" <usenet@bdijkstra.tmfweb.nl> wrote in news:414f23dc$0
$62354$5fc3050@dreader2.news.tiscali.nl:
> "overbored" <overboredNO@SPAMoverbored.net> schreef in bericht
> news:Xns956A1896DF4Eyangstaoverbored@127
.0.0.1...
>
> This will just assign a group of chars to two types. Try calling
> resetSyntax() first.
>
>
>
| |
|
| overbored <overboredNO@SPAMoverbored.net> wrote in message news:<Xns956A1896DF4Eyangstaoverbored@127.0.0.1>...
> How do I prevent StreamTokenizer from returning any TT_NUMBER items?
> Basically all I want is to have a steady stream of plain TT_WORD items,
> even if they're all digits. I tried wordChars('0', '9') but to no avail.
> Thanks in advance.
When you get TT_NUMBER, convert it to a String and concatenate all
strings and numbers into one string
Regards,
Yakov
| |
| Boudewijn Dijkstra 2004-09-21, 8:57 pm |
| "overbored" <overboredNO@SPAMoverbored.net> schreef in bericht
news:Xns956A8F336687Fyangstaoverbored@12
7.0.0.1...
> "Boudewijn Dijkstra" <usenet@bdijkstra.tmfweb.nl> wrote in news:414f23dc$0
> $62354$5fc3050@dreader2.news.tiscali.nl:
>
[color=darkred]
> But won't that clear out *everything*? Then I'd have to rebuild the entire
> table, and I would have to do research into locales and character sets and
> whatnot....
That is what you might expect in Java, but quite the opposite is true:
StreamTokenizer isn't very advanced. It says in the docs: "Each byte read
from the input stream is regarded as a character in the range '\u0000' through
'\u00FF'." So there are only 256 characters to be reckoned for.
> Is there no alternative?
I just read in the docs that the ordinaryChars() method does the same as
resetSyntax(), but only with the specified characters.
Remember this: javadoc is your friend.
|
|
|
|