Home > Archive > Prolog > January 2008 > Building token lists in SWI
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Building token lists in SWI
|
|
| Snapageno 2008-01-16, 7:11 pm |
| Hi, excuse me for not spending hours searching for this, but I want to use
DCG's on real files, and I would like to efficiently build a list of tokens
to analyse with the DCG. What do you recommend? Writing a C program??
Henri
| |
| Jan Wielemaker 2008-01-17, 4:24 am |
| On 2008-01-16, Snapageno <snapageno@free.fr> wrote:
> Hi, excuse me for not spending hours searching for this, but I want to use
> DCG's on real files, and I would like to efficiently build a list of tokens
> to analyse with the DCG. What do you recommend? Writing a C program??
As tokens generally don't spread lines, read the file using
read_line_to_codes/3 and use DCG rules for creating the tokens. That
will probably be very reasonable. If you really want the best
performance you may consider writing the tokenizer in C. Certainly,
generally most of the parsing time is spent in the tokeniser.
Cheers --- Jan
|
|
|
|
|