Home > Archive > PERL Programming > October 2005 > Word frequency analyser
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Word frequency analyser
|
|
|
| Hi,
Does anyone happen to know if there's a convenient module which will analyse
at least two XML files and list the most frequently-used words?
(It would have to be able to reject tags and certain words such as "the" and
"is")
| |
| John Bokma 2005-10-23, 6:56 pm |
| "DVH" <dvh@dvhdvhdvhdvdh.dvh> wrote:
> Hi,
>
> Does anyone happen to know if there's a convenient module which will
> analyse at least two XML files and list the most frequently-used
> words?
>
> (It would have to be able to reject tags
XML::Parser, which a Char handler.
> and certain words such as
> "the" and "is")
split in the handler on non-words, use a hash for counting. Delete
afterwards all occurences of the, is, etc.
Note that this is a very simplistic approach, since it words are hypenated,
it counts them as two different ones.
--
John Small Perl scripts: http://johnbokma.com/perl/
Perl programmer available: http://castleamber.com/
I ploink googlegroups.com :-)
| |
| Scott W Gifford 2005-10-24, 6:56 pm |
| John Bokma <john@castleamber.com> writes:
> "DVH" <dvh@dvhdvhdvhdvdh.dvh> wrote:
>
>
> XML::Parser, which a Char handler.
>
>
> split in the handler on non-words, use a hash for counting. Delete
> afterwards all occurences of the, is, etc.
>
> Note that this is a very simplistic approach, since it words are hypenated,
> it counts them as two different ones.
Seaching for "word frequency" on search.cpan.org turns up some modules
that are designed for this sort of thing, and may take some of the
trickier issues into account.
----Scott.
| |
|
|
|
|
|