Increase file reading efficiency

Hello, I am using the WordNet::QueryData module, "initialization step
is slow (appx. 10-15 seconds), but queries are very fast thereafter---
thousands of queries can be completed every second. " as stated by the
author, because it needs to read very big WordNet lexicon in many TEXT
files. I am new to perl, I like to know if there is a way I could
reduce the initialization time ? Thank you

Keenlearner <yingun@gmail.com> wrote:
> Hello, I am using the WordNet::QueryData module, "initialization step
> is slow (appx. 10-15 seconds), but queries are very fast thereafter---
> thousands of queries can be completed every second. " as stated by the
> author, because it needs to read very big WordNet lexicon in many TEXT
> files. I am new to perl, I like to know if there is a way I could
> reduce the initialization time ? Thank you

On my system, it only takes about 2 seconds.  So the easiest way would
be to get modern hardware :)  Or you could try to rearrange things so you
need to start up the script less and use it more for each time it is
started (for example, use mod_perl instead of CGI).

Another option would be to store the database in a form that can be
reloaded much easier, which is what the Storable module does.

This creates a serialized object in a file named "file":

$ perl
use WordNet::QueryData;
my $x= WordNet::QueryData->new();
delete $x->{data_fh};
use Storable;
store $x, "file";
__END__

The delete $x->{data_fh}; is there because Storable can't store open
file handles.

This script would need to be run once each time WordNet
database is updated.

Then, when you go to use the database, you would no longer load
the object with WordNet::QueryData->new(), instead would load it from
the saved data:

use WordNet::QueryData;
use Storable;
$x=retrieve "file";
$x->openData;
# now use $x just as if it was created via new
__END__

The purpose of $x->openData is to repopulate the previously deleted
$x->{data_fh}.  I haven't tested this rigorously.

This cuts the start-up time to 0.5 seconds, so only a 4 fold improvement.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.

On Mar 20, 2:37 am, xhos...@gmail.com wrote:
> Keenlearner <yin...@gmail.com> wrote: 
>
> On my system, it only takes about 2 seconds.  So the easiest way would
> be to get modern hardware :)  Or you could try to rearrange things so you
> need to start up the script less and use it more for each time it is
> started (for example, use mod_perl instead of CGI).
>
> Another option would be to store the database in a form that can be
> reloaded much easier, which is what the Storable module does.
>
> This creates a serialized object in a file named "file":
>
> $ perl
> use WordNet::QueryData;
> my $x= WordNet::QueryData->new();
> delete $x->{data_fh};
> use Storable;
> store $x, "file";
> __END__
>
> The delete $x->{data_fh}; is there because Storable can't store open
> file handles.
>
> This script would need to be run once each time WordNet
> database is updated.
>
> Then, when you go to use the database, you would no longer load
> the object with WordNet::QueryData->new(), instead would load it from
> the saved data:
>
> use WordNet::QueryData;
> use Storable;
> $x=retrieve "file";
> $x->openData;
> # now use $x just as if it was created via new
> __END__
>
> The purpose of $x->openData is to repopulate the previously deleted
> $x->{data_fh}.  I haven't tested this rigorously.
>
> This cuts the start-up time to 0.5 seconds, so only a 4 fold improvement.
>
> Xho
>
> --
> --------------------http://NewsReader.Com/--------------------
> The costs of publication of this article were defrayed in part by the
> payment of page charges. This article must therefore be hereby marked
> advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
> this fact.

Thank you very much  Xho, I had read about Storable which increase the
reading 4 times faster than normal text file.
I will try that out as only Storable is possible for me under my web
share hosting.

Thank you