| Tom Phoenix 2006-01-10, 4:02 am |
| On 1/6/06, Sai Tong <stong@fidelio.rutgers.edu> wrote:
> The problem is this code runs too slowly for a large array of objects.
> Its seems that it might be a little too slow to "push" each
> returned value into @return_values . Can anyone suggest the best way to
> improve the speed of putting the returned value of
> each object into an array?
It sounds as if you're using intuition to determine what needs
optimizing. Forgive me for not trusting your intuition, but I
recommend using a profiler to determine what needs optimizing. It's
rare that intuition can do as well as good debugging tools can do in
telling what your code is actually doing.
Having said that, I should also mention that the perldata manpage
says, "You can also gain some minuscule measure of efficiency by
pre-extending an array that is going to get big." I believe that that
means code like this:
my @return_values; # new, empty array
$#return_values =3D 999_999; # room for a million items
@return_values =3D (); # but empty to start
After that, your first up-to-one-million elements shouldn't take so
long to push. But I haven't been able to verify this by benchmarking,
so I suspect that either the documentation is right about the
improvement being "minuscule", or I've misunderstood what it says
about pre-extending arrays.
My own intuition (which is not necessarily better than any other)
suggests that you may be accumulating so much data in memory at once
that your process has become unwieldy. If you're collecting more than
a few million items in @return_values, or if those items are
collectively taking up a lot of memory, perhaps you should be keeping
them on disk instead of in memory. There are some modules on CPAN that
can help in making a tied data structure.
Hope this helps!
--Tom Phoenix
Stonehenge Perl Training
|