PERL Miscellaneous - Handling a 450,000x450,000 array with Perl

Author

Handling a 450,000x450,000 array with Perl

francescomoi@usa.com

2006-10-30, 7:10 pm

Hi.

I'd like to create a program to handle data (natural numbers)
within a 450,000 x 450,000 array. I'm considering Perl, but
I don't know is it's powerful enough to manage it.

Thank you very much.

bugbear

2006-10-30, 7:10 pm

francescomoi@usa.com wrote:
> Hi.
>
> I'd like to create a program to handle data (natural numbers)
> within a 450,000 x 450,000 array. I'm considering Perl, but
> I don't know is it's powerful enough to manage it.

If the data is "not sparse" that's a lot of data.

It's not a question of perl's power, more
your systems.

if the data *is* sparse, it will be crucial to exploit
the sparseness, which is down to the code, either your,
or from a module/library.

BugBear

Ingo Menger

2006-10-30, 7:10 pm

francescomoi@usa.com wrote:
> Hi.
>
> I'd like to create a program to handle data (natural numbers)
> within a 450,000 x 450,000 array. I'm considering Perl, but
> I don't know is it's powerful enough to manage it.

Perl is, but you'll find few computers that could store 202 billion
"natural numbers".
Remember, a PC has a RAM of around 2 billion bytes, give or take a few.
You could hold a maximum of 16 billion natural numbers as long as they
are 0 or 1. But, 16 billion is still not 202 billion.

In any case, this will be a system where native integers have 64 bits.

francescomoi@usa.com

2006-10-30, 7:10 pm

Hi.

Thank you very much for your quick answers. I try to handle data from a
450x450 meter surface.

I've got 100,000 elements distributed on this surface, and each
square millimeter can hold one or more elements. So the array to create
is a sparse one: most of square millimeters holds zero elements, and
the rest, 20 or less.

I'm trying to find the square meter with more elements inside.

Thank you very much.

On Oct 30, 3:24 pm, "Ingo Menger" <quetzalc...@consultant.com> wrote:
> francesco...@usa.com wrote:
>
> "natural numbers".
> Remember, a PC has a RAM of around 2 billion bytes, give or take a few.
> You could hold a maximum of 16 billion natural numbers as long as they
> are 0 or 1. But, 16 billion is still not 202 billion.
>
> In any case, this will be a system where native integers have 64 bits.

bugbear

2006-10-30, 7:10 pm

francescomoi@usa.com wrote:
> Hi.
>
> Thank you very much for your quick answers. I try to handle data from a
> 450x450 meter surface.
>
> I've got 100,000 elements distributed on this surface, and each
> square millimeter can hold one or more elements. So the array to create
> is a sparse one: most of square millimeters holds zero elements, and
> the rest, 20 or less.
>
> I'm trying to find the square meter with more elements inside.

Ah. OK.

Your question should really be "what data structure should
I use to..."

It's not really a perl question at all, it's
really an algorithm and data structure question.

BugBear

bugbear

2006-10-31, 3:58 am

francescomoi@usa.com wrote:
> Hi.
>
> Thank you very much for your quick answers. I try to handle data from a
> 450x450 meter surface.
>
> I've got 100,000 elements distributed on this surface, and each
> square millimeter can hold one or more elements. So the array to create
> is a sparse one: most of square millimeters holds zero elements, and
> the rest, 20 or less.
>
> I'm trying to find the square meter with more elements inside.

Guessing game.

Is this data from an archeaological dig?

BugBear

Peter Scott

2006-10-31, 7:58 am

On Mon, 30 Oct 2006 05:10:07 -0800, francescomoi wrote:
> I'd like to create a program to handle data (natural numbers)
> within a 450,000 x 450,000 array. I'm considering Perl, but
> I don't know is it's powerful enough to manage it.

Any time you want to work with a matrix of lots of numbers in Perl,
consider http://pdl.perl.org/.

--
Peter Scott
http://www.perlmedic.com/
http://www.perldebugged.com/

Sponsored Links