PERL Programming - memory allocation in perl

Author

memory allocation in perl

Winston

2006-11-20, 6:58 pm

I have tried to create a(n) array/hash in perl on a Unix machine that
has 8 GB memory. The biggest array/hash I could created with perl had
~260 million elements Does perl allocate space for arrray/hash only
to STACK which may has a upper limit? The reason I ask this is
because I tried dynamic allocation in C and the biggest array I could
create on the same machine contained 1.1 billion elements? This is the
size I need for my work. I know C uses heap when memory is
dynamically allocated. Is this the reason that C can do it but perl
cannot?

I am going to increase the memory to 16GB, but I still not sure if this
will resolve the problem if I still use perl.

Any comments, or suggestions?

Thank you very much in advance.

Winston

Sherm Pendley

2006-11-20, 6:58 pm

"Winston" <whairong@gmail.com> writes:

> I have tried to create a(n) array/hash in perl on a Unix machine that
> has 8 GB memory.

Was your Perl built with 64-bit support?

perl -V:use64bitall

If not, you're limited to a 32-bit address space.

sherm--

--
Web Hosting by West Virginians, for West Virginians: http://wv-www.net
Cocoa programming in Perl: http://camelbones.sourceforge.net

Winston

2006-11-20, 6:58 pm

Hi, Sherm, thank you very much for comments. The perl is V5.8.8 built
for x86_64-linux (I should correct my mistake, the machine is Linux).
The command you suggest returned me:

use64bitall='define';

Winston

Sherm Pendley wrote:
> "Winston" <whairong@gmail.com> writes:
>
>
> Was your Perl built with 64-bit support?
>
> perl -V:use64bitall
>
> If not, you're limited to a 32-bit address space.
>
> sherm--
>
> --
> Web Hosting by West Virginians, for West Virginians: http://wv-www.net
> Cocoa programming in Perl: http://camelbones.sourceforge.net

Sherm Pendley

2006-11-20, 6:58 pm

"Winston" <whairong@gmail.com> writes:

> Sherm Pendley wrote:
>
> Hi, Sherm, thank you very much for comments. The perl is V5.8.8 built
> for x86_64-linux (I should correct my mistake, the machine is Linux).
> The command you suggest returned me:
>
> use64bitall='define';

OK then, an SV (a Perl scalar) is defined like this in sv.h:

struct STRUCT_SV { /* struct sv { */
void* sv_any; /* pointer to something */
U32 sv_refcnt; /* how many references to us */
U32 sv_flags; /* what we are */
};

As you can see above, a Perl SV is 4x bigger than a 32-bit C int - this
being a 64-bit Perl, the void* is 64 bits wide. So, it's not surprising
that you could allocate only 1/4 as many SVs as ints.

That being the case, and given that you've determined that your Perl is 64-
bit, the simplest solution seems to be the one you've already arrived at:
Buy more RAM.

sherm--

--
Web Hosting by West Virginians, for West Virginians: http://wv-www.net
Cocoa programming in Perl: http://camelbones.sourceforge.net

Winston

2006-11-21, 6:57 pm

Hi, Sherm:

I guess you have answered my question thoroughly. I have found that I
could only created an array of 1/4 size of what I created in C. Your
explanation is very convincing.

Thank you very much.

Winston

Sherm Pendley wrote:
> "Winston" <whairong@gmail.com> writes:
>
>
> OK then, an SV (a Perl scalar) is defined like this in sv.h:
>
> struct STRUCT_SV { /* struct sv { */
> void* sv_any; /* pointer to something */
> U32 sv_refcnt; /* how many references to us */
> U32 sv_flags; /* what we are */
> };
>
> As you can see above, a Perl SV is 4x bigger than a 32-bit C int - this
> being a 64-bit Perl, the void* is 64 bits wide. So, it's not surprising
> that you could allocate only 1/4 as many SVs as ints.
>
> That being the case, and given that you've determined that your Perl is 64-
> bit, the simplest solution seems to be the one you've already arrived at:
> Buy more RAM.
>
> sherm--
>
> --
> Web Hosting by West Virginians, for West Virginians: http://wv-www.net
> Cocoa programming in Perl: http://camelbones.sourceforge.net

Sponsored Links