ithreads - perl doesn't release thread memory

Author

perl doesn't release thread memory

Gabriel Blum

2005-06-08, 9:03 pm

Dear Group,

I've made a simple stufy regarding memory deallocation and threads with
ActivePerl 5.8.6;
This simple script taken from the documentation takes up infinite memory.
Anyone sees any problem, or knows a workaround?

use threads;
my $thr;

for (;;) {
$thr = threads->new(\&sub1);
$thr->detach(); # bug also present with $thr->join()
sleep 1;
}
sub sub1 { print "In thread!\n"; }

Another interesting issue is that adding an explicit undef clears up memory
....
for (;;) {
$thr = threads->new(\&sub1);
$thr->detach(); # bug also present with $thr->join()
undef $thr; # crazy attempt
sleep 1;
}

and works with the sample script, but with my complicated program, it fails
with some ugly messages
Win32::OLE(0.1702): GetOleObject() Not a Win32::OLE object at
C:/Perl/site/lib/Win32/OLE/Lite.pm line 221.
Win32::OLE(0.1702): GetOleObject() Not a Win32::OLE object at
C:/Perl/site/lib/Win32/OLE/Lite.pm line 221.
Win32::OLE(0.1702): GetOleObject() Not a Win32::OLE object at
C:/Perl/site/lib/Win32/OLE/Lite.pm line 221.
Win32::OLE(0.1702): GetOleObject() Not a Win32::OLE object at
C:/Perl/site/lib/Win32/OLE/Lite.pm line 154.
Can't call method "Refresh" on an undefined value at
module/base/DataService.pmline 122.
Attempt to free non-existent shared string 'Open', Perl interpreter:
0x224014 at (eval 1) line 1.
Attempt to free non-existent shared string 'State', Perl interpreter:
0x224014 at (eval 1) line 1.
Free to wrong pool 223f50 not ab35988 at (eval 1) line 1.

This is disturbing. If perl is not deallocating memory naturally, I suppose
that there is some internal reference
to the thread. But the thread is supposed to be detached! That's strange
thing #1;

Strange thing #2 : On the other hand, when I explicitly undef $thr, and perl
deallocate it, that means $thr was the only reference to
the thread object. But if that is true, why in the first place didn't perl
deallocate the thread object when I overwrote the
reference with a new thread?

Thanks in advance. I look forward to hear a bit of the wisdom and experience
of the more experienced perl programmers from this list

My best regards,
Gabriel

Jan Dubois

2005-06-08, 9:03 pm

On Tue, 03 May 2005, Gabriel Blum wrote:
> and works with the sample script, but with my complicated program, it fails
> with some ugly messages
> Win32::OLE(0.1702): GetOleObject() Not a Win32::OLE object at
> C:/Perl/site/lib/Win32/OLE/Lite.pm line 221.

It is not safe to use Win32::OLE in multi-threaded programs. COM objects have
thread affinity and cannot simply be cloned across threads.

Cheers,
-Jan

B Lecomte

2005-06-08, 9:03 pm

this sample seems to work : how did you highlight the infinite memory trouble
of your simple script ?

use Net::FTP;use Net::SFTP;
use File::Basename;
use threads;
use Thread::Semaphore;
use Carp;

my $semaphore = new Thread::Semaphore;
my $GlobalVariable : shared = 0;

$thr1 = new threads (\&sample_sub, 'ftp', 'mymachine1', 'auserx',
'apwdx', 'test.tar.Z');
$thr2 = new threads (\&sample_sub, 'sftp', 'my2machine', 'auserx',
'apwdx', 'adummyfile.tar');
$thr3 = new threads (\&sample_sub,
'ftp','mylastmachine','auserx','apwdx','
/database/auserhome/gcc-3.2.2.tar.gz');

sub sample_sub {
my ($protocole, $ser, $user, $password, $filetoget) = @_;
$_ = $protocole;
SWITCH : {
/^ftp$/ && do {
my $ftp= Net::FTP->new($ser,Debug=>0) or die "cannot connect to
$ser: $@";
$ftp->login($user,$password) or die "cannot login",
$ftp->message;
$ftp->cwd(dirname ($filetoget)) or die "cannot change working
directory", $ftp->message;
$ftp->get(basename ($filetoget)) ;
$ftp->quit();
last SWITCH;
};
/^sftp$/ && do {
my %args=(user=>$user,password=>$password);
my $sftp= Net::SFTP->new($ser,%args);
$sftp->quit();
last SWITCH;
};
croak "Mc Donald protocol unknown $_";
}

my $TryCount = 10;
my $LocalCopy;
sleep 1;
while ($TryCount--) {
$semaphore->down;
$LocalCopy = $GlobalVariable;
print "$TryCount tries left for sub $protocole, $ser, $user,
(\$GlobalVariable is $GlobalVariable)\n";
sleep 2;
$LocalCopy++;
$GlobalVariable = $LocalCopy;
$semaphore->up;
}
}

$thr1->join;
$thr2->join;
$thr3->join;

Gabriel Blum

2005-06-08, 9:03 pm

My test program keeps creating threads on an infinite loop. Your program
creates only 3 threads;

I watched the virtual memory size of the process under 'Task
Manager'->Processes on my Windows XP box. I kept growing forever

Regards,
Gabriel

"B Lecomte" <b.lecomte@free.fr> wrote in message
news:1115200455.42789bc77d54e@imp4-q.free.fr...
> this sample seems to work : how did you highlight the infinite memory
> trouble
> of your simple script ?
>
>
> use Net::FTP;use Net::SFTP;
> use File::Basename;
> use threads;
> use Thread::Semaphore;
> use Carp;
>
> my $semaphore = new Thread::Semaphore;
> my $GlobalVariable : shared = 0;
>
> $thr1 = new threads (\&sample_sub, 'ftp', 'mymachine1',
> 'auserx',
> 'apwdx', 'test.tar.Z');
> $thr2 = new threads (\&sample_sub, 'sftp', 'my2machine',
> 'auserx',
> 'apwdx', 'adummyfile.tar');
> $thr3 = new threads (\&sample_sub,
> 'ftp','mylastmachine','auserx','apwdx','
/database/auserhome/gcc-3.2.2.tar.gz');
>
> sub sample_sub {
> my ($protocole, $ser, $user, $password, $filetoget) = @_;
> $_ = $protocole;
> SWITCH : {
> /^ftp$/ && do {
> my $ftp= Net::FTP->new($ser,Debug=>0) or die "cannot connect
> to
> $ser: $@";
> $ftp->login($user,$password) or die "cannot login",
> $ftp->message;
> $ftp->cwd(dirname ($filetoget)) or die "cannot change
> working
> directory", $ftp->message;
> $ftp->get(basename ($filetoget)) ;
> $ftp->quit();
> last SWITCH;
> };
> /^sftp$/ && do {
> my %args=(user=>$user,password=>$password);
> my $sftp= Net::SFTP->new($ser,%args);
> $sftp->quit();
> last SWITCH;
> };
> croak "Mc Donald protocol unknown $_";
> }
>
> my $TryCount = 10;
> my $LocalCopy;
> sleep 1;
> while ($TryCount--) {
> $semaphore->down;
> $LocalCopy = $GlobalVariable;
> print "$TryCount tries left for sub $protocole, $ser,
> $user,
> (\$GlobalVariable is $GlobalVariable)\n";
> sleep 2;
> $LocalCopy++;
> $GlobalVariable = $LocalCopy;
> $semaphore->up;
> }
> }
>
> $thr1->join;
> $thr2->join;
> $thr3->join;
>
>

Gabriel Blum

2005-06-08, 9:03 pm

Thanks for the tip Jan.

I still blame perl itself for the trouble I'm having. Is it correct to
'undef' the thread object after it is detached?
The documentation says nothing about it.

Regards,
Gabriel

"Jan Dubois" <jand@ActiveState.com> wrote in message
news:200505032303.j43N3E1V024862@smtp3.ActiveState.com...
> On Tue, 03 May 2005, Gabriel Blum wrote:
>
> It is not safe to use Win32::OLE in multi-threaded programs. COM objects
> have
> thread affinity and cannot simply be cloned across threads.
>
> Cheers,
> -Jan
>
>

B Lecomte

2005-06-08, 9:03 pm

Ok Gabriel, I see what you mean ;-)
my sample was about to use some safe multi-threaded module (I believe ;-)
Your second test is far better, the undef might be right to avoid
the memory growth of your perl.exe (stays at 4704 Ko, using cygwin 5.8.6).
Hopefully, both samples release memory when ^C. ;-)
I don't know if there is some gc as the java finalize ?

Regards,
Brice

Sponsored Links