PERL POE - IKC::Server memory leak per every connect

Author

IKC::Server memory leak per every connect

Vladimir M

2005-06-08, 9:04 pm

Hello All.

I found memory leak in POE::Component::IKC::Server per every client
connect (~16kB).

What's wrong?

Examples:

Server from http://poe.perl.org/?POE_Cookbook/Application_Servers_2
Client (sub SimplyTask from
http://poe.perl.org/?POE_Cookbook/Application_Servers_2):
------------------------------------------------------------------------------------------------------------
#!/usr/local/bin/perl

use warnings;
use strict;

use Data::Dumper;
use Time::HiRes;
use POE::Component::IKC::ClientLite;

my ($RequestsCount, $ParallelProcessesCount, $RandomDelayMax) = @ARGV;

$RequestsCount ||= 200;
$ParallelProcessesCount ||=20;
$RandomDelayMax ||= 0.1;

for(my $i=0; $i<$ParallelProcessesCount; $i++) {
if(fork()== 0) {
# Child
sleep(rand()*$RandomDelayMax);
print "Child: $$\n";
SimplyTask();
exit;
}
else {
$RequestsCount--;
}
};

while($RequestsCount) {
wait();
if(fork()== 0) {
# Child
sleep(rand()*$RandomDelayMax);
print "Child: $$\n";
SimplyTask();
exit;
}
else {
$RequestsCount--;
}
};

for(my $i=0; $i<$ParallelProcessesCount; $i++) {
wait();
}

sub SimplyTask {
# Create an IKC client. This also establishes a connection to an IKC
# server. Part of the registration process is choosing a unique name
# for ourselves, which we do naively by abusing the process ID.

my $name = "Client$$";
my $remote = create_ikc_client(
port => 31338,
name => $name,
timeout => 5,
);
die $POE::Component::IKC::ClientLite::error unless $remote;

# We want the server to add up a list of numbers. Using
# post_return(), we can send a detached event to the server and wait
# for its response. The response can be delayed up to the
# create_ikc_client() timeout. post_return() returns the value that
# was posted back to us from service_response() in the corresponding
# server example.

my $return_value;
my @numbers = qw(8 6 7 5 3 0 9);
print "Summing $$ : @numbers\n";

$return_value = $remote->post_respond( 'application/calc_sum', \@numbers );
die $POE::Component::IKC::ClientLite::error unless defined $return_value;

print "The sum is $$ : $return_value\n";
};
------------------------------------------------------------------------------------------------------------

philip@awale.qc.ca

2005-06-08, 9:04 pm

On 21-Mar-2005 Vladimir M wrote:
> Hello All.
>
> I found memory leak in POE::Component::IKC::Server per every client
> connect (~16kB).
>
> What's wrong?

There's a bug.

What version of POE, IKC and perl are you using? We suspect that IKC is
exercising some code in POE, that's falling into a bug in some versions of
perl, that is causing this leak.

-Philip

Vladimir M

2005-06-08, 9:04 pm

On Mon, 21 Mar 2005 12:38:21 -0500 (EST), philip@awale.qc.ca
<philip@awale.qc.ca> wrote:
>
> On 21-Mar-2005 Vladimir M wrote:
>
> There's a bug.

I write the PoCo Server and have found in it very similar memory leak.
Has corrected a mistake obvious closing of a socket.
I hope it to you something will help.

> What version of POE, IKC and perl are you using? We suspect that IKC is
> exercising some code in POE, that's falling into a bug in some versions of
> perl, that is causing this leak.

OS FreeBSD 5.0

$ perl -v

This is perl, v5.8.6 built for i386-freebsd-64int
(with 2 registered patches, see perl -V for more detail)

Module id = POE
CPAN_USERID RCAPUTO (Rocco Caputo <rcaputo@pobox.com> )
CPAN_VERSION 0.3009
CPAN_FILE R/RC/RCAPUTO/POE-0.3009.tar.gz
MANPAGE POE - portable multitasking and networking framework for Perl
INST_FILE /usr/local/lib/perl5/site_perl/5.8.4/POE.pm
INST_VERSION 0.3009

Module id = POE::Component::IKC::Server
CPAN_USERID GWYN (Philip Gwyn <perl@pied.nu> )
CPAN_VERSION 0.14
CPAN_FILE G/GW/GWYN/POE-Component-IKC-0.14.tar.gz
MANPAGE POE::Component::IKC::Server - POE Inter-kernel
Communication server
INST_FILE
/usr/local/lib/perl5/site_perl/5.8.4/POE/Component/IKC/Server.pm
INST_VERSION 0.14

liste@artware.qc.ca

2005-06-08, 9:04 pm

On 22-Mar-2005 Vladimir M wrote:
> I write the PoCo Server and have found in it very similar memory leak.
> Has corrected a mistake obvious closing of a socket.
> I hope it to you something will help.

We've been thrashing over this. It looks like POE and IKC are exercising
a bug in Perl. 0.1501
(http://pied.nu/Perl/POE-Component-IKC-0.1501.tar.gz) uses a work around.

Could you test to see if it clears up the problem? If it doesn't, could
you send me your test script?

Your english isn't clear, you seem to say you solved the problem. If so,
I'd like to see your patch.

Thank you,

-Philip

Vladimir M

2005-06-08, 9:04 pm

On Wed, 23 Mar 2005 02:28:40 -0500 (EST), liste@artware.qc.ca

> On 22-Mar-2005 Vladimir M wrote:
>
> We've been thrashing over this. It looks like POE and IKC are exercising
> a bug in Perl. 0.1501
> (http://pied.nu/Perl/POE-Component-IKC-0.1501.tar.gz) uses a work around.
>
> Could you test to see if it clears up the problem? If it doesn't, could
> you send me your test script?

No, it doesn't clears :(
But now IKC loses approximately 9 kB, was 16 earlier. An example of a
server and client below. I simply start them and I look the size of a
server script in top (it grow up per every connect).

server:
-------------------------------------------------------------------------------------------------------
#!/usr/local/bin/perl

use warnings;
use strict;
use POE qw(Session Component::IKC::Server);

POE::Component::IKC::Server->spawn(
port => 31338,
name => 'AppServer',
);

POE::Session->create(
inline_states => {
_start => \&service_start,
calc_sum => \&service_calc_sum,
did_something => \&service_response,
}
);

POE::Kernel->run();
exit 0;

sub service_start {
my ( $kernel, $heap ) = @_[ KERNEL, HEAP ];

my $service_name = "application";
$kernel->alias_set("application");
$kernel->call( IKC => publish => $service_name, ["calc_sum"] );
}

sub service_calc_sum {
my ( $kernel, $heap, $request ) = @_[ KERNEL, HEAP, ARG0 ];
my ( $data, $rsvp ) = @$request;

my $sum = 0;
if ( ref($data) eq "ARRAY" ) {
$sum += $_ foreach @$data;
}

$kernel->call( IKC => post => $rsvp, $sum );
}
-------------------------------------------------------------------------------------------------------

client:
-------------------------------------------------------------------------------------------------------
#!/usr/local/bin/perl

use warnings;
use strict;

use POE::Component::IKC::ClientLite;

for(my $i=0; $i<100; $i++) {
SimplyTask();
};

sub SimplyTask {
my $name = "Client$$";
my $remote = create_ikc_client(
port => 31338,
name => $name,
timeout => 5,
);
die $POE::Component::IKC::ClientLite::error unless $remote;

my $return_value;
my @numbers = qw(8 6 7 5 3 0 9);
print "Summing $$ : @numbers\n";

$return_value = $remote->post_respond( 'application/calc_sum', \@numbers );
die $POE::Component::IKC::ClientLite::error
unless defined $return_value;
print "The sum is: $return_value\n";
$remote->disconnect();
};
-------------------------------------------------------------------------------------------------------

> Your english isn't clear, you seem to say you solved the problem. If so,
> I'd like to see your patch.

Sorry, my english is ugly :(((((

It is necessary for us of the decision for communication mod_perl and
POE-application. I investigate IKC and in parallel I try to write the
own decision. In my decision there was a memory leak. Leak has fixed
when I began to close obviously a socket

Mohit Muthanna

2005-06-08, 9:04 pm

Interesting, because I just started looking at the RSS for my IKC
server and I see the same thing.

I installed the IKC-.1501 packages and now lose approx 8K per
connection/disconnect.

I'm using ClientLite from a mod_perl script to connect to my POE service.

--
mohit@reddwarf mohit $ perl -V
Summary of my perl5 (revision 5 version 8 subversion 5) configuration:
Platform:
osname=linux, osvers=2.6.10, archname=i686-linux
uname='linux reddwarf 2.6.10 #1 fri jan 14 10:25:13 local time
zone must be set--see zic manu i686 intel(r) pentium(r) 4 cpu 2.40ghz
genuineintel gnulinux '
config_args='-des -Darchname=i686-linux -Dcccdlflags=-fPIC
-Dccdlflags=-rdynamic -Dcc=gcc -Dprefix=/usr -Dvendorprefix=/usr
-Dsiteprefix=/usr -Dlocincpth= -Doptimize=-O2 -march=pentium4
-fomit-frame-pointer -Duselargefiles -Dd_semctl_semun
-Dscriptdir=/usr/bin -Dman1dir=/usr/share/man/man1
-Dman3dir=/usr/share/man/man3 -Dinstallman1dir=/usr/share/man/man1
-Dinstallman3dir=/var/tmp/portage/perl-5.8.5-r4/image//usr/share/man/man3
-Dman1ext=1 -Dman3ext=3pm -Dinc_version_list=5.8.0 5.8.0/i686-linux
5.8.2 5.8.2/i686-linux 5.8.4 5.8.4/i686-linux -Dcf_by=Gentoo -Ud_csh
-Di_ndbm -Di_gdbm -Di_db -Dusrinc=/usr/include'
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef
usemultiplicity=undef
useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-fno-strict-aliasing -pipe -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O2 -march=pentium4 -fomit-frame-pointer',
cppflags='-DPERL5 -fno-strict-aliasing -pipe'
ccversion='', gccversion='3.3.5 (Gentoo Linux 3.3.5-r1,
ssp-3.3.2-3, pie-8.7.7.1)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
ls

size=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lpthread -lnsl -lndbm -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=/lib/libc-2.3.4.so, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version='2.3.4'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'

Characteristics of this binary (from libperl):
Compile-time options: USE_LARGE_FILES
Built under linux
Compiled at Feb 15 2005 11:21:17
@INC:
/etc/perl
/usr/lib/perl5/site_perl/5.8.5/i686-linux
/usr/lib/perl5/site_perl/5.8.5
/usr/lib/perl5/site_perl
/usr/lib/perl5/vendor_perl/5.8.5/i686-linux
/usr/lib/perl5/vendor_perl/5.8.5
/usr/lib/perl5/vendor_perl
/usr/lib/perl5/5.8.5/i686-linux
/usr/lib/perl5/5.8.5
/usr/local/lib/site_perl

Sponsored Links