PERL POE - Defunct processess help

Author

Defunct processess help

King, John \

2006-06-24, 8:38 am

I am getting alot of defunct processess and being new to POE I'll go
ahead and point the finger at me :) if anyone can help me understand
why this is occuring though that would be great.

Process table shows : 00:00:00 [sched.pl] <defunct>

My scheduler program is the Child_Processes_3 merged with my other code.
The defunct processess do get cleaned up after each session call but I
am unsure of why I am getting the defunct processess in the first place.
From what I observe in the database, the children are running fine and
storing their gathered data.

Here are the important code pieces in the scheduler

thanks!,

Greg

#!/usr/bin/perl

use warnings;
use strict;
use Net::CIDR;
use DBI;
use Getopt::Long;
use Time::Local;
use POE qw(Wheel::Run Filter::Reference);

my $node_arg; my $debug_arg; my $seed_arg;
GetOptions( "debug" => \$debug_arg);

sub MAX_CONCURRENT_TASKS () { 5 }
my @tasks; #array of tasks to queue up and send to the POE
kernel

# to shorten this post the only change from example 3 is to the do_stuff
subroutine

sub do_stuff {
my $task = shift;
my $filter = POE::Filter::Reference->new();

# Simulate a long, blocking task.
my $res = `$task`;
my %result;

if ($res =~ /Success/) {

%result =
( task => $task,
status => "Poller Returned Successful.",
);
} else {
%result =
( task => $task,
status => "Poller Returned Failure.",
);
}

# Generate some output via the filter. Note the strange use of list
# references.
my $output = $filter->put( [ \%result ] );
print @$output;
}

# A whole lot of subs from my program

#then the main loop

# get various filter settings from the db
get_filters;
# Start by polling the seed devices
foreach my $srouter (@seed_devices) {
if (isIP($srouter)) {
#is this an allowed_ip?
if (!allowed_ip($srouter)) {
print "MAIN:$srouter is not an allowed IP.\n" if
$debug;
} else {
#does this exist in the db?
if (node_in_db ($srouter)) {
print "MAIN: Node ($srouter) found in
db, checking last poll time.\n" if $debug;
# if it does, has more than
$conf_time_pass seconds passed since last poll?

if (date_passed) {
print "MAIN: Node ($srouter)
time to re-poll.\n" if $debug;
#force causes the disco.pl
program to re-poll and update the database
#regardless if the node has
previously been discovered
push (@tasks,"/path_to/disco.pl
--node=$srouter --force");
} else {}

} else {
push (@tasks,"/path_to/disco.pl
--node=$srouter");
print "MAIN: Node ($srouter) not found
in db. Discovering node.\n" if $debug;
}
}
} else {
print "MAIN: $srouter is not an ip address\n" if $debug;
}
}

# create the POE session so that I can poll up to MAX_CONCURRENT_TASKS
in the @tasks array at the same time
POE::Session->create
( inline_states =>
{ _start => \&start_tasks,
next_task => \&start_tasks,
task_result => \&handle_task_result,
task_done => \&handle_task_done,
task_debug => \&handle_task_debug,
}
);
$poe_kernel->run();

# Ok the seed routers are done
# We now need to pull recrods from the disco_queue table. Nodes with a 1

# mark important nodes and take precedence (cdp connections for example
are flagged such)
# this will allow me to prioritize the polling order in the queue so
that backbone connections
# get built first

# Step 1. Is the queue empty?
while (queue_count) {
#poll the important connector nodes
while (important_node_count) {
#important_nodes calls the db for any nodes in the queue
with priority flag set, stores those in the @tasks array
# stored like push (@tasks,"/path_to/disco.pl
--node=$srouter");

important_nodes;

POE::Session->create
( inline_states =>
{ _start => \&start_tasks,
next_task => \&start_tasks,
task_result => \&handle_task_result,
task_done => \&handle_task_done,
task_debug => \&handle_task_debug,
}
);
$poe_kernel->run();
}

# ok now poll the rest of the nodes (should be leaf nodes at
this point)
# other_nodes calls the database for the queue and stores that
into the @tasks array
# stored like push (@tasks,"/path_to/disco.pl --node=$srouter");

other_nodes;
POE::Session->create
( inline_states =>
{ _start => \&start_tasks,
next_task => \&start_tasks,
task_result => \&handle_task_result,
task_done => \&handle_task_done,
task_debug => \&handle_task_debug,
}
);
$poe_kernel->run();

}

$dbh->disconnect ();
print "MAIN:Ending at " . localtime(time) . "\n" if $debug;
exit 0;

Kidney Bingos

2006-06-24, 8:38 am

On Fri, Jun 23, 2006 at 12:53:51PM -0500, King, John (Greg) (LMIT-HOU) wrote:
> I am getting alot of defunct processess and being new to POE I'll go
> ahead and point the finger at me :) if anyone can help me understand
> why this is occuring though that would be great.
>

Hi,

You need to register a signal handler for CHLD to reap the Wheel::Run processes.

It's mentioned in the POE::Wheel::Run documentation and I believe in the POE::Kernel
docs.

Cheers,

--
Chris Williams
aka BinGOs
PGP ID 0x4658671F
http://www.gumbynet.org.uk
==========================

Dougie!

2006-06-27, 4:20 am

Hey Greg,

(Hello again!)

You may want to setup the run wheels like queues...

Spawn the Run queues and connect them back... Put each Reference in al
element in an array in the heap. As you spawn off SNMP polling tasks,
simply shift the reference fo the next Run Queue off the array and push
it back on the end in a Round Robin fashion.

Doing things this way may make better sense.

I have had very good performance in setting up predetermined queue type
handlers to do the work in a distributed sort of way especially if I
have recurring tasks. If I was going to run something every now and
then or on a demand only type basis, I then use a run wheel to spawn
another process.

Dunno..

Dougie!!!

John King wrote:
> I am getting alot of defunct processess and being new to POE I'll go
> ahead and point the finger at me :) if anyone can help me understand
> why this is occuring though that would be great.
>
> Process table shows : 00:00:00 [sched.pl] <defunct>
>
> My scheduler program is the Child_Processes_3 merged with my other code.
> The defunct processess do get cleaned up after each session call but I
> am unsure of why I am getting the defunct processess in the first place.
> From what I observe in the database, the children are running fine and
> storing their gathered data.
>
> Here are the important code pieces in the scheduler
>
> thanks!,
>
> Greg
>
> #!/usr/bin/perl
>
> use warnings;
> use strict;
> use Net::CIDR;
> use DBI;
> use Getopt::Long;
> use Time::Local;
> use POE qw(Wheel::Run Filter::Reference);
>
> my $node_arg; my $debug_arg; my $seed_arg;
> GetOptions( "debug" => \$debug_arg);
>
> sub MAX_CONCURRENT_TASKS () { 5 }
> my @tasks; #array of tasks to queue up and send to the POE
> kernel
>
> # to shorten this post the only change from example 3 is to the do_stuff
> subroutine
>
> sub do_stuff {
> my $task = shift;
> my $filter = POE::Filter::Reference->new();
>
> # Simulate a long, blocking task.
> my $res = `$task`;
> my %result;
>
> if ($res =~ /Success/) {
>
> %result =
> ( task => $task,
> status => "Poller Returned Successful.",
> );
> } else {
> %result =
> ( task => $task,
> status => "Poller Returned Failure.",
> );
> }
>
> # Generate some output via the filter. Note the strange use of list
> # references.
> my $output = $filter->put( [ \%result ] );
> print @$output;
> }
>
>
> # A whole lot of subs from my program
>
> #then the main loop
>
> # get various filter settings from the db
> get_filters;
> # Start by polling the seed devices
> foreach my $srouter (@seed_devices) {
> if (isIP($srouter)) {
> #is this an allowed_ip?
> if (!allowed_ip($srouter)) {
> print "MAIN:$srouter is not an allowed IP.\n" if
> $debug;
> } else {
> #does this exist in the db?
> if (node_in_db ($srouter)) {
> print "MAIN: Node ($srouter) found in
> db, checking last poll time.\n" if $debug;
> # if it does, has more than
> $conf_time_pass seconds passed since last poll?
>
> if (date_passed) {
> print "MAIN: Node ($srouter)
> time to re-poll.\n" if $debug;
> #force causes the disco.pl
> program to re-poll and update the database
> #regardless if the node has
> previously been discovered
> push (@tasks,"/path_to/disco.pl
> --node=$srouter --force");
> } else {}
>
> } else {
> push (@tasks,"/path_to/disco.pl
> --node=$srouter");
> print "MAIN: Node ($srouter) not found
> in db. Discovering node.\n" if $debug;
> }
> }
> } else {
> print "MAIN: $srouter is not an ip address\n" if $debug;
> }
> }
>
> # create the POE session so that I can poll up to MAX_CONCURRENT_TASKS
> in the @tasks array at the same time
> POE::Session->create
> ( inline_states =>
> { _start => \&start_tasks,
> next_task => \&start_tasks,
> task_result => \&handle_task_result,
> task_done => \&handle_task_done,
> task_debug => \&handle_task_debug,
> }
> );
> $poe_kernel->run();
>
> # Ok the seed routers are done
> # We now need to pull recrods from the disco_queue table. Nodes with a 1
>
> # mark important nodes and take precedence (cdp connections for example
> are flagged such)
> # this will allow me to prioritize the polling order in the queue so
> that backbone connections
> # get built first
>
> # Step 1. Is the queue empty?
> while (queue_count) {
> #poll the important connector nodes
> while (important_node_count) {
> #important_nodes calls the db for any nodes in the queue
> with priority flag set, stores those in the @tasks array
> # stored like push (@tasks,"/path_to/disco.pl
> --node=$srouter");
>
> important_nodes;
>
> POE::Session->create
> ( inline_states =>
> { _start => \&start_tasks,
> next_task => \&start_tasks,
> task_result => \&handle_task_result,
> task_done => \&handle_task_done,
> task_debug => \&handle_task_debug,
> }
> );
> $poe_kernel->run();
> }
>
> # ok now poll the rest of the nodes (should be leaf nodes at
> this point)
> # other_nodes calls the database for the queue and stores that
> into the @tasks array
> # stored like push (@tasks,"/path_to/disco.pl --node=$srouter");
>
> other_nodes;
> POE::Session->create
> ( inline_states =>
> { _start => \&start_tasks,
> next_task => \&start_tasks,
> task_result => \&handle_task_result,
> task_done => \&handle_task_done,
> task_debug => \&handle_task_debug,
> }
> );
> $poe_kernel->run();
>
> }
>
> $dbh->disconnect ();
> print "MAIN:Ending at " . localtime(time) . "\n" if $debug;
> exit 0;
>
> ------_=_NextPart_001_01C696ED.FC6362BF--

Rob Bloodgood

2006-06-27, 7:10 pm

Dougie! wrote:
> Spawn the Run queues and connect them back... Put each Reference in al
> element in an array in the heap. As you spawn off SNMP polling tasks,
> simply shift the reference fo the next Run Queue off the array and push
> it back on the end in a Round Robin fashion.
>
> Doing things this way may make better sense.
>
Actually... If your child processes are doing SNMP polling, you may want
to consider using POE::Component::SNMP instead. This would eliminate
the need for child processes altogether.

<disclaimer>I am the author of PoCo::SNMP</disclaimer>

L8r,
Rob

Sponsored Links