PERL POE - POE not doing what I'm expecting it to......

Author

POE not doing what I'm expecting it to......

Peter Farmer

2006-06-17, 8:59 am

Hi all,

I'm fairly new to coding with POE, and I think I've not quite grasped
the concepts correctly.

I have a script which has two sessions, the first session is what I
call a scheduler session, basically it reads the content of a txt file
and "schedules" events into the other session with

$kernel->post("worker","job_runner",$arg);

the scheduler session then runs a

$kernel->delay_set("scheduler", 10);

What I'm expecting to happen is that the worker session chugs through
the events its been posted, but after 10 seconds the scheduler session
runs again to check the content of txt file and post more events to
the worker if needed. Basically what happens is that the worker
session chugs through it entire queue and then the scheduler runs
again.

Is there a way to get what I want to happen to happen?

Thanks,

--
Peter Farmer

Peter Farmer

2006-06-17, 8:59 am

On 08/06/06, Peter Farmer <pfarmer@gmail.com> wrote:
> Hi all,
>
> I'm fairly new to coding with POE, and I think I've not quite grasped
> the concepts correctly.
>
> I have a script which has two sessions, the first session is what I
> call a scheduler session, basically it reads the content of a txt file
> and "schedules" events into the other session with
>
> $kernel->post("worker","job_runner",$arg);
>
> the scheduler session then runs a
>
> $kernel->delay_set("scheduler", 10);
>
> What I'm expecting to happen is that the worker session chugs through
> the events its been posted, but after 10 seconds the scheduler session
> runs again to check the content of txt file and post more events to
> the worker if needed. Basically what happens is that the worker
> session chugs through it entire queue and then the scheduler runs
> again.
>
> Is there a way to get what I want to happen to happen?
>
>
>
> Thanks,
>
> --
> Peter Farmer
>

Hi again,

pastebins of the code:

http://poundperl.pastebin.com/767423
http://poundperl.pastebin.com/767424

Thanks,

--
Peter Farmer

Nicholas Perez

2006-06-17, 8:59 am

Hi Peter,

I don't think I would organize this the way it's been done. You are
basically providing the same code refs to two different sessions for
starters. This could easily be done with a single session and using
something like POE::Wheel::FollowTail, so that anytime there is new
input appended to the file, your events would get called. I don't know
how your input file gets generated, but I assume it is just a log file
that keeps growing if the file remains existent.

If you are set on doing it this way, I would aggregate your two
sessions into one. No real need to have two sessions. Also, your email
mentions one way of delayed events and your code uses another. Which
one are you using? I am not exactly sure alarm_set over and over again
is what you want to do. See POE::Kernel.

Just some things to think about.

On 6/8/06, Peter Farmer <pfarmer@gmail.com> wrote:
> On 08/06/06, Peter Farmer <pfarmer@gmail.com> wrote:
>
> Hi again,
>
> pastebins of the code:
>
> http://poundperl.pastebin.com/767423
> http://poundperl.pastebin.com/767424
>
>
> Thanks,
>
> --
> Peter Farmer
>

Peter Farmer

2006-06-17, 8:59 am

Hi Nicholas,

Thanks for the reply, I was using 1 single session, but then thought
the reason the delay time wasn't working was because it was all set in
the same session.

Basically the file is created with a list of networks in it, the
script then deletes it once it has read them all in. The idea is that
"someone" drops a file into the directory, the script reads it and
schedules the rbl lookups on the networks in the file, the results are
then written to a file somewhere else (yet to be coded).

I was initially using delay, but again tried alarm_set to see if that
worked instead, it didn't and I didn't change it back.

I'm still fairly

because having spoken to the nice ppl in
#poe on irc.perl.org I've been told that it should work as I'm
expecting.... but its not :(

Thanks for looking :)

--
Peter Farmer

On 08/06/06, Nicholas Perez <nicholasrperez@gmail.com> wrote:[color=darkred]
> Hi Peter,
>
> I don't think I would organize this the way it's been done. You are
> basically providing the same code refs to two different sessions for
> starters. This could easily be done with a single session and using
> something like POE::Wheel::FollowTail, so that anytime there is new
> input appended to the file, your events would get called. I don't know
> how your input file gets generated, but I assume it is just a log file
> that keeps growing if the file remains existent.
>
> If you are set on doing it this way, I would aggregate your two
> sessions into one. No real need to have two sessions. Also, your email
> mentions one way of delayed events and your code uses another. Which
> one are you using? I am not exactly sure alarm_set over and over again
> is what you want to do. See POE::Kernel.
>
> Just some things to think about.
>
> On 6/8/06, Peter Farmer <pfarmer@gmail.com> wrote:

Peter Farmer

2006-06-17, 8:59 am

Hi all,

Here is some simpler code to illustrate my point...

[pfarmer@optiplex poe]$ cat test.pl
#!/home/pfarmer/local/bin/perl

use strict;
use warnings;

use POE;

POE::Session->create (
inline_states => {
_start => sub { my $kernel = $_[KERNEL];
$kernel->yield("scheduler"); },
_stop => sub { print "STOP\n"; },
job_runner => sub { print time() . " - Sleeping\n"; sleep(2); },
scheduler => \&sche,
}
);

sub sche () {
print time() . " - Running Scheduler\n";
my $kernel = $_[KERNEL];

$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");
$kernel->yield("job_runner");

$kernel->delay_set("scheduler", 10);
}

POE::Kernel->run();
exit;

[pfarmer@optiplex poe]$ perl -v

This is perl, v5.8.8 built for i686-linux

Copyright 1987-2006, Larry Wall

Perl may be copied only under the terms of either the Artistic License or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl". If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

[pfarmer@optiplex poe]$ perl ./test.pl
1149783926 - Running Scheduler
1149783926 - Sleeping
1149783928 - Sleeping
1149783930 - Sleeping
1149783932 - Sleeping
1149783934 - Sleeping
1149783936 - Sleeping
1149783938 - Sleeping
1149783940 - Sleeping
1149783942 - Sleeping
1149783944 - Sleeping
1149783946 - Running Scheduler
1149783946 - Sleeping
1149783948 - Sleeping
..............

Now, when I run the script, because of the
$kernel->delay_set("scheduler",10) at the end of the "sche" sub, I'm
expecting to see "sche" running at about 1149783936 or 1149783938 or
as soon as possible after the 10 seconds have passed not 1149783946
once all the other event "job_runners" are finished.

My POE version is 0.3501.

Sorry to be a pain...... :)

--
Peter Farmer

Apocalypse

2006-06-17, 8:59 am

Peter Farmer wrote:
> Hi all,
>
> ...
>
> Now, when I run the script, because of the
> $kernel->delay_set("scheduler",10) at the end of the "sche" sub, I'm
> expecting to see "sche" running at about 1149783936 or 1149783938 or
> as soon as possible after the 10 seconds have passed not 1149783946
> once all the other event "job_runners" are finished.
>
> My POE version is 0.3501.
>
> Sorry to be a pain...... :)
>

Hi Peter,

The problem is that you have the "right" assumption of POE but the wrong
implementation. POE is a "cooperative multi-tasking" environment, which
means it doesn't guarantee that events will arrive at the specific time.
It processes *ALL* events in a "queue" ( we're working on having a
separate thread for each session so they can REALLY run concurrently,
but that's a topic for another time )

The problem here is that you're using "sleep" - this forces POE to HALT
everything and wait for the sleep to return. Hence the 'cooperative'
part - POE expects you to give control back to it as soon as possible.
You can greatly improve your script simply by changing the sleep(2) to a
timed event. However, the other suggestions I've seen on the list makes
more sense - the FollowTail example, for one.

If you want to see exactly what you are thinking, simply remove the 10+
yields for job_runner and replace the sleep() call with a delay_set(
'job_runner', 2 ). This will not exactly achieve what your program's
general logic wants to do, but it's a step in the right direction :)

Good luck and welcome to the magnificent world of POE!

--
Apocalypse

Homepage: http://JiNxEdStAr.0ne.us
IRC: Apocalypse@irc.perl.org
IRC: Apocalypse@irc.freenode.net
Perl Stuff: http://search.cpan.org/~APOCAL/

Peter Farmer

2006-06-17, 8:59 am

Thanks to everyone who took the time to reply, I now have a much
better understanding of how to write POE applications.....

Thanks,

--
Peter Farmer

Merijn Broeren

2006-06-17, 8:59 am

Quoting Peter Farmer (pfarmer@gmail.com):
>
> Now, when I run the script, because of the
> $kernel->delay_set("scheduler",10) at the end of the "sche" sub, I'm
> expecting to see "sche" running at about 1149783936 or 1149783938 or
> as soon as possible after the 10 seconds have passed not 1149783946
> once all the other event "job_runners" are finished.
>
Actually this is what I would expect to happen. You yield 10 slow
events. They all get scheduled to happen at the same time, the current
one. Poor POE tries to do them, but they keep being uncooperative and
not yielding back. Everytime POE gets a look in it sees events it should
have done in the past. What should it do, the old events or the new
'now' event, your alarm? It will do the old stuff first.

The general solution to this is be more cooperative. Be more fine
grained. Instead of this:

foreach my $line (@lines) {
# process $line
}

write something like this:

sub process_line {
# get kernel, heap, @lines could be in the heap
if (@lines) {
unshift $line;
# process $line
$kernel->yield(process_line, @lines)
}
}

or do a few chunks at a time.

Cheers,
--
Merijn Broeren | Sometime in the middle ages, God got fed up with us
Software G

| and put earth at sol.milky-way.univ in his kill-file.
| Pray all you want, it just gets junked.

Sponsored Links