PERL Beginners - Perl Async .10

Author

Perl Async .10

Todd English

2006-10-02, 6:59 pm

It was suggested that my initial post was too verbose for the Perl
newsgroups. I have rewritten my original example and have posted it
bellow. This code uses the "force" feature of async to help with code
compactness. I will post an additional example without the flag, but it
will be a bit longer.

The expected output would be 20 lines printed to the console saying the
following:
<some number> I have returned
What happens instead is that some of lines fail to print the "I have
returned" part of the message.

Sorry for the initial verbose posting.

Todd English

----------------sample code------------------
#!/usr/bin/perl -w

use strict;
use Async;

sub long_running_computation {
system qq{find ./ -type f 2>&1 >/dev/null};
return "I have returned";
}

my %proc_result;
my %build_result;

for (my $counter=0; $counter < 20; $counter++)
{
$proc_result{$counter} = Async->new( sub{
long_running_computation() } );
}

while ( my ($name, $proc) = each %proc_result)
{
$build_result{$name} = $proc->result('1');
}

while (my($host_name, $result) = each %build_result)
{
printf "%s\t%s\n", $host_name, $result;
}

DJ Stunks wrote:
> Todd English wrote:
>
> Could you boil all this down into one _short_ but _complete_ script
> which exhbits the problem you're describing? I, and I'm sure others as
> well due to your lack of responses, can't be bothered to wade through
> 134 lines of code and problem description to try and figure out what it
> is you're trying to do and what's not working as you intended it.
>
> Be absolutely sure that whatever you post compiles and runs clean with
> both use warnings and use strict in effect.
>
> HTH,
> -jp

Todd English

2006-10-02, 6:59 pm

Sample fot using the "force" option in the Async module.

-----------sample code--------------
#!/usr/bin/perl -w

use strict;
use Async;

sub long_running_computation {
system qq{find ./ -type f 2>&1 >/dev/null};
return "I have returned";
}

my %proc_result;
my %build_result;

for (my $counter=0; $counter < 10; $counter++)
{
$proc_result{$counter} = Async->new( sub{
long_running_computation() } );
}

while( 0 < scalar(keys %proc_result) )
{
my @host_list_to_delete;
while( (my $name) = each %proc_result)
{
if($proc_result{$name}->ready)
{
my $error;
if($error = $proc_result{$name}->error)
{
$build_result{$name} = "Async indicates a failure error
occured: $error !";
}
else
{
$build_result{$name} = $proc_result{$name}->result;
}
push(@host_list_to_delete, $name);
}
}

foreach my $item (@host_list_to_delete)
{
delete ( $proc_result{$item} );
}

sleep 10;
}

while (my($host_name, $result) = each %build_result)
{
printf "%s\t%s\n", $host_name, $result;
}

Todd English wrote:[color=darkred]
> It was suggested that my initial post was too verbose for the Perl
> newsgroups. I have rewritten my original example and have posted it
> bellow.
>
> The expected output would be 20 lines printed to the console saying the
> following:
> <some number> I have returned
> What happens instead is that some of lines fail to print the "I have
> returned" part of the message.
>
> Sorry for the initial verbose posting.
>
> Todd English
>
> ----------------sample code------------------
> #!/usr/bin/perl -w
>
> use strict;
> use Async;
>
> sub long_running_computation {
> system qq{find ./ -type f 2>&1 >/dev/null};
> return "I have returned";
> }
>
> my %proc_result;
> my %build_result;
>
> for (my $counter=0; $counter < 20; $counter++)
> {
> $proc_result{$counter} = Async->new( sub{
> long_running_computation() } );
> }
>
> while ( my ($name, $proc) = each %proc_result)
> {
> $build_result{$name} = $proc->result('1');
> }
>
> while (my($host_name, $result) = each %build_result)
> {
> printf "%s\t%s\n", $host_name, $result;
> }
>
> DJ Stunks wrote:

Todd English

2006-10-02, 6:59 pm

Tad,

Thank you for that constructive remark. I will not quote an entire post
in the future. I am not a Usenet wizard, but according to Google
perl.beginners is a highly active Usenet group
(http://groups.google.com/group/perl.beginners/about) so I am adding
them back into the thread.

Did you happen to have any insight into the Async issue?

-T

Tad McClellan wrote:
> [ removed non-existent perl.beginners newsgroup.
> Followups set.
> ]
>
>
> Todd English <toddenglish@gmail.com> wrote:
>
>
>
> foreach my $counter ( 0 .. 19 )
>
>
>
>
> [ snip TOFU. Please do not quote an entire post. ]
>
> --
> Tad McClellan SGML consulting
> tadmc@augustmail.com Perl programming
> Fort Worth, Texas

DJ Stunks

2006-10-02, 9:58 pm

Todd English wrote:
> Todd English wrote:

You don't have to be sorry, I only suggested that this could be the
reason you weren't getting any responses.

Also, I should mention that you'll wear out your welcome pretty quickly
if you don't post according to accepted norms - that is: don't top
post. instead snip appropriate remarks and relevant content and write
your responses below this information. that way someone can read from
top to bottom and understand what you're talking about.

Anyway, on to Perl.
[color=darkred]
> Sample fot using the "force" option in the Async module.
>
> -----------sample code--------------
> #!/usr/bin/perl -w
>
> use strict;
> use Async;
>
> sub long_running_computation {
> system qq{find ./ -type f 2>&1 >/dev/null};
> return "I have returned";
> }
>
> my %proc_result;
> my %build_result;
>
> for (my $counter=0; $counter < 10; $counter++)
> {
> $proc_result{$counter} = Async->new( sub{
> long_running_computation() } );
> }
>
> while( 0 < scalar(keys %proc_result) )
> {
> my @host_list_to_delete;
> while( (my $name) = each %proc_result)
> {
> if($proc_result{$name}->ready)
> {
> my $error;
> if($error = $proc_result{$name}->error)
> {
> $build_result{$name} = "Async indicates a failure error
> occured: $error !";
> }
> else
> {
> $build_result{$name} = $proc_result{$name}->result;
> }
> push(@host_list_to_delete, $name);
> }
> }
>
> foreach my $item (@host_list_to_delete)
> {
> delete ( $proc_result{$item} );
> }
>
> sleep 10;
> }
>
>
> while (my($host_name, $result) = each %build_result)
> {
> printf "%s\t%s\n", $host_name, $result;
> }
>

my hunch was that your problem occurred because you're deleting items
from the hash while iterating over it, but on second thought I'm not so
sure.

anyway, I changed up your script a bit (no more deleting) and it seems
to run ok for me. give it a try.

-jp

#!/usr/bin/perl

use strict;
use warnings;

use Async;
use List::MoreUtils qw{ any };

my @hosts = (
{ architecture => 'rhel3-ia32',
name => 'gray',
},
{ architecture => 'rhel4-ia32',
name => 'x.x.x.x',
},
{ architecture => 'sles9-ia32',
name => 'y.y.y.y',
},
{ architecture => 'sles10-ia32',
name => 'maroon',
},
{ architecture => 'rhel3-x64',
name => 'white',
},
{ architecture => 'sles9-ia64',
name => 'goldenrod',
},
{ architecture => 'rhel4-s390',
name => 'rh4as',
},
{ architecture => 'sles9-s390',
name => 'mambo',
},
{ architecture => 'sol10-x64',
name => 'brown',
},
{ architecture => 'sol10-sparc',
name => 'green',
},
);

for my $hashref (@hosts) {
$hashref->{process} = Async->new(
sub { long_running_computation($hashref->{name}) }
);
}

sleep 10 while ( any { not $_->{process}->ready() } @hosts);

print "\n";
for my $hashref (@hosts) {
printf "Host %-10s ", "$hashref->{name}:";

if( my $error = $hashref->{process}->error() ) {
print "Failure: '$error'";
}
else {
print $hashref->{process}->result();
}

print "\n";
}

sub long_running_computation {
my ($host_name) = @_;

#print "Process for host $host_name spawned...\n";
sleep 10 + rand 35;
#print "Process for host $host_name returning...\n";

return "$host_name result";
}

__END__

Todd English

2006-10-03, 6:58 pm

Thank you for the reply.

> my hunch was that your problem occurred because you're deleting items
> from the hash while iterating over it, but on second thought I'm not so
> sure.

I originally thought this as well, and had tried a couple of variants
to the code I posted where I didn't mess with the proc hash until all
procs had reported their return. But this didn't make any difference
and upon further reflection I didn't think this was the case.

> anyway, I changed up your script a bit (no more deleting) and it seems
> to run ok for me. give it a try.

I tried your example code 10 times on three different machines and I
don't see any difference between your output and mine. For example I
just ra the example and received:
Host gray:
Host x.x.x.x: x.x.x.x result
Host y.y.y.y: y.y.y.y result
Host maroon:
Host white:
Host goldenrod:
Host rh4as:
Host mambo:
Host brown:
Host green: green result

I'm wondering if this is environmental. Are you on a windows machine,
or a UNIX like machine? What version of perl are you using?

Thanks for the help,
Todd English

Todd English

2006-10-04, 3:57 am

> But you will in the present?

Sorry Tad, I misunderstood you.

> Google is not Usenet.

Very astute observation. I use Google to post because it is convenient.
However both my work and home isp's news servers list perl.beginners as
a valid group. I readily accept that yours doesn't and you couldn't
post your TOFU comments to a nonexistent group.

> No, else I would have given one.
>
> I don't have that module installed, and don't have the time to try it.

Well, thank you for taking the time to give me some Usenet etiquette
advice.

-T

DJ Stunks

2006-10-04, 3:57 am

Todd English wrote:
> Thank you for the reply.

Please don't snip attributions. That is, don't snip the part that says
what I wrote. Otherwise, who are you thanking?

>
> I originally thought this as well, and had tried a couple of variants
> to the code I posted where I didn't mess with the proc hash until all
> procs had reported their return. But this didn't make any difference
> and upon further reflection I didn't think this was the case.
>
>
> I tried your example code 10 times on three different machines and I
> don't see any difference between your output and mine. For example I
> just ra the example and received:
> Host gray:
> Host x.x.x.x: x.x.x.x result
> Host y.y.y.y: y.y.y.y result
> Host maroon:
> Host white:
> Host goldenrod:
> Host rh4as:
> Host mambo:
> Host brown:
> Host green: green result

This is the results from the _exact_ script I posted, or did you modify
it somehow?

Did you uncomment those diagnostic prints in
long_running_computation()? Did you see all your processes spawned,
and all of them eventually return?

> I'm wondering if this is environmental. Are you on a windows machine,
> or a UNIX like machine? What version of perl are you using?

That was on win32: Perl v5.8.7; Async v0.10.

-jp

PS - by the way, upon further reflection I decided I would change:

sleep 10 while any{ not $_->{process}->ready() } @hosts;

to

sleep 10 while notall{ $_->{process}->ready() } @hosts;

but that's a style issue (and one less keystrokes to boot) :-)~

Sponsored Links