ithreads - Work Crew example

Author

Work Crew example

Wong, Danny H.

2008-04-24, 8:27 pm

Hi GURUS,
I'm trying to implement a threaded program using work crew model. I
was wondering if someone can provide an example. I'm trying to copy some
files to say 100 machines ( for this example... 100 directories would be
work as well) as I'm looking for a real example of how to use the work
crew model thread. thanks.

- - - - - cisco - - - - -
This e-mail and any attachments may contain information which is confidential,
proprietary, privileged or otherwise protected by law. The information issolely
intended for the named addressee (or a person responsible for delivering it to
the addressee). If you are not the intended recipient of this message, you are
not authorized to read, print, retain, copy or disseminate this message or any
part of it. If you have received this e-mail in error, please notify the sender
immediately by return e-mail and delete it from your computer.

Chris Fowler

2008-04-24, 8:28 pm

On Thu, 2008-04-24 at 16:23 -0700, Wong, Danny H. wrote:
> Hi GURUS,

Thanks for the compliment :)

> I'm trying to implement a threaded program using work crew model. I
> was wondering if someone can provide an example. I'm trying to copy some
> files to say 100 machines ( for this example... 100 directories would be
> work as well) as I'm looking for a real example of how to use the work
> crew model thread. thanks.
>

A while back I worked on a program that did heartbeat monitoring (PING)
to about 20 machines.

Now the logical thing to do would be create a list and work the list in
a single thread. The problem here is if 5 of those machine were down
then you would be delayed getting to the end of the list. I needed a
method to guarantee that each machine would be tested every 60s.

I did it with threads. I created a thread for each machine that would
send the ping and wait for its return. I then created a manager thread
whose job it was to send alerts.

I did all this in C but it can be done in Perl. In C I basically had a
shared memory segment where the workers could write values. The manager
would lock a semaphore and then look at the memory seeing any changes.
It would then send a message to a program that handle notifications.
After this it would unlock the semaphore.

In Perl you could attack this type of idea many ways. You could create
shared objects that store info like state, last ping, etc and then the
manager would look at those objects ever 60s and act. You could create
pipes and have the child threads send message back to the manager on
state changes and then he acts.

The whole idea of threads here is that each thread is responsible for
one machine and can hit that machine easily every 60 seconds. Now it
does not scale well. If you wanted to do 100 machines then I would not
execute 100 threads. I would probably do 10 threads with 10 machines
each. In reality a host should take no longer than 5s to respond to a
ping. Use Net::Ping and specify a timeout of 5 seconds. 5x10 is 50 so
you should still be able to hit the 60s requirement even if all 10 are
down.

Chris

Chris Fowler

2008-04-24, 8:28 pm

On Thu, 2008-04-24 at 17:06 -0700, Wong, Danny H. wrote:
> Thanks for your comments. Do you have a perl example on how it can be
> done? I've read comments like yours on the websites, but don't have a
> concrete example. I'm trying to figure out how to implement a copy
> function with work crew thread model. I'm learning how to use the
> power
> of threads and of course using it correctly...

Honestly I've not worked in perl threads in a long time. I got started
and ran into so many obstacles learning about interpreter threads vs
<flamebait>real threads</flamebait> that I gave up. I'm hoping that
Parrot may do <flamebait>the right thing(tm)</flamebait> and implement
some real threads.

Here is a ROUGH simple example

use Net::Ping;
use threads;

sub worker {
my $target = shift;

while(1) {
my $p = Net::Ping->new();
my $r = $p->ping($target, 5);
if($r == 0) {
print "$target:DOWN\n";
} else {
print "$target:UP\n";

}

$p->close();

}
}

# main program

pipe R,W;

for(my $x = 1; $x <= 10; $x++) {
threads->create("worker", "192.168.7.$x");
}

while(<R> ) {
print "$_";
}

I use pipes (IPC) for the workers to talk back to the manager.
You can replace thread with fork and still use this method. The program
is not smart enough to track changes in state, etc. That logic can go
in the workers or the manager.

Chris

Chris Fowler

2008-04-24, 8:28 pm

On Thu, 2008-04-24 at 17:21 -0700, Wong, Danny H. wrote:
> Thanks. I'll take a look at it and give it a go...
>
Bug

> print "$target:DOWN\n";
print W "$target:DOWN\n";
> } else {
> print "$target:UP\n";
print W "$target:UP\n";

Even with those changes the pipes don't work so well.

GUIs are a better example of threads. In a GUI you need threads to do
work and then update an area.

Have you ever written a Java Swing application that had not threads?
You could clock on the button for it to do something and the whole
display seemed to freeze. If you iconized it and then restored it, it
would just be a square on the screen. The program was too busy working
on that one request. In a GUI you should spawn a thread and have it
update the GUI components. It may be blocking in connect() in the
background but your application still looks responsive.

Sponsored Links