PERL Beginners - Moving up, processes & threads

Author

Moving up, processes & threads

Chris Knipe

2006-01-30, 6:56 pm

Hi,

Last time I checked, perl's threads wasn't very popular to use. Now that's
a discussion on it's own I guess, and not the intensions of this email to
get into. I'm planning to develop a rather large perl application. Due to
complexity, I plan to run multiple processes, each process being spawned
from a single main process... Is there any way that I can share data
between them?

Say, uhm...
Main_Proc.pl
Sub_Proc1.pl - $var = blah
Sub_Proc2.pl - I want to use $var

The main thing is that they are all running on their own perl interpreter.
I therefore don't believe that it would be so easy to do. On the other
hand, I've seen this being done before on some really large perl
applications we used - unfortunately, I was not part of the development
group, so I don't know how / what they did.

In VB for example, I would just use a single class that is shared between
multiple threads. Each thread can then for example set or alter any
variable inside the class, and the change would reflect obviously on the
other threads utilising that class. But - that's threads, not processes.
Can something similar be used in Perl? Say, multiple processes using the
same module perhaps?

Is there perhaps somewhere I can read up on things like this with some good
examples and documentation?

Thanks,
Chris

Tom Phoenix

2006-01-30, 9:55 pm

On 1/30/06, Chris Knipe <savage@savage.za.org> wrote:

> Last time I checked, perl's threads wasn't very popular to use. Now that=
's
> a discussion on it's own I guess, and not the intensions of this email to
> get into. I'm planning to develop a rather large perl application. Due =
to
> complexity, I plan to run multiple processes, each process being spawned
> from a single main process... Is there any way that I can share data
> between them?

Lots of ways. Here's one that's worked for me. The shared data live in
some convenient location, like a directory, a file, or a database,
depending upon your needs. A little glue code in a module handles
access to the data in a consistent way, maybe using something from the
Tie::* hierarchy to provide a simple interface.

Is that enough to get you started? Hope this helps!

--Tom Phoenix
Stonehenge Perl Training

Chris Knipe

2006-01-30, 9:55 pm

>> Last time I checked, perl's threads wasn't very popular to use. Now
>
> Lots of ways. Here's one that's worked for me. The shared data live in
> some convenient location, like a directory, a file, or a database,
> depending upon your needs. A little glue code in a module handles
> access to the data in a consistent way, maybe using something from the
> Tie::* hierarchy to provide a simple interface.

Hi,

Thanks for the suggestion. This has been recommended to me by someone off
the list as well (or something relatively close to it), and unfortunately is
not going to be very efficient. It's going to kill the system as far as
disk IO is concerned. I'm talking about 200+ variables here, about half of
which will change approximately every 10ms (some even less). Doing 100 odd
disk writes/reads every 10ms, plus more than likely searching through open
files for a specific variable, and closing it in time so that it can be
written again... Don't think it will be feasable.

I'll need to do this in memory I'm afraid... :(

--
Chris

Chas Owens

2006-01-30, 9:55 pm

On 1/30/06, Chris Knipe <savage@savage.za.org> wrote:
to[color=darkred]
ue[color=darkred]
ed[color=darkred]
>
> Hi,
>
> Thanks for the suggestion. This has been recommended to me by someone of=
f
> the list as well (or something relatively close to it), and unfortunately=
is
> not going to be very efficient. It's going to kill the system as far as
> disk IO is concerned. I'm talking about 200+ variables here, about half =
of
> which will change approximately every 10ms (some even less). Doing 100 o=
dd
> disk writes/reads every 10ms, plus more than likely searching through ope=
n
> files for a specific variable, and closing it in time so that it can be
> written again... Don't think it will be feasable.
>
> I'll need to do this in memory I'm afraid... :(
>
> --
> Chris

You might consider the following modules:

Cache::Memcached - client library for memcached (memory cache daemon)
POE - portable multitasking and networking framework for Perl
IPC::Shareable - share Perl variables between processes
Cache::RamDisk - Sharing of Perl Objects Between Processes on Several RAM D=
rives

Tom Phoenix

2006-01-30, 9:55 pm

On 1/30/06, Chris Knipe <savage@savage.za.org> wrote:

> Thanks for the suggestion. This has been recommended to me by someone of=
f
> the list as well (or something relatively close to it), and unfortunately=
is
> not going to be very efficient. It's going to kill the system as far as
> disk IO is concerned. I'm talking about 200+ variables here, about half =
of
> which will change approximately every 10ms (some even less). Doing 100 o=
dd
> disk writes/reads every 10ms, plus more than likely searching through ope=
n
> files for a specific variable, and closing it in time so that it can be
> written again... Don't think it will be feasable.
>
> I'll need to do this in memory I'm afraid... :(

Nothing about this suggestion precludes that.

One way to handle this would be for you to make a "data server" that
does, as you say, keep your 200 variables in memory. (It could, of
course, write those variables to a file when it shuts down, and read
them from the file when it starts up again. It could even be clever
enough to periodically save the data while it's running.) Every other
program communicates with the data server to work with the data,
probably via a module. The data server provides quick access to the
data, ensures that data updates can be made atomically when needed,
perhaps even putting restrictions on the data to ensure data
integrity.

Because there will be some overhead associated with opening a
connection to the data server, client programs will probably prefer to
open a single connection that stays open for the lifetime of the
application.

If you make your data server especially full-featured, you could call
it a database. But I know you don't want to use a database, because
that would be too slow for your needs, so feel free to call it
something else. :-)

Cheers!

--Tom Phoenix
Stonehenge Perl Training

Chris Knipe

2006-01-30, 9:55 pm

> IPC::Shareable - share Perl variables between processes

IPC! That's what we used :) Thanks,

--
Chris.

Sponsored Links