ithreads - Issues and doubts about ithreads

Author

Issues and doubts about ithreads

Martín Ferrari

2005-06-08, 9:03 pm

Hi,

I am in the process of writing an application which is a central
manager which should do a lot of IPC with databases, worker processes
(already written in C, receiving commands via UNIX sockets), and user
control via another UNIX socket.

Initially I tried using nonblocking IO, but I found a lot of headaches
finding out the state of the different filehandles (I think they
aren't well suited for nonblocking IO). The next step would be to use
select loop and low-level IO, but it scared me a bit :)

Also I tried POE, but didn't have nothing near good performance.

Then threads came as a natural solution as I could have different subs
doing blocking IO without worring (a similar approach I used for the
workers, many of them blocking, not consuming CPU and being very tiny
in size). But then I found out the new ithreads, which seem to me
unacceptably big, but since I will use just 4 or 5, I can cope with
that.

My first problem is, I don't understand yet the inner workings, and
I've seen different behaviors in debian sarge and woody; but the most
troubling thigs is I find dead IPC shared memory blocks which
sometimes I can't even free with ipcrm!!

I don't understand why use IPC in the first place, there isn't any
option to use real shared memory (they are threads!), or
shm_open+mmap? (not sure if it is possible under perl)

Also, I found today the Elizabeth Mattijsen's forks.pm module, and
read that it uses TCP communication (why not UNIX, or -again- mmap?).
Anyway, that seems to me the best approach, as it would be lighter
than ithreads, and I just need to pass messages and a couple of shared
variables across threads.

I read a little of the code but couldn't understand anything :). I'm
very worried about performance issues, and some hints would be useful
before deciding what to use for the project.

Having spent many hours trying to figure out what to do, there is very
little stopping me from trying a complete C solution, but that would
be painful! Please, give me some hints so I can still be a lazy perl
lover :)

Thanks a lot for your time!

--=20
Mart=EDn Ferrari

Paul Archer

2005-06-08, 9:03 pm

Elizabeth Mattijsen

2005-06-08, 9:03 pm

At 12:18 PM -0300 1/21/05, Mart=EDn Ferrari wrote:
>Also, I found today the Elizabeth Mattijsen's forks.pm module, and
>read that it uses TCP communication (why not UNIX, or -again- mmap?).

Patches are welcome. ;-)

Also: I''ve recently become to appreciate=20
memcached, and was wondering whether that would=20
be a good alternative for setting and fetching=20
shared values.

Unfortunately I don't have the tuits until at least the beginning of March.

>I read a little of the code but couldn't understand anything :).

Is there anything in particular that you don't understand more than the rest=
?

>I'm very worried about performance issues, and some hints would be useful
>before deciding what to use for the project.

Using TCP/IP will add latency and some CPU=20
overhead to your inter-thread communication. How=20
this offsets with using a lot less memory, is=20
really dependent on your situation. Basically I=20
would say, don't expect forks.pm to be faster=20
than threads once threads have started. On the=20
other hand, for any serious program (with any=20
serious number of modules loaded), there's a good=20
chance that forks.pm will be faster in starting a=20
thread.

>Having spent many hours trying to figure out what to do, there is very
>little stopping me from trying a complete C solution, but that would
>be painful! Please, give me some hints so I can still be a lazy perl
>lover :)

If speed is what you ultimately want, then C would be the way.

If you don't need any inter-thread locking (which=20
may happen in some applications), then maybe a=20
fork / memcached combo would be a solution if you=20
have a large amount of shared data. See=20
Cache::Memcached on CPAN.

Hope this helps.

Liz

Sponsored Links