Passing an IO:Handle to a Thread

See embedded:

Chris Hall wrote:
> With your indulgence, I s enlightenment...
>
> I have looked at the documentation and studied various modules,
> including Thread::Queue, Thread::Queue::Any, Thread::Conveyor, ...  I
> have read through the archive for this mailing list.
>
> I am failing to grasp how stuff can be shared or passed between threads,
> particularly Objects in general and IO::Handle objects in particular.
>
> IO:: stuff doesn't say it's thread-safe -- but it doesn't say it isn't.
> If threads and IO are incompatible that would be a serious deficiency
> (Shirley?).
>
> I can see that scalars, arrays and hashes can be shared, provided they
> are either contain simple values or references to other shared scalars,
> arrays and hashes.  I see examples of packaging up complex data to be
> able to store it as a simple scalar.
>
> 1. how Objects fit into this in general.
>
>    I note advice that the contents of an Object may be passed between
>    threads, provided the receiving thread re-blesses the received data.
>
>    I can see that Thread:Queue creates a shared array and then blesses
>    it.  For several threads to use the queue one needs to pass the
>    object.  If the queue object (reference) is stored in a shared
>    scalar, will that work ?  Does each thread need to bless the thing ?
>
>    Does an object shared in this way get DESTROYed once or once per
>    thread ?
>

Note that its *not* required to pass the the queue - if you create the
queue *before* the threads, the threads automagically get a CLONEd
version of the queue (like process fork(), which ithreads attempts
to emulate).

As to shared object destruction:

pre-5.10.0: once per thread, requires some extra bookkeeping to make sure
it "does the right thing"

5.10.0: once in the last referencing thread (fixed in CORE)

> 2. GLOBS in general and IO::Handle objects in particular.
>
>    I find that I cannot do: my $FH :shared ;  open $FH, "...." ;
>
>                        nor: my $FH ;  open $FH, "...." ;
>                             my $sFH :shared = $FH ;
>
>    I suppose I can pass $FH->fileno and use fdopen in the worker thread.
>
>    If so, I've got two IO::Handle objects in separate threads, both
>    capable of doing stuff...
>
>    What happens when I close one of the IO::Handles ?  Or one is
>    DESTROYed ?

Yes, you have to pass the fileno and recreate the handle object
in the recving thread. As this is essentially a dup() operation, each fileha
ndle
is independent of the others (other than they access the same I/O
target). Which is fine for block I/O, but for streams (e.g., sockets),
its best to make sure only 1 thread is using the handle. PS: if passing
a socket from a listener to a child worker thread, be sure the child
got the socket reopened before closing in the listener!

>
> 3. Cannot assign sub addresses to a shared variable.
>
>    If find that I cannot do: my $r_sub :shared = \&sub...
>
>    which requires some other work around ?

My first question is "Why would you want to do that ?"

The reason it doesn't work is rather lengthy, but in general,
keep in mind that all shared variables are actually just proxies
to the real version maintained in a global shared interpretter context.
Which means that any reference assigned to a shared variable must
also be shared (ie, reside in the shared interpretter). Since closures
are references, assigning a closure to such a variable would entail trying t
o
resurrect the entire closure context inside the shared interpretter
- which could be *very* challenging to do.

FWIW: Thread::Apartment supports proxied closures to pass closures
between threads, but the closure resides and executes
inside its originating thread.

>
> 4. More specifically -- IO::Socket objects.
>
>    Assuming a listening socket in one thread, but each incoming
>    connection should be handled in a separate thread...  The accept
>    action:
>
>       $connection_socket = $listen_socket->accept()
>
>    requires that the connection thread can be passed either the
>    $connection_socket or the $listen_socket.
>
>    Assuming the fdopen trick works, what about all the properties held
>    in the hash is the IO:Socket object ?

I've used this effectively wo/ issues in a number of apps.
I believe HTTP::Daemon::Threaded has code for that, you might
want to review it. (but I could be misremembering; ly, its been awhile
since I touched it)

>
> 5. Arguments passed to new thread and return values from a thread.
>
>    It would appear that these values are cloned.  Is that the case ?

I'll leave that to Jerry H. to explain ;^)

>
> 6. Objects which are cloned either when a thread is created or passed as
>    an argument.
>
>    Cloned objects are apparently independent and hence will be DESTROYed
>    separately.  This appears straightforward -- but what happens to
>    shared objects ?

See note above wrt pre-5.10 vs. 5.10+.

>
>    Cloned IO::Handle or IO::Socket objects must somehow be linked to the
>    same thing.  What happens when the first clone object is closed or
>    DESTROYed.

*If* the receiver has already re-opened the handle, then the parent
close if safe. See note above.

>
> It may be that 5.10.0 may be better than 5.8.8 at these things ?
>
> Thanks,
>
> Chris

HTH,
Dean Arnold
Presicient Corp.

On Wed, Feb 27, 2008 at 08:15:36AM -0800, Dean Arnold wrote:
> As to shared object destruction:
>
> pre-5.10.0: once per thread, requires some extra bookkeeping to make sure
> it "does the right thing"
>
> 5.10.0: once in the last referencing thread (fixed in CORE)

Huh?

use warnings;
use strict;

use threads;
use threads::shared;

sub DESTROY { warn "DESTROY $_[0]\n" }

my $h = bless {};
share $h;
threads->new(sub {})->join;

$ perl5100t /tmp/p
DESTROY main=HASH(0x98fb844)
DESTROY main=HASH(0x98748c4)
$


--
I before E. Except when it isn't.

Dave Mitchell wrote:
> On Wed, Feb 27, 2008 at 08:15:36AM -0800, Dean Arnold wrote: 
>
> Huh?
>
>     use warnings;
>     use strict;
>
>     use threads;
>     use threads::shared;
>
>     sub DESTROY { warn "DESTROY $_[0]\n" }
>
>     my $h = bless {};
>     share $h;
>     threads->new(sub {})->join;
>
> $ perl5100t /tmp/p
> DESTROY main=HASH(0x98fb844)
> DESTROY main=HASH(0x98748c4)
> $
>
>

Hmmm, it looks like the PL_destroyhook change in CORE
made it into 5.10.0, but the version of threads::shared
that uses it didn't. It looks like the CPAN version has
it...or am I missing something else ?

Dean Arnold

On Fri, Feb 29, 2008 at 07:38:48PM -0500, Jerry D. Hedden wrote:
> Dean Arnold wrote: 
>
> That's exactly correct.  You need 5.10.0 + threads::shared from CPAN.

Ah, PL_destroyhook had completely passed me by.

I've just had a look at the bleed version of threads::shared, and there
appears to be a destroyhook race condition:

If both threads try to free a shared object (RC=2) around the same time,
they both call Perl_shared_object_destroy, both conclude they shouldn't
call DESTROY since RC > 1, and thus the object is freed with no-one
calling the destructor.

--
The Enterprise's efficient long-range scanners detect a temporal vortex
distortion in good time, allowing it to be safely avoided via a minor
course correction.
-- Things That Never Happen in "Star Trek" #21

On Sat, Mar 01, 2008 at 02:51:19PM +0000, Chris Hall wrote:
> Whatever share $h does, it doesn't appear useful when $h is a ref to
> something.
>
> Whatever my $h = &share({}) does, it's not the same as
>          my %b : shared ; my $h = \%b ;
>
> Whatever &share({a => 1}) does, it doesn't return a ref to an anonymous
> hash containing a => 1 !!

Share is very similar to tie; when you share a container, its original
contents become hidden/lost


--
The Enterprise is involved in a bizarre time-warp experience which is in
some way unconnected with the Late 20th Century.
-- Things That Never Happen in "Star Trek" #14