ithreads - Thread-safe perl XS modules

Author

Thread-safe perl XS modules

Thomas Busch

2007-06-26, 7:30 pm

Hi,

I'm the author of a perl XS module which is a wrapper around
a C++ library that is thread-safe. However in the
context of Perl is doesn't seem to be.

When I do the following:

===============================
#!/usr/bin/perl

use Lucene;
use threads;
use strict;

my $analyzer = new Lucene::Analysis::Standard::StandardAnal
yzer;

sub test {
print "pass\n";
};

my $thr = threads->create('test');
$thr->join();

undef $analyzer;

print "pass2\n";
exit(0);
===============================

I get:

pass
Segmentation fault

It seems there is a problem during the destruction of the
object that was created before threads->create. What
could be the cause of this problem ?

The source code of the XS module is here:
http://search.cpan.org/src/TBUSCH/Lucene-0.13/

What are the usual things to look out for when
writing thread-safe XS modules that use external
libraries ?

Thomas.

Dean Arnold

2007-06-26, 7:30 pm

Thomas Busch wrote:
> Hi,
>
> I'm the author of a perl XS module which is a wrapper around
> a C++ library that is thread-safe. However in the
> context of Perl is doesn't seem to be.
>
> When I do the following:
>
> ===============================
> #!/usr/bin/perl
>
> use Lucene;
> use threads;
> use strict;
>
> my $analyzer = new Lucene::Analysis::Standard::StandardAnal
yzer;
>
> sub test {
> print "pass\n";
> };
>
> my $thr = threads->create('test');
> $thr->join();
>
> undef $analyzer;
>
> print "pass2\n";
> exit(0);
> ===============================
>
> I get:
>
> pass
> Segmentation fault
>
> It seems there is a problem during the destruction of the
> object that was created before threads->create. What
> could be the cause of this problem ?
>
> The source code of the XS module is here:
> http://search.cpan.org/src/TBUSCH/Lucene-0.13/
>
> What are the usual things to look out for when
> writing thread-safe XS modules that use external
> libraries ?
>
> Thomas.
>

I didn't see any CLONE() method in your code (tho admittedly,
I didn't do a deep inspection, theres a lot of XS files there).
Perhaps you need a CLONE() method to cleanup some context in
the child thread's cloned copy of the StandardAnalyzer object ?
(Or maybe in the parent's version of the object ?)

In similar situations, I've had to tag resources with the thread id
in which they were created, and make sure they get destroyed
in that same thread, to avoid destruction in each
cloned instance. (Or use some other explicit tag to indicate
the resource has already been destroyed...but that could
require mutexes if multiple threads are concurrently
destroying)

Alternately, you might try using a threads::shared version
of the object, tho that will complicate your constructor.

You may eventually need to do a deeper debug of your segfault
to get exact lines where the fault occurs.

Dean Arnold
Presicient Corp.

Dean Arnold

2007-06-26, 7:30 pm

Thomas Busch wrote:
> ah I see. I just had a look at the threads.xs file.
> So basically perl_clone triggers a call to the CLONE()
> method of all variables ?
>
> Thomas.
>

*If* they have a CLONE() method, yes. Otherwise, it just
makes a copy of the variable (including contents of arrays/hashes),
fixing up references as it goes. So if your object has a field
thats actually a ptr to a C++ object, it gets copied as a scalar.
Which may cause your DESTROY() to invoke the C++ object's
destructor twice, or reference a dead ptr the 2nd time DESTROY()
is called.

Note that a thread::shared objects don't get cloned
(tho the thread-private "proxy" wrapper does), and its DESTROY
won't get called until the last referencing thread exits.

Dean Arnold
Presicient Corp.

Sponsored Links