Thread-safe reference counts.

I have some code where objects are dynamically allocated by some
thread, and then used by multiple threads and freed when they are no
longer needed. I think that reference counting is the most appropriate
way to handle this in my situation.

However, my code currently has a very obvious problem in it. Here is
how I have reference counting implemented for my objects, basically
(sorry about any errors I am typing this here in this message):

=== BEGIN CODE ===

class A {
public:
A ();
void AddRef ();
void Release ();
private:
~A ();
int refs_; // reference count
pthread_mutex_t mtx_; // protects refs_
};

// constructor inits reference count to 1
A::A ()
: refs_(1), mtx_(PTHREAD_MUTEX_INITIALIZER)
{
}

// increment count
void A::AddRef () {
pthread_mutex_lock(&mtx_);
++ refs_;
pthread_mutex_unlock(&mtx_);
}

// decrement count, destroying object at 0.
void A::Release () {
bool deleteme;
pthread_mutex_lock(&mtx_);
-- _refs;
deleteme = (_refs == 0);
pthread_mutex_unlock(&mtx_);
// <--- and here is the problem!
if (deleteme)
delete this;
}

=== END CODE ===

The problem is in Release(). There is a short period of time where the
mutex is unlocked but the object is already doomed for destruction; if
another thread calls AddRef() during that time, that other thread now
has a pointer to garbage and doesn't know it.

I can't delete this before unlocking the mutex. There is also a second
problem, where even if, hypothetically speaking, I could do the delete
inside the mutex lock in release, there's still the problem where
another thread may have called AddRef() in the mean time, and it is
blocking on that mutex, and by the time Release() returns if the
reference count had decreased to 0, the object is deleted, the thread
blocking in AddRef() continues, and operates on the object that's
already been deleted.

So I have two questions:

1) I have no specific reason for implementing this all by hand. I am
not completely familiar with boost, though. Is there some boost thread-
safe reference counting thing that I can use to take care of this all
for me?

2) In any case, how can I fix the above problems in my own code? I
can't quite get my head around it, it seems like it's not possible to
do reference counting from "within" the object; that I need some kind
of external "wrapper" around it instead. Also I am unsure about the
logic that I'd need to use to protect against the case where AddRef
blocks on the mutex, Release destroys the object, then AddRef
continues on a deleted object. Really I guess what I said in #1 was
kind of a lie: I certainly wouldn't *mind* not using boost, as I'm
currently not using boost for anything else in this project, and
installing boost on all the development machines is a *minor* hassle
(nothing I can't overcome, but I don't mind implementing this by hand
at this stage in this particular project, at least).

Thanks,
Jason

On Mar 26, 8:50 pm, "jason.cipri...@gmail.com"
<jason.cipri...@gmail.com> wrote:
> Is there some boost thread-
> safe reference counting thing that I can use to take care of this all
> for me?

I see boost::shared_ptr, which appears to do exactly what I want. I'll
just accept the fact that it works by magic and move on with my life.

Jason

Thanks for the detailed reply, Sam. I appreciate it and it confirms a
lot of my suspicions.

On Mar 26, 10:00 pm, Sam <s...@email-scan.com> wrote:
> That's what Boost's shared_ptr does. Although -- much like the rest of
> Boost's code -- it is horribly inefficient, and suffers from certain desig
n
> flaws -- it is often a good starting point for your own reference-counted
> pointers.

I can't comment on shared_ptr's efficiency since I have no experience
with it. Blinding efficiency is not important for this particular
application, however, but as far as "certain design flaws"... is there
anything in particular that I should watch out for? I'm not doing
anything weird, I have STL containers of shared_ptr<Object>'s, I
sometimes return shared_ptr<Object>'s from functions, other than that
it's pretty basic stuff. So far in my shared_ptr experiments it seems
to be well-behaved; although using "ptr.reset(new Object)" is a little
strange when you are used to "ptr = new Object" -- but it makes sense
(similarily, "thelist.push_back(MyPtr(new Object))" as opposed to
"thelist.push_back(new Object)" -- but I'm thankful for the explicit
constructor, it's helping me catch some mistakes).


> You need to wrap your brain around the concept that /every/ time a pointer
> to the object gets instantiated, the reference count gets increased, and
> /every/ time a pointer to the object goes out of scope, the reference coun
t
> gets decreased, so when the reference count goes to 0, by definition, no
> more pointers to the object exist and it's safe to delete it.

Thanks for explaining it this way; I had been thinking about it on a
wider scope, like "when this thread gets a pointer, the reference
count is incremented, and when this thread is done with it, decrement
it" -- which was confusing the heck out of me.

> [snip] Define and implement your own
> pointer class, and use this as a learning experience. True, your reference
> counting implementation will be much slower than Boost's.[snip]
> But this is a good
> way to learn some important principles of thread-safe programming.

Definitely; at least for a learning experience, I was planning on
doing this at the end of this project. I appreciate the shared_ptr's
magic but it's always good to know what's going on under the hood.

Thanks again,
Jason

[comp.programming.threads added]

<jason.cipriani@gmail.com> wrote in message
news:66accfe0-0a27-4da4-828b-01450b2ea3a3@s13g2000prd.googlegroups.com...
>I have some code where objects are dynamically allocated by some
> thread, and then used by multiple threads and freed when they are no
> longer needed. I think that reference counting is the most appropriate
> way to handle this in my situation.
>
> However, my code currently has a very obvious problem in it. Here is
> how I have reference counting implemented for my objects, basically
> (sorry about any errors I am typing this here in this message):
[...]

Your question about another thread calling AddRef while one is calling
Release and dropping to zero means that you need strong thread-safety which
shared_ptr does not provide. Here are some implementations you can take a
look at:


Joe Seighs work:

http://atomic-ptr-plus.sourceforge.net/

http://groups.google.com/group/comp...br />
da537d8f



My work:


http://appcore.home.comcast.net/~appcore/vzoom/refcount


http://groups.google.com/group/comp...br />
eff4a466
(100% portable version (e.g., POSIX Threads)



Here is some information on strong thread-safety:

http://groups.google.com/group/comp...br />
d32340c6



If you have any questions, please feel free to ask. I know that we can work
out a soultion.

"Chris Thomasson" <cristom@comcast.net> wrote in message
 news:E6idnXXztKuPvnbanZ2dnUVZ_tajnZ2d@co
mcast.com...
> [comp.programming.threads added]
>
> <jason.cipriani@gmail.com> wrote in message
> news:66accfe0-0a27-4da4-828b-01450b2ea3a3@s13g2000prd.googlegroups.com... 
> [...]
>
> Your question about another thread calling AddRef while one is calling
> Release and dropping to zero means that you need strong thread-safety
> which shared_ptr does not provide. Here are some implementations you can
> take a look at:
[...]

Here is context that I accidentally snipped:


<jason.cipriani@gmail.com> wrote in message
news:66accfe0-0a27-4da4-828b-01450b2ea3a3@s13g2000prd.googlegroups.com...
{
The problem is in Release(). There is a short period of time where the
mutex is unlocked but the object is already doomed for destruction; if
another thread calls AddRef() during that time, that other thread now
has a pointer to garbage and doesn't know it.

I can't delete this before unlocking the mutex. There is also a second
problem, where even if, hypothetically speaking, I could do the delete
inside the mutex lock in release, there's still the problem where
another thread may have called AddRef() in the mean time, and it is
blocking on that mutex, and by the time Release() returns if the
reference count had decreased to 0, the object is deleted, the thread
blocking in AddRef() continues, and operates on the object that's
already been deleted.
}


According to the text above, you need strong thread-safety; period. You not
following the rule of basic thread-safety which says that a thread _cannot_
acquire a reference to an object that it does not already own a reference
to. If your application does not follow that rule, then you need something
like Joe's excellent atomic_ptr.

"Chris Thomasson" <cristom@comcast.net> wrote in message
 news:E6idnXXztKuPvnbanZ2dnUVZ_tajnZ2d@co
mcast.com...
> [comp.programming.threads added]
>
> <jason.cipriani@gmail.com> wrote in message
> news:66accfe0-0a27-4da4-828b-01450b2ea3a3@s13g2000prd.googlegroups.com... 
> [...]
>
> Your question about another thread calling AddRef while one is calling
> Release and dropping to zero means that you need strong thread-safety
> which shared_ptr does not provide. Here are some implementations you can
> take a look at:

Jason, when you get some time, I would advise you to go ahead and read
through the following paper:

http://citeseer.ist.psu.edu/cache/p...s01lockfree.pdf

This should clear up all of your questions. If not, well,
'comp.programming.threads' is there to help you. IMHO, the traffic in that
group has been sort of sparse lately; we welcome any of your subsequent
queries!

:^)

jason.cipriani@gmail.com wrote:
> On Mar 26, 8:50 pm, "jason.cipri...@gmail.com"
> <jason.cipri...@gmail.com> wrote: 
>
> I see boost::shared_ptr, which appears to do exactly what I want. I'll
> just accept the fact that it works by magic and move on with my life.

Alot of libraries do this.  I implemented it in Austria C++.

at::PtrTarget_MT provides the definition of the thread safe reference
counted base class.
(defined on line 210)
http://austria.svn.sourceforge.net/...ew=markup#l_210

It's also portable across Windows and Linux(x86 +amd64).

Chris Thomasson wrote:
> [comp.programming.threads added]
>
> <jason.cipriani@gmail.com> wrote in message
> news:66accfe0-0a27-4da4-828b-01450b2ea3a3@s13g2000prd.googlegroups.com...
> 
>
> [...]
>
> Your question about another thread calling AddRef while one is calling
> Release and dropping to zero means that you need strong thread-safety
> which shared_ptr does not provide. Here are some implementations you can
> take a look at:

This just means, that you are making a copy of an object where the
destructor is in progress. That's simply a bug and should be avoided ;-)

best regards,
Torsten

--
kostenlose Wirtschaftssimulation: http://www.financial-rumors.de

In article <cone.1206583245.205953.6601.500@commodore.email-scan.com>,
Sam  <sam@email-scan.com> wrote:
>-=-=-=-=-=-
>
>That's what Boost's shared_ptr does. Although -- much like the rest of
>Boost's code -- it is horribly inefficient, and suffers from certain design
>flaws -- it is often a good starting point for your own reference-counted
>pointers.

[...]

>Then don't. Resist the temptation to use Boost as long as you can. Many
>years from now, you'll be thankful for that. Define and implement your own
>pointer class, and use this as a learning experience. True, your reference
>counting implementation will be much slower than Boost's. Boost does not us
e
>mutexes for this, rather it uses CPU-specific atomic increment/decrement
>instructions (and gets it partially wrong, by the way). But this is a good
>way to learn some important principles of thread-safe programming.

Care to clarify?

And does this also apply to std::tr1::shared_ptr which is based on
boost impelmentation on many platfroms.

Yan