VC STL - Spent all day tracking down this VC8/VC7.1 STL bug.

Author

Spent all day tracking down this VC8/VC7.1 STL bug.

2006-03-12, 7:00 pm

(crossposted from GameDev.net fourms)

So here it is. Tried it with both compilers. First I thought it was a
problem with my program, but alas it is not, and I've boiled the bug down to
a couple of small source files. Basically, it has to do with locale, DLLs,
and a situation dinkum never expected, I suppose.

Here's the first source file. This is the EXE and we'll call it
"client.exe":

#define WIN32_LEAN_AND_MEAN
#include <windows.h>
#include <fstream>

typedef unsigned char byte;

int main()
{
std::basic_fstream<byte> foo;
HMODULE h = LoadLibrary( "server.dll" );
FreeLibrary(h);
}

The second source file is for the DLL. We'll call it "server.dll":

#include <fstream>

typedef unsigned char byte;

// it doesn't matter if this is static or if
// it's instantiated on the stack or the heap,
// the result is the same.
std::basic_fstream<byte> bar;

// this will force the linker to produce a DLL
__declspec(dllexport) void TestMethod()
{
}

Now, when these are compiled in "Multithreaded DLL" or more specifically in
my case "Debug Multithreaded DLL", the client will crash post-main. It
appears to be doing this because it's not reference counting the "facet" for
the byte type correctly, and it's trying to delete the "facet" object twice!

When you change the type from byte to a type it expects, e.g., char,
wchar_t, or unsigned short, it doesn't crash.

In my code, I had changed my stream type to byte, as shown in this example,
because I'm reading UTF-8 data and I thought it made the most sense,
considering that libxml also treats UTF-8 data as "unsigned char" as its
xmlChar type. Since I'm reading data into libxml from files using fstream I
thought it would make sense to use basic_fstream<unsigned char> as a type.

So the lesson here is that using VC8/VC7.1 you simply can't reliably use
basic_fstream<byte> or presumably any stream type with a non-character
type... I tried 'int' and 'unsigned' and the results were the same. Traceing
through the code reveals that it automatically creates the facet objects for
what it expects to be the basic character types char, wchar_t, and unsigned
short.

I haven't tried this on gcc yet; it looks like a MS/dinkum specific issue.

/sigh

P.J. Plauger

2006-03-12, 7:00 pm

"JM" <asdlkfj@oiajsdf.com> wrote in message
news:e50Rf.16784$733.1242@tornado.texas.rr.com...

> (crossposted from GameDev.net fourms)
>
> So here it is. Tried it with both compilers. First I thought it was a
> problem with my program, but alas it is not, and I've boiled the bug down
> to a couple of small source files. Basically, it has to do with locale,
> DLLs, and a situation dinkum never expected, I suppose.
>
> Here's the first source file. This is the EXE and we'll call it
> "client.exe":
>
>
> #define WIN32_LEAN_AND_MEAN
> #include <windows.h>
> #include <fstream>
>
> typedef unsigned char byte;
>
> int main()
> {
> std::basic_fstream<byte> foo;
> HMODULE h = LoadLibrary( "server.dll" );
> FreeLibrary(h);
> }
>
>
>
> The second source file is for the DLL. We'll call it "server.dll":
>
>
> #include <fstream>
>
> typedef unsigned char byte;
>
> // it doesn't matter if this is static or if
> // it's instantiated on the stack or the heap,
> // the result is the same.
> std::basic_fstream<byte> bar;
>
> // this will force the linker to produce a DLL
> __declspec(dllexport) void TestMethod()
> {
> }
>
>
>
> Now, when these are compiled in "Multithreaded DLL" or more specifically
> in my case "Debug Multithreaded DLL", the client will crash post-main. It
> appears to be doing this because it's not reference counting the "facet"
> for the byte type correctly, and it's trying to delete the "facet" object
> twice!

More likely, it's allocating static counters a couple of different
places, once in the DLL and once in the .EXE. But that's just a
guess. It's not guess that you have to be *very* careful stuffing
something as complex as a locale facet into a DLL.

> When you change the type from byte to a type it expects, e.g., char,
> wchar_t, or unsigned short, it doesn't crash.

Right, because we (Microsoft and Dinkumware) have shaken down the
use of the common facets in DLLs over several releases now. It
wasn't easy.

> In my code, I had changed my stream type to byte, as shown in this
> example, because I'm reading UTF-8 data and I thought it made the most
> sense, considering that libxml also treats UTF-8 data as "unsigned char"
> as its xmlChar type. Since I'm reading data into libxml from files using
> fstream I thought it would make sense to use basic_fstream<unsigned char>
> as a type.

No real need to do that -- char is sufficiently transparent for eight-bit
bytes. And there are *lots* of other considerations you haven't tripped
over yet.

> So the lesson here is that using VC8/VC7.1 you simply can't reliably use
> basic_fstream<byte> or presumably any stream type with a non-character
> type...

In a DLL, perhaps, with naive code. We do it all the time.

> I tried 'int' and 'unsigned' and the results were the same. Traceing
> through the code reveals that it automatically creates the facet objects
> for what it expects to be the basic character types char, wchar_t, and
> unsigned short.
>
> I haven't tried this on gcc yet; it looks like a MS/dinkum specific issue.

Works fine with our libraries on a gazillion platforms.

P.J. Plauger
Dinkumware, Ltd.
http://www.dinkumware.com

Hendrik Schober

2006-03-30, 7:07 pm

P.J. Plauger <pjp@dinkumware.com> wrote:
> [...]
>
> Works fine with our libraries on a gazillion platforms.

AFAIK this bug generally plagues plugin architectures.
(For example, I happen to know that Acrobat plugins
are prone to this. InDesign plugins as well.)
ISTR that it's caused by some function decoration being
wrong. As these decorations are probably put into the
std lib by MS, you won't see it on other platforms. I
had sent the following analysis (which wasn't done by
me, BTW) to Marty Lovell (of MS) on 2004-10-27:

> [...] Basically, std::isspace() (and perhaps
> other locale-dependent calls) will call locale::facet::_Register(), which
> registers with the C++ runtime. This registration is torn-down when the
> runtime terminates, not necessarily when the module that initiated it
> terminates. Thus, since <some EXE> calls FreeLibrary on <some DLL>, that dll
> is still registered with the C++ runtime and will crash when the runtime is
> torn down and tries to perform unregistration tasks. A sample app has been
> written that exhibits this problem and it has been sent to Microsoft.
> [...]
> I crashed it in my debug build. It appears that in the file xdebug
> _DebugHeapDelete() tries to call through a virtual dtor (_Ptr->~_Ty()) and
> crashes because the vtable memory is gone. The vtable memory was in
> <some DLL> which has been unloaded at this point. It looks like
> locale::facet::_Register() is called underneath std::isspace() which is
> called by <some func>. This method slaps the this pointer on a linked list
> that is enumerated by _Fac_tidy() which by virtue of a call to _Atexit() is
> called when the shared CRT library is unloaded. So the problem is that an
> object with a vptr into code in <some DLL> is placed on a linked list that is
> walked and the dtor for the object is called after <some DLL> is unloaded.

I had also later sent him a sample app which repros
this. Marty later acknowledged this to be a bug in
Whidbey and said he had logged it as such into some
internal bug system.

I had hoped that, given the time it was reported by
me (and I wasn't even the first one who reported it
to MS), and the fact that virtually /all/ plugin
architectures are affected by this, MS would have
found the time to change that function decoration
for VC8, but so far I hadn't checked myself yet.

The only workaround that I know of is to use the
static RTL in the DLL. Of course, this will bloat
the DLL and slow down its loading... <sigh>

HTH somehow.

> P.J. Plauger

Schobi

--
SpamTrap@gmx.de is never read
I'm Schobi at suespammers dot org

"The sarcasm is mightier than the sword."
Eric Jarvis

Sponsored Links