Home > Archive > VC Language > January 2006 > std::string not calling my global operators new/delete
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
std::string not calling my global operators new/delete
|
|
| Daniel 2006-01-13, 7:05 pm |
| hi all,
i'm using Visual Studio 2005 to target WinXP and WM2003 and i've got
the following problem:
i'm overloading global new/delete and new[]/delete[] in order to use a
custom memory manager i wrote. this works fine with most classes, but
not with std::string.
when step into an assignment to std::string i find that:
under WinXP my global new/delete are ignored.
under WinCE my glnbal new/delete are used.
under WinXP a CRT operator new is called which has the following
signature (in new.cpp):
void *__CRTDECL operator new(size_t size) _THROW1(_STD bad_alloc)
any ideas on that?
thx,
Daniel
| |
| Victor Bazarov 2006-01-13, 7:05 pm |
| Daniel wrote:
> i'm using Visual Studio 2005 to target WinXP and WM2003 and i've got
> the following problem:
>
> i'm overloading global new/delete and new[]/delete[] in order to use a
> custom memory manager i wrote. this works fine with most classes, but
> not with std::string.
Doesn't 'std::basic_string' template has some kind of Allocator, which
can, if it wants to, have 'new' and 'delete' overloaded, and not use your
global ones?
> when step into an assignment to std::string i find that:
>
> under WinXP my global new/delete are ignored.
> under WinCE my glnbal new/delete are used.
So, it seems that for two different targets different 'Allocator'
implementations are used.
> under WinXP a CRT operator new is called which has the following
> signature (in new.cpp):
> void *__CRTDECL operator new(size_t size) _THROW1(_STD bad_alloc)
>
> any ideas on that?
On what?
V
| |
| Tim Roberts 2006-01-14, 3:58 am |
| Daniel <__daniel___@icg.tu-graz.ac.at> wrote:
>
>i'm using Visual Studio 2005 to target WinXP and WM2003 and i've got
>the following problem:
>
>i'm overloading global new/delete and new[]/delete[] in order to use a
>custom memory manager i wrote. this works fine with most classes, but
>not with std::string.
>
>when step into an assignment to std::string i find that:
>
>under WinXP my global new/delete are ignored.
>under WinCE my glnbal new/delete are used.
>
>under WinXP a CRT operator new is called which has the following
>signature (in new.cpp):
>void *__CRTDECL operator new(size_t size) _THROW1(_STD bad_alloc)
>
>any ideas on that?
Bring up the project properties, go to C/C++, go to Code Generation. Change
"Runtime Library" from "Multi-threaded DLL (/MD)" to "Multi-threaded
(/MT)".
When you use the DLL version of the run-time library, all the calls to
"operator new" within msvcp80.dll are satisfied by the standard "operator
new" in msvcr80.dll.
Also remember that the VC2005 implementation of std::string stores strings
of 16 characters or less directly in the string structure. No allocation
is required.
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.
| |
| Bo Persson 2006-01-14, 7:57 am |
|
"Daniel" <__daniel___@icg.tu-graz.ac.at> skrev i meddelandet
news:506gs15e17jue6h9596firqiuak2c9eoln@
4ax.com...
> hi all,
>
> i'm using Visual Studio 2005 to target WinXP and WM2003 and i've got
> the following problem:
>
> i'm overloading global new/delete and new[]/delete[] in order to use
> a
> custom memory manager i wrote. this works fine with most classes,
> but
> not with std::string.
std::string isn't using new[] and delete[], but the non-array new and
delete. That is part of the std::allocator spec.
Bo Persson
| |
| Tom Widmer [VC++ MVP] 2006-01-16, 8:05 am |
| Daniel wrote:
> hi all,
>
> i'm using Visual Studio 2005 to target WinXP and WM2003 and i've got
> the following problem:
>
> i'm overloading global new/delete and new[]/delete[] in order to use a
> custom memory manager i wrote. this works fine with most classes, but
> not with std::string.
>
> when step into an assignment to std::string i find that:
>
> under WinXP my global new/delete are ignored.
> under WinCE my glnbal new/delete are used.
>
> under WinXP a CRT operator new is called which has the following
> signature (in new.cpp):
> void *__CRTDECL operator new(size_t size) _THROW1(_STD bad_alloc)
>
> any ideas on that?
std::string is special in VC in that it (and wstring) are compiled into
the CRT DLL, which means you can't intercept their implementation,
including how they allocate memory. The easiest way around this is to
link statically with the CRT, so that std::string is compiled directly
into your source wherever you use it (via the usual template
instantiation rules), and thus uses your operator new. Think carefully
about how something was allocated before passing it across a DLL
boundary though (if you are using DLLs with std containers in the
interface).
Tom
| |
| Daniel 2006-01-17, 7:08 pm |
| thanks for all the information.
my solution now is to use another string implementation which is quite
similar to std::string (does not implement the whole featureset
though) that simply does not have this problem.
anyway, thanks for giving the insight on why this problem is there at
all,
Daniel
| |
| Tom Widmer [VC++ MVP] 2006-01-18, 4:00 am |
| Daniel wrote:
> thanks for all the information.
>
> my solution now is to use another string implementation which is quite
> similar to std::string (does not implement the whole featureset
> though) that simply does not have this problem.
If you're happy to use something other than std::string, how about :
typedef std::basic_string<char, std::char_traits<char>,
specialallocator<char> > mystring;
Seems better than using a non-standard string implementation.
For how to write specialallocator, see e.g.
http://www.codeguru.com/Cpp/Cpp/cpp...icle.php/c4079/
Tom
| |
| Tim Roberts 2006-01-20, 3:59 am |
| Daniel <__daniel___@icg.tu-graz.ac.at> wrote:
>thanks for all the information.
>
>my solution now is to use another string implementation which is quite
>similar to std::string (does not implement the whole featureset
>though) that simply does not have this problem.
Did you see my note that you can also solve the problem by switching from
dynamically linking the run-time library (/MD) to statically linking the
run-time library (/MT)?
--
- Tim Roberts, timr@probo.com
Providenza & Boekelheide, Inc.
| |
| David Wilkinson 2006-01-20, 7:59 am |
| Tom Widmer [VC++ MVP] wrote:
> Daniel wrote:
>
>
>
> std::string is special in VC in that it (and wstring) are compiled into
> the CRT DLL, which means you can't intercept their implementation,
> including how they allocate memory. The easiest way around this is to
> link statically with the CRT, so that std::string is compiled directly
> into your source wherever you use it (via the usual template
> instantiation rules), and thus uses your operator new. Think carefully
> about how something was allocated before passing it across a DLL
> boundary though (if you are using DLLs with std containers in the
> interface).
>
> Tom
Strike five for static linking:
1. No DLL deployment issues.
2. Dramatically smaller package size.
3. Can use simple installer program.
4. Works with MSLU without recompiling MFC/CRT
5. Can make alterations to std::string (Dinkumware patches for VC5/6,
custom allocator, etc).
I'm sure there are more ...
David Wilkinson
| |
| adebaene@club-internet.fr 2006-01-20, 7:59 am |
|
David Wilkinson a =E9crit :
>
> Strike five for static linking:
<snip>
And the BIG ONE against : not possible to pass objects that manage an
internal memory buffer (eg, std::string, std containers, etc....) over
DLLs boundary.
Oh yes, another one : If a security flaw (eg, buffer overrun) is
discovered in the current implementation, MS can ship an patch through
Windows Update. If you are linking statically, the fix won't make it
into your app.
This is why the recommended deployement mechanism in VC2005 is linking
to the DLL CRT, and put the DLL in the SxS folder, where Windows Update
can find it and fix it (using a manifest redirection).
Arnaud
MVP - VC
| |
| David Wilkinson 2006-01-20, 7:12 pm |
| Arnaud:
OK, I'll bite ...
adebaene@club-internet.fr wrote:
> David Wilkinson a écrit :
>
>
> <snip>
>
> And the BIG ONE against : not possible to pass objects that manage an
> internal memory buffer (eg, std::string, std containers, etc....) over
> DLLs boundary.
Why is that exactly? I know you cannot new on one side and delete on the
other, but that is not typically what happens when you use the STL,
unless you pass STL references as out parameters. If you don't do that,
doesn't the STL copy semantics save you from disaster?
Anyway, perhaps my situation is special, but I do not use 3rd party
DLL's. I actually make little use of DLL's at all, except for plug-ins,
and I always static-link them. I actually use a "do-it-myself COM" with
a pure abstract base class with exported Create (factory) and Destroy
(release) functions. I would never think of passing an std::string
across a DLL boundary; I just pass its c_str() and copy it on the other
side (so it costs a copy, beat me). This way my DLL's do not even have
to use the same compiler version (same as COM).
> Oh yes, another one : If a security flaw (eg, buffer overrun) is
> discovered in the current implementation, MS can ship an patch through
> Windows Update. If you are linking statically, the fix won't make it
> into your app.
Well I'm still using VC6, so Microsoft isn't going to help me there. And
if there are still buffer over-run problems in the current CRT and/or
MFC DLL's, someone at Microsoft should be taken out and shot.
> This is why the recommended deployement mechanism in VC2005 is linking
> to the DLL CRT, and put the DLL in the SxS folder, where Windows Update
> can find it and fix it (using a manifest redirection).
Here's what I don't understand. If this happens, why aren't these files
part of the operating system in the first place? The .NET runtime
also, if they want me to use that. That way the developers would be
freed once and for all from the responsibility for shipping these files.
Then I really would consider moving to VC8 and using dynamic linking
(and even letting the dreaded .NET into my application). Hell, I'll even
abandon support for Win9x/Me/NT which is also what MS wants me to do.
But as it is, I will likely not move to VC8 because I will still want to
static link, and the IDE sucks.
I mean, I really don't get it. MS wants everyone to use the new and
wonderful (and expensive) Visual Studio for everything; why don't they
make it easy to do? Side-by-side is nice, but why do I have to maintain
it? Except that some cowboy can't trash my application by screwing up
the DLL's, the situation is no better than before.
Really, the key issue for me is deployment size, and there static
linking wins hands down.
David
| |
| Arnaud Debaene 2006-01-22, 7:24 pm |
| David Wilkinson wrote:
> Arnaud:
>
> OK, I'll bite ...
>
> adebaene@club-internet.fr wrote:
>
>
> Why is that exactly? I know you cannot new on one side and delete on
> the other, but that is not typically what happens when you use the
> STL, unless you pass STL references as out parameters. If you don't
> do that, doesn't the STL copy semantics save you from disaster?
No. If you export from a DLL a function that take a std::string as parameter
(for example), both the dll and calling exe must be linked against the CRT
dll, otherwise you may risk a GPF when the string destructor try to free the
string in the DLL.
>
> Anyway, perhaps my situation is special, but I do not use 3rd party
> DLL's. I actually make little use of DLL's at all, except for
> plug-ins, and I always static-link them.
Uhhh??? We mustn't have the same definition of "plug-in". For me, a plugin
is a DLL that is loaded dynamically by LoadLibrary (or equivalent).
> I actually use a
> "do-it-myself COM" with a pure abstract base class with exported
> Create (factory) and Destroy (release) functions. I would never think
> of passing an std::string across a DLL boundary; I just pass its
> c_str() and copy it on the other side (so it costs a copy, beat me).
> This way my DLL's do not even have to use the same compiler version
> (same as COM).
Yes, sure : if you want your DLL to be compatible with various
compilers/languages, you must stick to either a C-style or COM interface,
but this isn't always the case.
>
> Well I'm still using VC6, so Microsoft isn't going to help me there.
Yes, but you aren't going to stick with VC6 forever, are you? The increased
conformance level of newer compiler version is sufficient an argument IMHO.
> And if there are still buffer over-run problems in the current CRT
> and/or MFC DLL's, someone at Microsoft should be taken out and shot.
>
>
> Here's what I don't understand. If this happens, why aren't these
> files part of the operating system in the first place?
Because some OSes versions are anteriors to your Visual Studio version, of
course! You don't expect the VC++2005 CRT dll to be present in Windows 2000
or XP, do you?
> The .NET
> runtime also, if they want me to use that. That way the developers would
> be
> freed once and for all from the responsibility for shipping these
> files.
This is impossible of course, for simple chronological reasons. No
technology will change that...
> Then I really would consider moving to VC8 and using dynamic
> linking (and even letting the dreaded .NET into my application).
> Hell, I'll even abandon support for Win9x/Me/NT which is also what MS
> wants me to do. But as it is, I will likely not move to VC8 because I
> will still want to static link, and the IDE sucks.
- You can statically link to the crt with VC2005.
- A lot of people coming from VC6 said the new IDE "sucked" when VC2003 and
2003 came out. I do not see many of these complaints today, and I believe
it's mostly a matter of habitude.
>
> I mean, I really don't get it. MS wants everyone to use the new and
> wonderful (and expensive) Visual Studio for everything; why don't they
> make it easy to do?
What do you find difficult in VS2005 exactly?
> Side-by-side is nice, but why do I have to
> maintain it? Except that some cowboy can't trash my application by
> screwing up the DLL's, the situation is no better than before.
Ahemm.... The whole point of side-by-side is to prevent DLL trashing as
occured in the old days of "DLL hell". You should read some more about it
before judging.
> Really, the key issue for me is deployment size, and there static
> linking wins hands down.
For you perhaps, but most people consider that size is less and less
important with today's harddisk costs and internet throughputs.
Now, to be fully honest, I join you on one point about increased complexity
with VS2005 : It's true that distribution of native C++ apps is getting much
more complicated with VS2005 (manifests, signing, etc...) that it was.
But IMHO we see here a symptom from a recent change at Microsoft for which I
am gratefull to them (through it is sometimes apain in the %*$£µ) : they now
prefer to add more security instead of new features or increased usability.
Arnaud
MVP - VC
| |
| David Wilkinson 2006-01-22, 7:24 pm |
| Hi Arnaud:
Replies inline:
Arnaud Debaene wrote:
> David Wilkinson wrote:
>
>
>
> No. If you export from a DLL a function that take a std::string as parameter
> (for example), both the dll and calling exe must be linked against the CRT
> dll, otherwise you may risk a GPF when the string destructor try to free the
> string in the DLL.
Not if you pass it by const reference. I agree there are things you
cannot do, but for me it is worth not doing them if it means I can
static link everything.
>
>
> Uhhh??? We mustn't have the same definition of "plug-in". For me, a plugin
> is a DLL that is loaded dynamically by LoadLibrary (or equivalent).
>
Static linked INTERNALLY.
>
>
>
> Yes, sure : if you want your DLL to be compatible with various
> compilers/languages, you must stick to either a C-style or COM interface,
> but this isn't always the case.
>
For me it is worth making sure that it always is the case.
>
>
> Yes, but you aren't going to stick with VC6 forever, are you? The increased
> conformance level of newer compiler version is sufficient an argument IMHO.
>
And the only one, IMHO. MS is not making any significant improvements in
MFC, apart from .NET integration. And VC6 is a very stable product, with
6 service packs. How many service packs have there been for VC7 or 7.1?
<snip>
>
> Because some OSes versions are anteriors to your Visual Studio version, of
> course! You don't expect the VC++2005 CRT dll to be present in Windows 2000
> or XP, do you?
>
Why not? They have Windows Update. If MS is going to repair these DLL's
(your argument) why can't they install them? When MS stops supporting an
operating system, I will also. It's what MS wants, actually (user X
cannot use my app, so he upgrades his/her operating system!). If it is
not easy or efficient for me to deploy dynamic linking, I will stay with
static linking and support the old OS's (not what MS wants).
>
>
> This is impossible of course, for simple chronological reasons. No
> technology will change that...
>
>
>
>
> - You can statically link to the crt with VC2005.
Yes, I can, and I would, but you are arguing against static linking.
> - A lot of people coming from VC6 said the new IDE "sucked" when VC2003 and
> 2003 came out. I do not see many of these complaints today, and I believe
> it's mostly a matter of habitude.
>
I don't think many of these people have changed their mind, they just
have said what they have to say. I have used VC7.1 a lot for a big
non-GUI project, and it's OK, but for MFC work it's a real pain. And I
do not get the impression that MS has made a whole lot of changes in the
IDE for VC8, despite the howls of protest.
>
>
> What do you find difficult in VS2005 exactly?
>
I actually don't have it yet. But I'm talking about the MS-recommended
method of deployment (dynamic linking). If I stick with static linking,
then I just have to trade off better compiler against sucky IDE. A close
call.
>
>
> Ahemm.... The whole point of side-by-side is to prevent DLL trashing as
> occured in the old days of "DLL hell". You should read some more about it
> before judging.
I understand what side-by-side is, and it's a good idea. It would be
even better if I didn't have to maintain it. Again, if MS will repair
these DLL's by Windows Update, why can't they install them?
>
>
> For you perhaps, but most people consider that size is less and less
> important with today's harddisk costs and internet throughputs.
Hard disk size, I agree. Downloading, not. It's like the web itself;
there are so many sites that are completely unusable without broadband
or other high-speed connection.
> Now, to be fully honest, I join you on one point about increased complexity
> with VS2005 : It's true that distribution of native C++ apps is getting much
> more complicated with VS2005 (manifests, signing, etc...) that it was.
> But IMHO we see here a symptom from a recent change at Microsoft for which I
> am gratefull to them (through it is sometimes apain in the %*$£µ) : they now
> prefer to add more security instead of new features or increased usability.
Security is good. But most exploited vulnerabilities are in Microsoft's
applications, not mine.
David
| |
| Arnaud Debaene 2006-01-24, 7:07 pm |
| >>
>
> Not if you pass it by const reference.
And if you want to have somewhere a std::vector<std::string> ?
>
> And the only one, IMHO. MS is not making any significant improvements
> in MFC, apart from .NET integration. And VC6 is a very stable
> product, with 6 service packs. How many service packs have there been
> for VC7 or 7.1?
AFAIK, a service pack is the proof that there was defaults in the product,
no? ;-) Anyway, VC6 support for template is so poor that I found it barely
usable in this area.
>
> Why not? They have Windows Update. If MS is going to repair these
> DLL's (your argument) why can't they install them?
Well, a CRT version is tied to a particular version of the complier, so it
doesn't make any sense to install the 7.1 CRT on a machine where there is no
VC7.1 compiled app. Next, if you satically link the CRT (or even
redistribute the DLL in the application private directory), how do you
expect Windows Update to find it and patch it? That's the whole point of
having it in a central, known location (the SxS folder)...
<snip>
>
> I don't think many of these people have changed their mind, they just
> have said what they have to say. I have used VC7.1 a lot for a big
> non-GUI project, and it's OK, but for MFC work it's a real pain.
I am personaly more than happy not to have to mess with the MFC ugly stuff
for a couple of years now. When I need to do GUI stuff, I use far better
conceived libraries, either .NET Forms, either WTL, Win32Gui or another
"alternative" GUI library, all of them being IMHO far superior to MFC. Of
course, those libraries may need a decent template compiler that VC6 is
not...
> And I
> do not get the impression that MS has made a whole lot of changes in
> the IDE for VC8, despite the howls of protest.
Have you tried a native debugging session? There is a *lot* of improvement
in this area; for example, STL containers are now directly browsable in the
debugger.
> I understand what side-by-side is, and it's a good idea. It would be
> even better if I didn't have to maintain it.
What do you mean by "maintain" exactly?
> Again, if MS will repair
> these DLL's by Windows Update, why can't they install them?
The MS approach is to deliver only security fixes through Windows Update,
not features improvements (outside of Service Packs). We *may* see the CRT
installed with the next SP, but I doubt it since the VC++ CRT has never been
part of the Windows distribution anytime, back to Windows 1.0
> Security is good. But most exploited vulnerabilities are in
> Microsoft's applications, not mine.
Yep, and we are talking about closing an hypothetic security hole in MS CRT
code...
Anyway, to be truly honest, yours and mine applications have at least as
much security holes as Microsoft's, but their code is subject to far much
closer attention from black hats than ours ;-)
Arnaud
MVP - VC
|
|
|
|
|