Home > Archive > VC STL > February 2006 > lexical_cast eating memory - or the STL
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
lexical_cast eating memory - or the STL
|
|
|
| Actually, there's a post in "gmane.comp.lib.boost.user" that
blames this problem on the STL in VC++ 8.0. I found it
yesterday.
Consider the following simple program:
#include "stdafx.h"
#include <string>
#include <boost\lexical_cast.hpp>
using namespace std;
using namespace boost;
int _tmain(int argc, _TCHAR* argv[])
{
int anInt;
string anIntString( "12345" );
for ( size_t i = 0; i < 500000; ++i )
anInt = lexical_cast<int>( anIntString );
return 0;
}
Prior to the for() loop, Process Explorer shows 321.8 MB in use.
Break pointing at the return shows 352.1 MB in use.
In real life, my app reads from a DB table for almost 700,000
records. A string (VARCHAR) field is cast to an int. Memory
usage goes from about 450 MB at the start to 1.4 GB at the end of
the loop. I've had to revert to atoi(). Any ideas?
- Arnie
| |
| Pete Becker 2006-02-18, 6:59 pm |
| Arnie wrote:
>
> Prior to the for() loop, Process Explorer shows 321.8 MB in use.
> Break pointing at the return shows 352.1 MB in use.
>
So memory was allocated to the process that wasn't released from the
process. That doesn't tell you much, because each process manages its
own free memory. Sometimes a process releases unused memory back to the
OS, but more often it hangs on to it, for future use.
--
Pete Becker
Roundhouse Consulting, Ltd.
| |
|
| "Pete Becker" <petebecker@acm.org> wrote in message
news:koidnUwZFpLWGGreRVn-vQ@giganews.com...
> Arnie wrote:
>
>
> So memory was allocated to the process that wasn't released
> from the process. That doesn't tell you much, because each
> process manages its own free memory. Sometimes a process
> releases unused memory back to the OS, but more often it hangs
> on to it, for future use.
>
> --
>
> Pete Becker
> Roundhouse Consulting, Ltd.
I know that what you say is true, but it's not the point. The
for loop, doing nothing but a lexical_cast, linearly increases
the memory used by 30 MB as it runs. My real world program
increases by about 1 GB. Where's it all going?
- Arnie
| |
| Ulrich Eckhardt 2006-02-20, 3:57 am |
| "Arnie" <none> wrote:
> Actually, there's a post in "gmane.comp.lib.boost.user" that
> blames this problem on the STL in VC++ 8.0. I found it
> yesterday.
[...]
> for ( size_t i = 0; i < 500000; ++i )
>
> anInt = lexical_cast<int>( anIntString );
[...]
> Prior to the for() loop, Process Explorer shows 321.8 MB in use.
> Break pointing at the return shows 352.1 MB in use.
The source of lexical_cast doesn't directly use any allocations but it
instantiates a stream object which in turn creates a locale and a
streambuffer. Any of these (I'd suspect the locale though) could cause
problems.
However, it definitely looks like a bug to me.
Uli
| |
| Tom Widmer [VC++ MVP] 2006-02-20, 7:57 am |
| Arnie wrote:
> Actually, there's a post in "gmane.comp.lib.boost.user" that
> blames this problem on the STL in VC++ 8.0. I found it
> yesterday.
>
> Consider the following simple program:
>
> #include "stdafx.h"
>
> #include <string>
>
> #include <boost\lexical_cast.hpp>
>
> using namespace std;
>
> using namespace boost;
>
> int _tmain(int argc, _TCHAR* argv[])
>
> {
>
> int anInt;
>
> string anIntString( "12345" );
>
> for ( size_t i = 0; i < 500000; ++i )
>
> anInt = lexical_cast<int>( anIntString );
>
> return 0;
>
> }
>
> Prior to the for() loop, Process Explorer shows 321.8 MB in use.
> Break pointing at the return shows 352.1 MB in use.
>
> In real life, my app reads from a DB table for almost 700,000
> records. A string (VARCHAR) field is cast to an int. Memory
> usage goes from about 450 MB at the start to 1.4 GB at the end of
> the loop. I've had to revert to atoi(). Any ideas?
This is a known bug:
http://lab.msdn.microsoft.com/produ...da-ed313e0eafcc
It should be fixed in the first SP for VS2005, and there's a workaround
to keep you going until then. You might also request a hotfix.
Tom
| |
| Pete Becker 2006-02-20, 7:57 am |
| Arnie wrote:
> "Pete Becker" <petebecker@acm.org> wrote in message
> news:koidnUwZFpLWGGreRVn-vQ@giganews.com...
>
>
> I know that what you say is true, but it's not the point. The
> for loop, doing nothing but a lexical_cast, linearly increases
> the memory used by 30 MB as it runs.
If lexical_cast is what I think it is, it does an allocation from the
free store each time it's used, and then releases the allocated memory.
It may also allocate some memory that isn't released, for various locale
facets. Of course, that's a one-time allocation, and it won't be
anywhere near 30M.
> My real world program
> increases by about 1 GB. Where's it all going?
>
Into the application's free store, as a guess. As the amount of memory
in the free store increases, so does the program's overall memory
footprint. That doesn't, in itself, mean that there's anything wrong.
Memory managers try to reuse memory when they can, but sometimes they
can't. Without slogging through the details it's not possible to say
what's going on. Seems to me that VC++ comes with a leak detector, which
runs when you're in the debugger. That's the place to start.
--
Pete Becker
Roundhouse Consulting, Ltd.
| |
|
|
|
|
|