VC Language - malloc() & page boundaries -- Difference between VC6 and VC7 CRT?

Author

malloc() & page boundaries -- Difference between VC6 and VC7 CRT?

Andrew

2005-11-23, 3:59 am

I was wondering if anyone knew of any changes in the CRT that would cause
malloc behavior to change between code built with VC6/VC7? We discovered a
segfault bug in production code built with VC6 that was not crashing
frequently at all. We fixed the bug, but in the meantime, our builds were in
the process of being
switched to VC7. In the VC7 built code, the same crash occurs much more
frequently. This alarmed us, so we are investigating why VC7 behavior would
differ so much from VC6.

The nature of the bug is such that many small buffers (~16 bytes) are being
malloc'd but the bug causes the code to read past the end of the buffer by 8
bytes. Straightforward, and this crashes whenever the address to the 17th
byte
lives on the next memory page which happens to not be allocated.

The issue is in VC6 built code, this bug rarely occurs, almost as if the
memory
allocation never allocates that close to an invalid page, or never completely
fills up an entire page. When the code is built in VC7, the crash occurs
very,
very frequently, as if the 16 bytes are being allocated at the end of the
page
prior to an invalid one.

My question is if anything regarding malloc or the heap changed between
VC6/VC7 that could be behind this. Do any of the compiler optimization flags
affect malloc's ability to alloc up to the last available byte in a page?

We have discovered the source of the bug in remote code and have fixed it,
but
we are questioning if it is safe to move to VC7 so quickly.

Carl Daniel [VC++ MVP]

2005-11-23, 3:59 am

Andrew wrote:
> I was wondering if anyone knew of any changes in the CRT that would
> cause malloc behavior to change between code built with VC6/VC7? We
> discovered a segfault bug in production code built with VC6 that was
> not crashing frequently at all. We fixed the bug, but in the
> meantime, our builds were in the process of being
> switched to VC7. In the VC7 built code, the same crash occurs much
> more frequently. This alarmed us, so we are investigating why VC7
> behavior would differ so much from VC6.
>
> The nature of the bug is such that many small buffers (~16 bytes) are
> being malloc'd but the bug causes the code to read past the end of
> the buffer by 8 bytes. Straightforward, and this crashes whenever the
> address to the 17th byte
> lives on the next memory page which happens to not be allocated.
>
> The issue is in VC6 built code, this bug rarely occurs, almost as if
> the memory
> allocation never allocates that close to an invalid page, or never
> completely fills up an entire page. When the code is built in VC7,
> the crash occurs very,
> very frequently, as if the 16 bytes are being allocated at the end of
> the page
> prior to an invalid one.
>
> My question is if anything regarding malloc or the heap changed
> between VC6/VC7 that could be behind this. Do any of the compiler
> optimization flags affect malloc's ability to alloc up to the last
> available byte in a page?
>
> We have discovered the source of the bug in remote code and have
> fixed it, but
> we are questioning if it is safe to move to VC7 so quickly.

Yes. The behavior of malloc for small blocks changed dramatically in VC7.
See _set_sbh_threshold in the documentation for the particulars.

-cd

Oleg Starodumov

2005-11-23, 7:59 am

In addition:

> My question is if anything regarding malloc or the heap changed between
> VC6/VC7 that could be behind this. Do any of the compiler optimization flags
> affect malloc's ability to alloc up to the last available byte in a page?
>

It is also possible that PageHeap is enabled on the system where VC7 build
was tested. Are there any of the following Registry entries set?

HKLM\Software\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\yourapp.exe
GlobalFlag
PageHeapFlags
(replace yourapp.exe with the name of the application's .exe file)

There is one more change in VC7 that could sometimes lead to similar effect -
VC6's CRT uses allocation granularity of 16 when allocating memory from the heap
(that is, the size of all allocated blocks is rounded to the next 16 byte boundary),
while VC7 does not impose any allocation granularity at all.

Regards,
Oleg
[VC++ MVP]

Sponsored Links