Tcl - TCL memory allocation model

Author

TCL memory allocation model

Daniel Stasinski

2006-10-23, 10:03 pm

What is TCL's internal memory allocation model?

I'm working on a project that will do quite a lot of Tcl_Alloc's,
TCl_Free's and TCL Hash lists being created and freed, and I am
concerned about memory fragmentation over time. I've seen similar
apps to the one I am working on allocate chunks of memory and then
allocate out of those chunks. Empty chunks could possibly be returned
to the heap later. Is this something I need to code on my own for or
does TCL handle it this way internally already?

Daniel

Arjen Markus

2006-10-24, 8:03 am

I am not intimately familiar with the allocation model, but as I
understand it
the following happens:
- Memory for, say, a Tcl_Obj is taken from a pool, to restrict memory
allocation, fragmentation, overhead from allocation etc.
- Memory that is to be freed is not freed immediately, this is
postponed
to a convenient moment.

As it is, use the API for allocating memory in the proper, documented
way and you should not have to worry about the details.

Regards,

Arjen

Donal K. Fellows

2006-10-24, 8:03 am

Daniel Stasinski wrote:
> What is TCL's internal memory allocation model?

When built non-threaded, Tcl directs almost all allocation and release
requests through to malloc() and free(). The exception[*] is for Tcl_Obj
handling, where Tcl always uses its own chunking-and-reusing allocator,
a win because the structure is by far the most common thing allocated.
I've measured this in practical programs, and the difference in scale is
staggering. I've also experimented a bit with doing a variation on the
Tcl_Obj allocator that can release chunks back to the OS. It worked for
what I was doing at the time, but I didn't have the time to fully study
whether it was robust. Alas, I don't remember where the code is right
now either.

When built threaded, I believe that Tcl uses its own per-thread memory
allocator (a massive win for memory-intensive threaded applications like
Tcl itself!) and that that allocator uses a high-water-mark policy. As
policies go, that one is fairly nice for much code, since it is often
(mostly?) the case that applications have relatively constant peak
requirements for memory. If it used 20MB before, chances are it will
need that much again, so keeping it around will make things faster in
the future. Lots of other apps do this too. (If it gets too much, the OS
can just page out some of the unused memory!)

Does this help?

Donal.
[* This is why you must never allocate a Tcl_Obj for yourself; always
use Tcl_NewObj(), or one of the front-ends to it. Well, "never" does
not allow that you might write your own Tcl_Obj manager, but then
that has to be the manager for *all* Tcl_Objs in the process. ]

Daniel Stasinski

2006-10-30, 7:32 pm

Donal K. Fellows wrote:
> Does this help?

Yep, in typical comp.lang.tcl fashion, all my questions are answered
and I know how to proceed. Thanks, guys. :)

Daniel

miguel sofer

2006-10-30, 7:32 pm

Donal K. Fellows wrote:
> When built non-threaded, Tcl directs almost all allocation and release
> requests through to malloc() and free(). The exception[*] is for Tcl_Obj
> handling, where Tcl always uses its own chunking-and-reusing allocator,
> a win because the structure is by far the most common thing allocated.

Just to remark that the Tcl sources also include a chunking-and-reusing
allocator for non-threaded builds - it is just not enabled by default.
The code is in generic/tclAlloc.c

In order to enable it, you have to compile tcl with -DUSE_TCLALLOC.

Miguel

Sponsored Links