Home > Archive > PERL POE > January 2006 > Debugging memory usage
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Debugging memory usage
|
|
| Dave C 2006-01-10, 12:24 am |
| Is there a recommended "Best Practices" for debugging memory consumption in
POE?
I ask because, while doing some testing with a script that tails maillogs,
I'm noticing that the resident RAM usage is increasing about 1MB per hour
and never freed.
Any help or suggestions would be appreciated!
Dave
| |
| Dave C 2006-01-10, 12:24 am |
| If it helps, I'm running perl 5.8.5 for RHEL4 and POE 0.32.
I'm using POE to tail multiple logfiles using POE::Wheel::FollowTail to pull
out IP addresses that are attempting to use dictionary attacks to harvest
email addresses on our network in one session. I store the originating
connection's IP, along with a timestamp, and other information in the
session's heap, then loop through every 5 minutes to expire information
older than 1 minute (we do alot of mail). In a child session, I'm reporting
the offending IPs.
Although I'm deleting the expired members of the heap, RAM is still being
consumed and not released.
I've read that a Session won't be cleaned up until all references to it have
been removed. However, I'm not sure this is ever happening because I'm
saving a copy of the Wheels in the Session's heap. Could this be the cause
of my problem, or am I  ?
Dave
| |
| Gavin McDonald 2006-01-10, 12:24 am |
| Dave,
Not that I'm a perl/POE guru by any measure, but in thinking about your
problem, I came up with something:
"The heap parameter defines a session's heap. The heap is passed into states
as the $_[HEAP] field. Heaps are anonymous hash references by default."
Taken in context with this:
http://www.perl.com/doc/manual/html...the_difference_
between_d
I have to ask if you are "undef"ing or "delete"ing your heap variables,
since it seems that undef'ing them might leave some memory in use, thus
causing your memory leak...
HTH,
-Gavin.
> -----Original Message-----
> From: Dave C [mailto:skinnydill@gmail.com]
> Sent: Thursday, December 29, 2005 11:13 AM
> To: poe@perl.org
> Subject: Re: Debugging memory usage
>
> If it helps, I'm running perl 5.8.5 for RHEL4 and POE 0.32.
>
> I'm using POE to tail multiple logfiles using POE::Wheel::FollowTail to
> pull
> out IP addresses that are attempting to use dictionary attacks to harvest
> email addresses on our network in one session. I store the originating
> connection's IP, along with a timestamp, and other information in the
> session's heap, then loop through every 5 minutes to expire information
> older than 1 minute (we do alot of mail). In a child session, I'm
> reporting
> the offending IPs.
>
> Although I'm deleting the expired members of the heap, RAM is still being
> consumed and not released.
>
> I've read that a Session won't be cleaned up until all references to it
> have
> been removed. However, I'm not sure this is ever happening because I'm
> saving a copy of the Wheels in the Session's heap. Could this be the
> cause
> of my problem, or am I ?
>
> Dave
| |
| Jonathan Steinert 2006-01-10, 12:24 am |
| (I wrote this earlier, but sent it only to Dave... here's a message to
archive on the list)
-------- Original Message --------
Subject: Re: Debugging memory usage
Date: Thu, 29 Dec 2005 13:34:35 -0600
From: Jonathan Steinert <hachi@kuiki.net>
To: Dave C <skinnydill@gmail.com>
References:
< 8924227e0512291031h67cdb5dfqfea7cb838a6d
1f3a@mail.gmail.com>
<43B43013.7000601@netilla.com>
< 8924227e0512291113j72c1fd82t731d8b8ff2ac
e8d7@mail.gmail.com>
Dave,
That is correct, wheels hold their sessions alive and is very likely the
source of your problems. You should clean out the heap of all wheel
references when you want the session to destruct. Also you should
probably delete a wheel (or put a Null wheel in place) when you get an
error event of some kind indicating the socket has close, apply as
necessary.
You can also put a state in your sessions to handle the _stop event,
which is fired as the session removed from the kernel and destructed.
This may help you find that your sessions are not destructing at the end
of their work.
--hachi
Dave C wrote:
>If it helps, I'm running perl 5.8.5 for RHEL4 and POE 0.32.
>
>I'm using POE to tail multiple logfiles using POE::Wheel::FollowTail to pull
>out IP addresses that are attempting to use dictionary attacks to harvest
>email addresses on our network in one session. I store the originating
>connection's IP, along with a timestamp, and other information in the
>session's heap, then loop through every 5 minutes to expire information
>older than 1 minute (we do alot of mail). In a child session, I'm reporting
>the offending IPs.
>
>Although I'm deleting the expired members of the heap, RAM is still being
>consumed and not released.
>
>I've read that a Session won't be cleaned up until all references to it have
>been removed. However, I'm not sure this is ever happening because I'm
>saving a copy of the Wheels in the Session's heap. Could this be the cause
>of my problem, or am I ?
>
>Dave
>
>
>
| |
|
|
| Dmitri Tikhonov 2006-01-10, 12:24 am |
| Dave C wrote:
> Is there a recommended "Best Practices" for debugging memory consumption in
> POE?
>
> I ask because, while doing some testing with a script that tails maillogs,
> I'm noticing that the resident RAM usage is increasing about 1MB per hour
> and never freed.
>
> Any help or suggestions would be appreciated!
You should probably specify what POE modules you are using and in what
context, along with your version of perl.
I had a problem with memory leaks when I played with POE (which was a
while ago). My solution was to get rid of closures. That happened to
fix my memory problem, so you may want to try that as well.
- Dmitri.
|
|
|
|
|
has several methods for getting the approximate memory