PERL Miscellaneous - How do I end processes that time out?

Author

How do I end processes that time out?

J. Romano

2004-07-16, 3:57 pm

Dear Perl community,

I've written some Perl code that runs the Unix "finger" command
inside of backticks (in order to capture its output). Occasionally
the finger command will hang for about thirty seconds or so (in which
case it almost always fails).

I want my Perl code to only wait three seconds for the finger
command to finish. After that, if the finger command has not
finished, I want to just discard the output and move on. Therefore, I
made use of the alarm() function:

$SIG{CHLD} = 'IGNORE'; # avoid creating zombie processes

# This next part (the eval/die blocks) is
# right out of "perldoc -f alarm":
eval {
local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required
alarm 3; # wait a maximum of three seconds
$text = `finger some_user\@some_host`;
alarm 0;
};
if ($@) {
die unless $@ eq "alarm\n"; # propagate unexpected errors
# Timed out:
print "Timed out!\n";
}

Now, this is difficult to test because the finger command almost
always works. However, in the rare case where it doesn't work, my
script prints "Timed out!" (as it should) and continues on until it
finishes.

The problem is that the timed-out finger command is apparently
still running even after my Perl script that called it finishes
running, because a short time later (like thirty seconds or a minute
later), an error message pops up on my terminal saying something like
"Unable to connect to host ...".

Now, I explicitly set $SIG{ENV} to 'IGNORE' so that this wouldn't
happen, but now I think that setting $SIG{ENV} to 'IGNORE' only
terminates (that is, reaps) child processes that have finished BEFORE
the Perl script has finished. (If I'm wrong in saying this, please
correct me.) But if the child process created by the backtick
operator finishes AFTER my Perl script finishes, it stays around well
longer than needed.

To summarize, my question is: If I use the alarm() function to
break out of a command that I started with the backtick operator, how
do I force that child process to exit so that it doesn't outlive my
Perl script?

In case anyone is interested, here is the output of "perl -v":

This is perl, v5.6.1 built for alpha-netbsd

Thanks in advance for any help,

J.

Fiftyvolts

2004-07-16, 3:57 pm

J. Romano wrote:
> Now, I explicitly set $SIG{ENV} to 'IGNORE' so that this wouldn't
> happen, but now I think that setting $SIG{ENV} to 'IGNORE' only
> terminates (that is, reaps) child processes that have finished BEFORE
> the Perl script has finished. (If I'm wrong in saying this, please
> correct me.) But if the child process created by the backtick
> operator finishes AFTER my Perl script finishes, it stays around well
> longer than needed.
>
> To summarize, my question is: If I use the alarm() function to
> break out of a command that I started with the backtick operator, how
> do I force that child process to exit so that it doesn't outlive my
> Perl script?

Your assumption seems correct to me. My suggestion is that if you want
to get fancy with IPC you should leave the backticks behind and do the
operations manually with fork and exec. That way when you fork you'll
get the PID of the kid and can kill it if there is an alarm. Here's a
rough (untested since I am at work) idea of what I am suggesting:
my $pid;
$SIG{CHLD} = 'IGNORE'; # avoid creating zombie processes

eval {
local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required
alarm 3; # wait a maximum of three seconds

die "Can't fork: $!" unless defined($pid = open(CLD, "-|"));

if ($pid) { # parent
while (<CLD> ) {
print; #or whatever
}
close CLD;
} else { # child
exec 'finger', 'some_user@somehost';
}

alarm 0;
};

if ($@) {
die unless $@ eq "alarm\n"; # propagate unexpected errors
# Timed out:
print "Timed out!\n";
kill 9, $pid;
}

ctcgag@hotmail.com

2004-07-16, 3:57 pm

jl_post@hotmail.com (J. Romano) wrote:
> Dear Perl community,
>
> I've written some Perl code that runs the Unix "finger" command
> inside of backticks (in order to capture its output). Occasionally
> the finger command will hang for about thirty seconds or so (in which
> case it almost always fails).
>
> I want my Perl code to only wait three seconds for the finger
> command to finish. After that, if the finger command has not
> finished, I want to just discard the output and move on. Therefore, I
> made use of the alarm() function:
>
> $SIG{CHLD} = 'IGNORE'; # avoid creating zombie processes
>
> # This next part (the eval/die blocks) is
> # right out of "perldoc -f alarm":
> eval {
> local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required
> alarm 3; # wait a maximum of three seconds
> $text = `finger some_user\@some_host`;
> alarm 0;
> };
> if ($@) {
> die unless $@ eq "alarm\n"; # propagate unexpected errors
> # Timed out:
> print "Timed out!\n";
> }
>
> Now, this is difficult to test because the finger command almost
> always works. However, in the rare case where it doesn't work, my
> script prints "Timed out!" (as it should) and continues on until it
> finishes.

You could replace "finger" with "sleep", just for testing purposes.

>
> The problem is that the timed-out finger command is apparently
> still running even after my Perl script that called it finishes
> running, because a short time later (like thirty seconds or a minute
> later), an error message pops up on my terminal saying something like
> "Unable to connect to host ...".
>
> Now, I explicitly set $SIG{ENV} to 'IGNORE' so that this wouldn't
> happen, but now I think that setting $SIG{ENV} to 'IGNORE' only
> terminates (that is, reaps) child processes that have finished BEFORE
> the Perl script has finished.

I think you meant CHLD, not ENV.

"terminate" ne "reap". reaping a child just means you wait for it to die
naturally, and then you bury it. What you want to do is kill the child
before it's time. $SIG{CHLD} doesn't do that.

> To summarize, my question is: If I use the alarm() function to
> break out of a command that I started with the backtick operator, how
> do I force that child process to exit so that it doesn't outlive my
> Perl script?

Well, on at least some shells of at least some OSes, the
parent can end by killing itself and all of it's children, instead of
exiting normally:

kill -15, $$;

However, it would be better to kill them off as soon as they alarmed,
rather than waiting for the end of the program. There are about 27,000
modules on CPAN relating to this problem, if you find one you like, please
let me know. I got tired of wading through them.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
Usenet Newsgroup Service $9.95/Month 30GB

Fiftyvolts

2004-07-16, 8:58 pm

FYI I tested this now and it seems to work properly

Ben Morrow

2004-07-17, 8:56 pm

Quoth jl_post@hotmail.com (J. Romano):
> Dear Perl community,
>
> I've written some Perl code that runs the Unix "finger" command
> inside of backticks (in order to capture its output). Occasionally
> the finger command will hang for about thirty seconds or so (in which
> case it almost always fails).
>
> I want my Perl code to only wait three seconds for the finger
> command to finish. After that, if the finger command has not
> finished, I want to just discard the output and move on. Therefore, I
> made use of the alarm() function:
>
>
> $SIG{CHLD} = 'IGNORE'; # avoid creating zombie processes
>
> # This next part (the eval/die blocks) is
> # right out of "perldoc -f alarm":
> eval {
> local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required
> alarm 3; # wait a maximum of three seconds
> $text = `finger some_user\@some_host`;
> alarm 0;
> };
> if ($@) {
> die unless $@ eq "alarm\n"; # propagate unexpected errors
> # Timed out:
> print "Timed out!\n";
> }
>
> The problem is that the timed-out finger command is apparently
> still running even after my Perl script that called it finishes
> running, because a short time later (like thirty seconds or a minute
> later), an error message pops up on my terminal saying something like
> "Unable to connect to host ...".

You need to fork and exec yourself, and save the pid of the finger
command. Then kill the finger process with SIGTERM if the alarm goes
off. See perlipc for how to emulate backticks with fork/exec.

Ben

--
'Deserve [death]? I daresay he did. Many live that deserve death. And some die
that deserve life. Can you give it to them? Then do not be too eager to deal
out death in judgement. For even the very wise cannot see all ends.'
ben@morrow.me.uk

J. Romano

2004-07-19, 3:59 pm

Dear Perl community,

Thanks for all your help. Using your suggestions I was able to
make a toy program that isolated my problem. I'll post the toy
program, but let me point out a few things first:

* Xho was correct in saying that I meant to say "$SIG{CHLD}"
and not "$SIG{ENV}".

* Working from Xho's suggestion, I wrote a test program
that used "sleep" instead of finger. I just set the
sleep-time to a value greater than the timeout value
to see if my code works.

* Fiftyvolts' and Ben Morrow's suggestion to open a new
process from open() using "-|" (explained in "perldoc -f open")
worked fairly well. Since the code handled by the child
process was fairly short, I was able to put it all inside
the open() statement.

* I encountered only one problem:
The toy program (i.e., test program) I'm posting below worked
perfectly on a system with the following "perl -v" output:

This is perl, v5.6.1 built for i386-linux

but it didn't work on a different system with the following
"perl -v" output:

This is perl, v5.6.1 built for alpha-netbsd

I finally got it to work by not killing the child's $pid
(process id), but $pid + 1. For some reason, the pid
assigned to the child process by the OS was one more than
what was returned from the open() statement. Would anyone
know why?

Well, here is the toy program I used to test this with. You are
asked for the run time of the child process and a time-out value. If
the run-time value (for example, 10) is greater than the time-out
value (for example, 3), the process should time out, and the remaining
child process SHOULD then be killed. But if the time-out value is
greater than the run-time value, the process shouldn't time out, and
the child process should not be killed.

If the child process isn't killed, you should see the line:

*** If you can read this, the child was not killed.

printed to STDERR. You will naturally see this if the time-out value
is greater than the run-time value, but if the run-time value is
greater than the time-out value, then you should only see this if the
child process was not successfully killed.

Here is the toy/test program I used:

#!/usr/bin/perl -w
# File: childTest.pl

use strict;

my $sleepTime;
my $timeout;
my @text; # holds the text of the child process

print "Enter the run time for the child process: ";
chomp($sleepTime = <STDIN> );
print "Enter the timeout value: ";
chomp($timeout = <STDIN> );
print "\n";

my $tinyProgram = <<"END_OF_TINY_PROGRAM";
print "Starting...\n";
sleep $sleepTime;
print "Finished!\n";
warn "*** If you can read this, the child was not killed.\n";
END_OF_TINY_PROGRAM

my $pid = open(CHILD, '-|', "perl -e '$tinyProgram'")
or die "Could not open child process";
print "\$pid = $pid\n";

eval {
local $SIG{ALRM} = sub { die "alarm\n" }; # NB: \n required
alarm $timeout;
@text = <CHILD>;
alarm 0;
};
if ($@) {
die unless $@ eq "alarm\n"; # propagate unexpected errors
# timed out
print "Timed out!\n";
print "Killing process...\n";
kill 'KILL', $pid;
}
else {
print "Successful read!\n";
}
close(CHILD);

print "\nText received:\n", @text;

__END__

If anyone can explain why the alpha-netbsd OS consistently assigns
the child a process id that is one greater than what is returned from
the open() statement, please share it with me. Under alpha-netbsd I
can make this code work if I replace the line:

kill 'KILL', $pid;

with:

kill 'KILL', $pid + 1;

Anyway, thanks for all the help I received.

-- Jean-Luc

Ilmari Karonen

2004-07-19, 8:57 pm

On 2004-07-19, J. Romano <jl_post@hotmail.com> wrote:
>
> I finally got it to work by not killing the child's $pid
> (process id), but $pid + 1. For some reason, the pid
> assigned to the child process by the OS was one more than
> what was returned from the open() statement. Would anyone
> know why?

Presumably because perl hands the command to a shell for parsing. The
$pid you get is for the shell, not for the actual command executed.
For some reason, killing the shell has different effects on different
platforms.

You can avoid this by using the list form of open(). Simply replace

"perl -e '$tinyProgram'"

in your example code with

"perl", "-e", $tinyProgram

and everything should work as expected.

(The reason why killing $pid+1 appears to work is simply that process
id numbers are often assigned sequentially, so the pid of the actual
command often just happens to be one greater than that of the shell.)

--
Ilmari Karonen
If replying by e-mail, please replace ".invalid" with ".net" in address.

J. Romano

2004-07-23, 3:56 am

> On 2004-07-19, J. Romano <jl_post@hotmail.com> wrote:

Ilmari Karonen <usenet@vyznev.invalid> replied in message
news:<slrncfo8pm.14p.usenet@yhteiskone.vyznev.net>...[color=darkred]
>
> Presumably because perl hands the command to a shell for parsing. The
> $pid you get is for the shell, not for the actual command executed.
> For some reason, killing the shell has different effects on different
> platforms.
>
> (The reason why killing $pid+1 appears to work is simply that process
> id numbers are often assigned sequentially, so the pid of the actual
> command often just happens to be one greater than that of the shell.)

Your explanation makes sense, Ilmari. So I would think that
killing $pid+1 will usually work in my case, but won't work 100% of
the time (and therefore should not be relied on).

> You can avoid this by using the list form of open(). Simply replace
>
> "perl -e '$tinyProgram'"
>
> in your example code with
>
> "perl", "-e", $tinyProgram
>
> and everything should work as expected.

Oddly enough, that doesn't work for me. When I use the line:

my $pid = open(CHILD, '-|', 'perl', '-e', $tinyProgram);

I get the following fatal error message at run-time:

Can't use an undefined value as filehandle reference ...

This happens even with the following line:

my $pid = open(CHILD, '-|', 'ls', '-l');

but doesn't happen when I use the one-argument form of "ls", like
this:

my $pid = open(CHILD, '-|', 'ls');

I can't figure out why it's giving me an error message, because
according to "perldoc -f open", the following four lines should be
more or less equivalent:

open(FOO, "cat -n '$file'|");
open(FOO, '-|', "cat -n '$file'");
open(FOO, '-|') || exec 'cat', '-n', $file;
open(FOO, '-|', "cat", '-n', $file);

so if the line:

my $pid = open(CHILD, '-|', "perl -e' $tinyProgram");

works, likewise the line:

my $pid = open(CHILD, '-|', 'perl', '-e', $tinyProgram);

should also work, just as you said it would. But it doesn't. I keep
getting the fatal run-time error message that I can't use an undefined
value as a filehandle reference when I use the list form of open().
Am I missing something?

(Just so you know, this same error happens on two separate Unix
machines. Their "perl -v" output is:

This is perl, v5.6.1 built for alpha-netbsd
This is perl, v5.6.1 built for i386-linux
)

I did get around this error, however. I used the third form of the
open() statements listed above (the one with the exec() call).
Therefore, I replaced the line:

my $pid = open(CHILD, '-|', "perl -e' $tinyProgram");

with:

my $pid = open(CHILD, '-|') || exec 'perl', '-e', $tinyProgram;

and everything worked correctly (meaning that I also received the
correct $pid used to kill the process).

So in the end, I solved my problem of killing child processes that
time out, but now I don't understand why the list form of open() keeps
giving me a fatal error message. If anyone knows why, please let me
know.

Anyway, thanks for all your help. I appreaciate it.

-- Jean-Luc

Ilmari Karonen

2004-07-23, 3:56 pm

On 2004-07-23, J. Romano <jl_post@hotmail.com> wrote:
>
> Oddly enough, that doesn't work for me. When I use the line:
>
> my $pid = open(CHILD, '-|', 'perl', '-e', $tinyProgram);
>
> I get the following fatal error message at run-time:
>
> Can't use an undefined value as filehandle reference ...

That appears to be a bug in perl 5.6.x. For details, see:

http://groups.google.com/groups?thr...7%40rt.perl.org

Your open() || exec() workaround is just fine, though you might want
to check that $pid is false _but defined_ before calling exec().

--
Ilmari Karonen
If replying by e-mail, please replace ".invalid" with ".net" in address.

Sponsored Links