PERL Beginners - executing external binaries

Author

executing external binaries

Saurabh Singhvi

2006-01-10, 4:02 am

hi people

i have a script that executes external binaries using the system command.
now the problem is this:

the external binary is HUGE in terms of ram usage so i need a way to keep it
loaded in the memory
so that i can keep passing the next parameters, instead of loading it each
time.

kindly guide me about this

thanks
Saurabh

Tom Phoenix

2006-01-10, 4:02 am

On 1/7/06, Saurabh Singhvi <saurabhsinghvi@gmail.com> wrote:
> the external binary is HUGE in terms of ram usage so i need a way to keep=
it
> loaded in the memory
> so that i can keep passing the next parameters, instead of loading it eac=
h
> time.

It sounds as if you need to recode that external binary so that it
will stay in memory. The calling program can't force it not to exit.
Good luck with that!

--Tom Phoenix
Stonehenge Perl Training

Dr.Ruud

2006-01-10, 4:02 am

Saurabh Singhvi:

> the external binary is HUGE in terms of ram usage so i need a way to
> keep it loaded in the memory

Maybe this: http://www.clapper.org/software/daemonize/

But it most likely will still deallocate all that memory between runs.
Can't you let that binary do more per run? Or just rewrite it?

--
Affijn, Ruud

"Gewoon is een tijger."

Shawn Corey

2006-01-10, 4:02 am

Saurabh Singhvi wrote:
> hi people
>
> i have a script that executes external binaries using the system command.
> now the problem is this:
>
> the external binary is HUGE in terms of ram usage so i need a way to keep it
> loaded in the memory
> so that i can keep passing the next parameters, instead of loading it each
> time.
>
> kindly guide me about this
>
> thanks
> Saurabh
>

If you're running UNIX you can set the binary's sticky bit. See `man
chmod` and `man 2 chmod`. The sticky bit tells UNIX to leave the program
in RAM for a little while in case it is called again very soon.

--

Just my 0.00000002 million dollars worth,
--- Shawn

"Probability is now one. Any problems that are left are your own."
SS Heart of Gold, _The Hitchhiker's Guide to the Galaxy_

* Perl tutorials at http://perlmonks.org/?node=Tutorials
* A searchable perldoc is available at http://perldoc.perl.org/

Zentara

2006-01-10, 4:02 am

On Sat, 7 Jan 2006 11:09:25 +0000, saurabhsinghvi@gmail.com (Saurabh
Singhvi) wrote:

>hi people
>
>i have a script that executes external binaries using the system command.
>now the problem is this:
>
>the external binary is HUGE in terms of ram usage so i need a way to keep it
>loaded in the memory
>so that i can keep passing the next parameters, instead of loading it each
>time.
>
>kindly guide me about this

Assuming you are talking about a single script execution:

If you only need to feed it on STDIN and not collect output, you can use
the piped form of open.
########################################
##############
open( FH, " | my_huge_program") or warn "$!\n";

print FH "data1";
#wait
print FH "data2";

# etc etc etc
close FH; #will remove it from memory
########################################
################

If you need to collect the STDOUT and STDERR too, you
can open it with IPC::Open3, feed it's STDIN and collect
it's STDOUT and STDERR

########################################
##############

use warnings;
use strict;
use IPC::Open3;
use IO::Select;

#interface to "bc" calculator
my $pid = open3(\*WRITE, \*READ,\*ERROR,"bc");

my $sel = new IO::Select();

$sel->add(\*READ);
$sel->add(\*ERROR);

my($error,$answer)=('','');

while(1){

print "Enter expression for bc, i.e. 2 + 2\n";
chomp(my $query = <STDIN> );

#send query to bc
print WRITE "$query\n";

foreach my $h ($sel->can_read)
{
my $buf = '';
if ($h eq \*ERROR)
{
sysread(ERROR,$buf,4096);
if($buf){print "ERROR-> $buf\n"}
}
else
{
sysread(READ,$buf,4096);
if($buf){print "$query = $buf\n"}
}
}
}

waitpid($pid, 1);
# It is important to waitpid on your child process,
# otherwise zombies could be created.
__END__

--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

Saurabh Singhvi

2006-01-10, 4:02 am

hi all

thanks for the help!! i am hoping to use the chmod thing
or the one from zentara.

thanks again
Saurabh

On 1/8/06, zentara <zentara@highstream.net> wrote:
>
> On Sat, 7 Jan 2006 11:09:25 +0000, saurabhsinghvi@gmail.com (Saurabh
> Singhvi) wrote:
>
> it
> each
>
> Assuming you are talking about a single script execution:
>
> If you only need to feed it on STDIN and not collect output, you can use
> the piped form of open.
> ########################################
##############
> open( FH, " | my_huge_program") or warn "$!\n";
>
> print FH "data1";
> #wait
> print FH "data2";
>
> # etc etc etc
> close FH; #will remove it from memory
> ########################################
################
>
> If you need to collect the STDOUT and STDERR too, you
> can open it with IPC::Open3, feed it's STDIN and collect
> it's STDOUT and STDERR
>
> ########################################
##############
>
> use warnings;
> use strict;
> use IPC::Open3;
> use IO::Select;
>
> #interface to "bc" calculator
> my $pid = open3(\*WRITE, \*READ,\*ERROR,"bc");
>
> my $sel = new IO::Select();
>
> $sel->add(\*READ);
> $sel->add(\*ERROR);
>
> my($error,$answer)=('','');
>
> while(1){
>
> print "Enter expression for bc, i.e. 2 + 2\n";
> chomp(my $query = <STDIN> );
>
> #send query to bc
> print WRITE "$query\n";
>
> foreach my $h ($sel->can_read)
> {
> my $buf = '';
> if ($h eq \*ERROR)
> {
> sysread(ERROR,$buf,4096);
> if($buf){print "ERROR-> $buf\n"}
> }
> else
> {
> sysread(READ,$buf,4096);
> if($buf){print "$query = $buf\n"}
> }
> }
> }
>
> waitpid($pid, 1);
> # It is important to waitpid on your child process,
> # otherwise zombies could be created.
> __END__
>
>
>
>
>
>
>
> --
> I'm not really a human, but I play one on earth.
> http://zentara.net/japh.html
>
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> <http://learn.perl.org/> <http://learn.perl.org/first-response>
>
>
>

Dr.Ruud

2006-01-10, 4:02 am

Shawn Corey:
> Saurabh Singhvi:

>
> If you're running UNIX you can set the binary's sticky bit. See `man
> chmod` and `man 2 chmod`. The sticky bit tells UNIX to leave the
> program in RAM for a little while in case it is called again very
> soon.

If the binary allocates the massive amount of memory at init and frees
it at exit, then you still need a malloc-cloak that keeps that memory at
hand. Maybe 'daemonize' can be adjusted to do that.

--
Affijn, Ruud

"Gewoon is een tijger."

Sponsored Links