PERL Beginners - using open3 to interact with external program

Author

using open3 to interact with external program

D. J. Birkett

2005-06-03, 8:58 am

Hi,

I'm trying to use open3 to control input to an external program (in this
case gpg). I would use Expect, but I need to use packages that are
installed as standard with perl as this script will be running on many
platforms. I simply want to enter interactive mode of gpg, pass it 3
commands, then exit back to my script. The following code runs gpg in
interactive mode, but none of the commands get passed to it and it just
sits there till I exit manually, then it comes back to my script.

use IPC::Open3;

local(*HIS_IN, *HIS_OUT, *HIS_ERR);

$childpid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, 'gpg --edit root');

print HIS_IN "trust\n5\yes\n";
close (HIS_IN);
my @outlines = <HIS_OUT>;
my @errlines = <HIS_ERR>;
close HIS_OUT;
close HIS_ERR;
waitpid($childpid, 0);
if ($?) {
print "Child exited with status of $?\n";
}

If I remove the "close (HIS_IN)" statement

Is it possible to do what I want with open3, and how?

Thanks
Darren

Zentara

2005-06-03, 3:56 pm

On Fri, 03 Jun 2005 08:35:32 +0100, darren@birkett.com (D. J. Birkett)
wrote:

>I'm trying to use open3 to control input to an external program (in this
>case gpg). I would use Expect, but I need to use packages that are
>installed as standard with perl as this script will be running on many
>platforms. I simply want to enter interactive mode of gpg, pass it 3
>commands, then exit back to my script. The following code runs gpg in
>interactive mode, but none of the commands get passed to it and it just
>sits there till I exit manually, then it comes back to my script.

>use IPC::Open3;
>
>local(*HIS_IN, *HIS_OUT, *HIS_ERR);
>
>$childpid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, 'gpg --edit root');
>
>print HIS_IN "trust\n5\yes\n";
>close (HIS_IN);
>my @outlines = <HIS_OUT>;
>my @errlines = <HIS_ERR>;
>close HIS_OUT;
>close HIS_ERR;
>waitpid($childpid, 0);
>if ($?) {
> print "Child exited with status of $?\n";
>}
>
>If I remove the "close (HIS_IN)" statement
>
>
>Is it possible to do what I want with open3, and how?

I'm not testing this locally, with gpg,
but from what I know from using IPC::Open3, you may have to put delays
in your script, to allow
gpg to return, before giving the next command.
Otherwise, gpg will get the 'trust\n5\yes\n' as 1 long command
and won't understand it.

Like:
print HIS_IN "trust\n";
sleep 1;
print HIS_IN "5\n";
sleep 1
print HIS_IN "yes\n";

but better yet, don't rely on delays because your
system may be overloaded and the delay may not work.

The best way is to print a command to HIS_IN and then read
the HIS_OUT and HIS_ERR and use regexes to process the output.
If the expected output is there, print the next line and do the same.
Then you are making IPC::Open3 run like Expect.

Something along these lines

print HIS_IN "trust\n";
chomp(my $answer = <HIS_OUT> );
if( $answer =~ /^How many years?$/ )
{ print HIS_IN "5\n"; }else{print "years error $!\n"; exit}

chomp(my $answer1 = <HIS_OUT> );
if( $answer1 =~ /^Are you sure?$/ )
{ print HIS_IN "yes\n"; }else{print "confirm error $!\n"; exit}

But you need to run the procedure yourself a few times, and
see what the regexes should be.

--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

D. J. Birkett

2005-06-08, 3:57 am

Zentara wrote:

>
> Something along these lines
>
> print HIS_IN "trust\n";
> chomp(my $answer = <HIS_OUT> );
> if( $answer =~ /^How many years?$/ )
> { print HIS_IN "5\n"; }else{print "years error $!\n"; exit}
>
> chomp(my $answer1 = <HIS_OUT> );
> if( $answer1 =~ /^Are you sure?$/ )
> { print HIS_IN "yes\n"; }else{print "confirm error $!\n"; exit}
>
>
> But you need to run the procedure yourself a few times, and
> see what the regexes should be.
>
>
>

OK I've tried altering my code as you suggested, replacing the regexes
with ones that would work. gpg still just sits there as soon as it has
entered it's intereactive mode, and perl doesn't seem to be passing any
commands to it at all.

Any other ideas?

Zentara

2005-06-08, 8:57 am

On Tue, 07 Jun 2005 14:38:41 +0100, darren@birkett.com (D. J. Birkett)
wrote:

>Zentara wrote:
>
>
>OK I've tried altering my code as you suggested, replacing the regexes
>with ones that would work. gpg still just sits there as soon as it has
>entered it's intereactive mode, and perl doesn't seem to be passing any
>commands to it at all.
>
>Any other ideas?

Since you are not showing us your code, we can only guess.
My guess is that you are not setting up the gpg command properly.

The first thing that jumps out at me when I look at man gpg, is:

Please remember that option parsing stops as soon as a non
option is encountered, you can explicitly stop option
parsing by using the special option "--".

So gpg is just sitting there, waiting for more input, you need to find
out why.

Alot of little details can go wrong. Maybe you need to tweak the way
the filehandles are referenced.

But Thomas Baltzer's advice is right-on: look at the other modules, and
see how they do it.

--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

Zentara

2005-06-08, 8:56 pm

On Wed, 08 Jun 2005 06:39:29 -0400, zentara <zentara@highstream.net>
wrote:

>
>Since you are not showing us your code, we can only guess.
>My guess is that you are not setting up the gpg command properly.
>
>The first thing that jumps out at me when I look at man gpg, is:
>
> Please remember that option parsing stops as soon as a non
> option is encountered, you can explicitly stop option
> parsing by using the special option "--".
>
>So gpg is just sitting there, waiting for more input, you need to find
>out why.
>
>Alot of little details can go wrong. Maybe you need to tweak the way
>the filehandles are referenced.
>
>But Thomas Baltzer's advice is right-on: look at the other modules, and
>see how they do it.

Yeah, I've tried running gpg thru IPC::Open3 and you are right, it
dosn't repond. Some apps are tricky the way they use
STDIN, STDOUT and STDERR.

I looked at the modules that use it, and it looks like Crypt::GPG uses
IPC::Run and it gets quite involved. One thing I did notice is that you
need to close the IN filehandle before gpg will return anything.

It all looks very complicated, and you should probably just use the
modules instead of trying to roll your own.

Here is an IPC::Open3 script, which does coax gpg to return output,
but I see problems with having to close the IN filehandle, as shown in
the code below. Maybe there is a way to send and 'eof' thru IN, without
closing it, so you can send multiple commands? If I find it, I'll let
you know. Maybe print a null char?

#!/usr/bin/perl
use warnings;
use strict;
use IPC::Open3;
use IO::Select;
$|++;

my $pid1 = open3(\*IN, \*READ,\*ERROR,"gpg");
#if \*ERROR is false, STDERR is sent to STDOUT

my $sel = new IO::Select();

$sel->add(\*READ);
$sel->add(\*ERROR);

print IN "\n";
close IN;

get_output();

print IN "\n";
close IN;

get_output();

########################################
##################
sub get_output{

if( $sel->can_read ){
foreach my $h ($sel->can_read){
my $buf = '';
if ($h eq \*ERROR){
$buf = <ERROR>;
if($buf){print "ERROR-> $buf\n"}
}else{
$buf = <READ>;
if($buf){print "$buf\n"}
}
}
}

}
__END__

--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

D. J. Birkett

2005-06-08, 8:56 pm

Thomas Bätzler wrote:
> D. J. Birkett <darren@birkett.com> asked:
>
>
>
> You could always go ahead and look at Crypt::GPG to see how
> this module does its interaction with GPG.
>
> http://search.cpan.org/~agul/Crypt-GPG-1.52/GPG.pm
>
> HTH,
> Thomas

Thanks. I've had a look through and it appears to use IPC::Run as an
alternative way of interacting with gpg. Unfortunately, IPC::Run also
is not part of the standard distro.

D. J. Birkett

2005-06-08, 8:56 pm

Zentara wrote:

>
>
> Since you are not showing us your code, we can only guess.
> My guess is that you are not setting up the gpg command properly.

Here is what I now have...

use IPC::Open3;

local(*HIS_IN, *HIS_OUT, *HIS_ERR);

$childpid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, 'gpg --edit root');
sleep 5;

print HIS_IN "trust\n";
chomp(my $answer = <HIS_OUT> );
print $answer;
if ($answer =~ /Your decision/) {
print HIS_IN "5\n";
}
else {
print "decision error $!\n";
exit;
}

chomp (my $answer1 = <HIS_OUT> );
if ($answer1 =~ /^Do you really/) {
print HIS_IN "yes\n";
}
else {
print "sure error $!\n";
exit;
}

>
> The first thing that jumps out at me when I look at man gpg, is:
>
> Please remember that option parsing stops as soon as a non
> option is encountered, you can explicitly stop option
> parsing by using the special option "--".
>
> So gpg is just sitting there, waiting for more input, you need to find
> out why.

gpg is starting up fine. It is not a question of options that I'm
passing it to open it. It's the fact that my code is not interacting
with it once it's open
>
> Alot of little details can go wrong. Maybe you need to tweak the way
> the filehandles are referenced.

This is really where I am after help as my perl is not really that advanced.
>
> But Thomas Baltzer's advice is right-on: look at the other modules, and
> see how they do it.

yes I checked out Crypt::GPG, and that interacts with gpg by using
IPC::Run in an Expect like manner. Unfortunately, IPC::Run does not
come as part of the standard perl distro either.

Thanks
Darren

Wiggins d'Anconia

2005-06-08, 8:56 pm

D. J. Birkett wrote:
> Thomas Bätzler wrote:
>
>
>
> Thanks. I've had a look through and it appears to use IPC::Run as an
> alternative way of interacting with gpg. Unfortunately, IPC::Run also
> is not part of the standard distro.
>

When I was writing a set of client libraries in a similar manner,
GnuPG::Interface was by far the best around, though I don't recall
whether it went to the level you are trying (aka interacting with the
client in such a manner). The problem is that gnupg provides the
status/log/etc. fd switches that work with file descriptors (aka a pipe)
but it isn't precisely documented and the output that it provides isn't
sufficient to do everything. So in some cases (encrypt/decrypt) it works
great, in others (getting a key fingerprint information IIRC) it fails
miserably and you have to screen scrape (at least at the time I was
developing, been a while). I would check out the source for
GnuPG::Interface, and see how it works then emulate it. (The reason we
did the same is because of a forking issue unrelated to gnupg, but
related to POE::Wheel::Run. Sorry can't release code, license issues.)
If this fails you, you might post a message to the gnupg-user or
gnupg-devel mailing lists for help there. I am glad to see Crypt::GPG
was finally upgraded, it had been languishing with deprecated methods.

As a guess it might be which switches you are passing depending on the
combination you use gpg might be passing in or out of interactive mode,
again the gnupg lists will be able to help with that.

As an aside remember that a module unless it has a compiled component
can just be included with whatever distribution your script is
requiring. Assuming you have written any of your own libraries, or are
providing documentation, etc. it would be just as easy to distribute a
module with your script as any other file. Definitely something to consider.

HTH, good luck,

http://danconia.org

JupiterHost.Net

2005-06-08, 8:56 pm

> Here is what I now have...
>
> use IPC::Open3;

use strict;
use warnings;

before IPC::Open3

> local(*HIS_IN, *HIS_OUT, *HIS_ERR);
>
> $childpid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, 'gpg --edit root');

Make sure open3 worked

open3(...) or die $!;

Zentara

2005-06-09, 8:56 am

On Wed, 08 Jun 2005 15:52:21 +0100, darren@birkett.com (D. J. Birkett)
wrote:

>Here is what I now have...
>
>gpg is starting up fine. It is not a question of options that I'm
>passing it to open it. It's the fact that my code is not interacting
>with it once it's open

>yes I checked out Crypt::GPG, and that interacts with gpg by using
>IPC::Run in an Expect like manner. Unfortunately, IPC::Run does not
>come as part of the standard perl distro either.
>
>Thanks
>Darren

I see what is happening but I don't know how to resolve it. I modified
your code above to run on my machine, and it works, but interactively.
When I first run the code, gpg grabs control of the tty away from the
perl script, and it works. This interferes with IPC::Open3's attempt to
sit between the terminal and gpg. I guess he does it for some security
reason?

I noticed in Crypt::GPG, that it uses the undocumented "--no-tty" option
of gpg. If you use that option, gpg will not grab the tty, but then you
have another problem...how to talk to it, since you need to close HIS_IN
to get gpg going?

I see 2 choices: either find a way to use --no-tty and put all the
options on the original command line like (guessing):
"gpg --no-tty --trust=5 --confirm=y --edit zentara"

or

find a way to intercept the tty which gpg is using. I noticed in the
modules, that they do some tricky things with fileno, and they are
probably using it, to grab the fileno's of the tty in order to read and
write to them directly.

#!/usr/bin/perl
use warnings;
use strict;
use IPC::Open3;
use IO::Select;
local(*HIS_IN, *HIS_OUT, *HIS_ERR);

my $childpid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, 'gpg --edit zentara');
#my $childpid = open3(*HIS_IN, *HIS_OUT, *HIS_ERR, 'gpg --no-tty --edit
zentara');

close HIS_IN;
#close HIS_IN;

print HIS_IN "trust\n";
chomp(my $answer = <HIS_OUT> );
print $answer;
if ($answer =~ /Your decision/) {
print HIS_IN "5\n";
}else {
print "decision error $!\n";
exit;
}

chomp (my $answer1 = <HIS_OUT> );
if ($answer1 =~ /^Do you really/) {
print HIS_IN "yes\n";
}else{
print "sure error $!\n";
exit;
}
__END__

--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

Sponsored Links