PERL Beginners - Question on using http rather than ftp

Author

Question on using http rather than ftp

Paul King

2006-11-01, 6:57 pm

Hello All:
We currently have a perl process that places our purchase orders
on our vendors' servers via ftp. I am now working with a new vendor and
their requirement is for transfer via https rather than ftp. Ftp allows
the entire exchange process to be initiated and run from our end. We
establish the connection, change directories as needed, push (put) the
file(s) to the server, and disconnect the connection, leaving the vendor
to deal with the file on their own time frame from thereon out.
Is this scenario possible using http? If this is possible, can someone
point me in a direction where I might get some insight into this and how
to begin? If it's not possible, does anyone have any other options for
this type of client side initiated and controlled exchange? I have
looked at several Perl books: Perl & LWP, Building Scalable Web Sites,
Programming Web Services with Perl, etc and everything I can see
indicates to me that the transaction needs to be initiated from what
would be the vendor's side (pull) rather than ours (push). Any pointers
would be appreciated.

Paul King
paulk@marvin.com

usenet@DavidFilmer.com

2006-11-01, 6:57 pm

Paul King wrote:
> establish the connection, change directories as needed, push (put) the
> file(s) to the server, and disconnect the connection, leaving the vendor
> to deal with the file on their own time frame from thereon out.
> Is this scenario possible using http?

Not quite, but somewhat almost.

First, the vendor has previously been running an FTP server for you to
connect to. Now the vendor must run an HTTP server for you to connect
to (ie, a webserver, such as Apache) . Furthermore, since you want SSL,
the vendor must be be running an SSL-enabled webserver (with a digital
site certificate and everything). The vendor may be doing this already
(but maybe not with SSL support).

Now the vendor must have a "webpage" for you to interact with. That
webpage could be generated with a Perl CGI script. It must contain a
form (and, since it must have an "upload" field, it must be a multipart
form). So the vendor publishes a webpage with a form that has an
upload field in it. You use Mechanize (or cURL or whatever) to "fill
out" and submit that form. The form can be password-controlled for
security (using .htaccess or authentication within the script itself,
or both, or some other method (cookies, etc)). The script could be
coded to allow access only from a particular host or IP address, etc.

See the CGI.pm documentation for instructions about how to set up a
file upload form.

It's then up to the target CGI to receive and process the incoming
file. This would usually involve security checks and placement into a
particular directory. Unlike FTP, you have no direct control (on your
end) over what directory the file ends up in (though the CGI could be
configured so that you could pass a directory as a parameter which the
CGI would use).

--
The best way to get a good answer is to ask a good question.
David Filmer (http://DavidFilmer.com)

Oleg V. Volkov

2006-11-02, 7:56 am

"Paul King" <PAULK@marvin.com> wrote:
> I have looked at several Perl books: Perl & LWP, Building Scalable Web
> Sites, Programming Web Services with Perl, etc and everything I can see
> indicates to me that the transaction needs to be initiated from what
> would be the vendor's side (pull) rather than ours (push). Any pointers
> would be appreciated.

Small CGI script that would work on vendor's site and accept file uploads
with some kind authorisation and, optionally, other parameters if you need
them. Look in those books for examples that deal with forms and
<input type="file">.

--
Oleg "Rowaa[SR13]" V. Volkov

Zentara

2006-11-02, 7:56 am

On Wed, 1 Nov 2006 15:38:22 -0600, PAULK@marvin.com ("Paul King") wrote:

> We currently have a perl process that places our purchase orders
>on our vendors' servers via ftp. I am now working with a new vendor and
>their requirement is for transfer via https rather than ftp. Ftp allows
>the entire exchange process to be initiated and run from our end. We
>establish the connection, change directories as needed, push (put) the
>file(s) to the server, and disconnect the connection, leaving the vendor
>to deal with the file on their own time frame from thereon out.
>Is this scenario possible using http? If this is possible, can someone
>point me in a direction where I might get some insight into this and how
>to begin? If it's not possible, does anyone have any other options for
>this type of client side initiated and controlled exchange? I have
>looked at several Perl books: Perl & LWP, Building Scalable Web Sites,
>Programming Web Services with Perl, etc and everything I can see
>indicates to me that the transaction needs to be initiated from what
>would be the vendor's side (pull) rather than ours (push). Any pointers
>would be appreciated.

From what I can gather from your post, you need to connect to their
https server, and deposit some files?

If so, you are looking for a "lwp-upload-script", and that will do it.

First install the libwww-perl module, and read the README.ssl
file, which tells you which modules need to be installed for ssl
support.

Once that is done, an lwp-upload script can be written, to automatically
connect thru https to a cgi script running on their https server. Then
the directory location where your file will be placed will be controlled
by the cgi script.

The question is, are you able to provide the vendor, with your own
custom cgi script, or are you forced to use their script?

Below are a couple of sample scripts, which should help you
get started. One is the upload.cgi script, and the other is the
lwp-uploader.

#### the cgi script up1.cgi ##########################3

#!/usr/bin/perl
use warnings;
use strict;
use CGI;
use CGI::Carp 'fatalsToBrowser'; #use only for debugging

#for testing fileupload-w-auth

#my $maxsize = 1024 * 100; #max 100K
my $maxsize = 1024 * 20000; #max 20M

#$CGI::POST_MAX= $maxsize; # max 100K posts !not working right?
#$CGI::DISABLE_UPLOADS = 1; # no uploads

my $query = new CGI;
my $upload_dir = "uploads"; #permissions for dir are set 777

print $query->header();

if($ENV{CONTENT_LENGTH} > $maxsize){
print "file too large - must be less than $maxsize bytes";
exit;
}

my $file = $query->param("file");
my $filename = $file;
$filename =~s/.*[\/\\](.*)/$1/;

# you can have multiple dirs, and have which dir to
# upload to as another param
open (UPLOADFILE, ">$upload_dir/$filename");

$/= \8192; # sets 8192 byte buffer chunks, perldoc perlvar
while ( <$file> ){
print UPLOADFILE $_;

# for testing locally, to slow it down
# select(undef,undef,undef,.05);
# print 1;

}
close UPLOADFILE;

print <<END_HTML;
<HTML>
<HEAD> <TITLE>Thanks!</TITLE> </HEAD>
<BODY bgcolor="#ffffff"><br>
<P>Thanks for uploading file : $filename!</P>
</BODY>
</HTML>
END_HTML

__END__

################################
################################
################################

######## the uploader script ########
#!/usr/bin/perl
use warnings;
use strict;
use HTTP::Request::Common qw(POST);
use LWP::UserAgent;

my $url ='https://zentara.zentara.net/~zentara/cgi-bin/uploads/up1.cgi';
my $file = 'testout.tgz';

my $ua = new LWP::UserAgent;

my $req = POST $url,
Content_Type => 'multipart/form-data',
Content => [
file => [$file]
];

# if you want a htaccess file to limit access
# my $user = 'zentara';
# my $pass = 'foobar';
# $req->authorization_basic($user, $pass);

my $res = $ua->request($req);
if ($res->is_success){print $res->as_string;
}else{print $res->status_line;
}
exit 0;
__END__

--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

Paul King

2006-11-02, 9:57 pm

-----Original Message-----
From: Linux@lists.develooper.com [mailto:Linux@lists.develooper.com] On
Behalf Of zentara
Sent: Thursday, November 02, 2006 8:04 AM
To: beginners@perl.org
Subject: Re: Question on using http rather than ftp

On Wed, 1 Nov 2006 15:38:22 -0600, PAULK@marvin.com ("Paul King") wrote:

> We currently have a perl process that places our purchase orders

>on our vendors' servers via ftp. I am now working with a new vendor
and

>their requirement is for transfer via https rather than ftp. Ftp
allows

>the entire exchange process to be initiated and run from our end. We

>establish the connection, change directories as needed, push (put) the

>file(s) to the server, and disconnect the connection, leaving the
vendor

>to deal with the file on their own time frame from thereon out.

>Is this scenario possible using http? If this is possible, can someone

>point me in a direction where I might get some insight into this and
how

>to begin? If it's not possible, does anyone have any other options for

>this type of client side initiated and controlled exchange? I have

>looked at several Perl books: Perl & LWP, Building Scalable Web Sites,

>Programming Web Services with Perl, etc and everything I can see

>indicates to me that the transaction needs to be initiated from what

>would be the vendor's side (pull) rather than ours (push). Any
pointers

>would be appreciated.

From what I can gather from your post, you need to connect to their

https server, and deposit some files?

If so, you are looking for a "lwp-upload-script", and that will do it.

First install the libwww-perl module, and read the README.ssl

file, which tells you which modules need to be installed for ssl

support.

Once that is done, an lwp-upload script can be written, to automatically

connect thru https to a cgi script running on their https server. Then

the directory location where your file will be placed will be controlled

by the cgi script.

The question is, are you able to provide the vendor, with your own

custom cgi script, or are you forced to use their script?

Below are a couple of sample scripts, which should help you

get started. One is the upload.cgi script, and the other is the

lwp-uploader.

---< Scripts cut for space - see previous email under this subject >

Thanks Zentara and Oleg for your input.

Zentara,

The sample scripts you provided will be quite helpful in
providing me with a place to start. I'm almost positive I won't be able
to provide the vendor with my own script to be invoked on their server.
Consequently, will I be able to call a script written in some arbitrary
language from the POST function provided they are using CGI?

Thanks again,

Paul

paulk@marvin.com

Zentara

2006-11-03, 7:57 am

On Thu, 2 Nov 2006 12:08:41 -0600, PAULK@marvin.com ("Paul King") wrote:

> The sample scripts you provided will be quite helpful in
>providing me with a place to start. I'm almost positive I won't be able
>to provide the vendor with my own script to be invoked on their server.
>Consequently, will I be able to call a script written in some arbitrary
>language from the POST function provided they are using CGI?

Sure. They will have to provide you with the URL of the cgi script
that accepts the uploads. After that, it dosn't matter if they
are using Perl, they will have to follow the CGI protocol, so your
lwp-upload script will work.

There are some possible pitfalls though, like they may require
you send in a couple of form fields along with the file, or they
may want you to allow them to set a cookie first, or have javascript
enabled, or some other stupid Microsoft trick to force you to
use a Microsoft browser or system.

But you won't know that until they give you they URL. A decent
company would have a test URL setup for you to practice with,
maybe ask them.

If it gets too complex of an interaction, you will probably have to
switch to WWW::Mechanize from LWP.
Mechanize, will let you get around any obstacle, by exactly simulating
the behavior of a real browser.

Mechanize comes with a cookbook of examples, you can google
for example scripts, or go to http://perlmonks.org and search
their site for WWW::Mechanize.

Such as:
http://perlmonks.org?node_id=469979
http://perlmonks.org?node_id=521497
http://perlmonks.org?node_id=437928

If you do run into a problem, and need help with
Mechanize, you will get better answers at
http://perlmonks.org signup, it's free

See also this tutorial:
http://www.developer.com/lang/other...10942_3454041_2

Goodluck, zentara

--
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

Paul King

2006-11-03, 7:57 am

Thanks again Zentara. I appreciate the help; the information you've
provided will give me a big jump in getting this going.

Regards,
Paul

-----Original Message-----
From: Linux@lists.develooper.com [mailto:Linux@lists.develooper.com] On
Behalf Of zentara
Sent: Friday, November 03, 2006 6:45 AM
To: beginners@perl.org
Subject: Re: Question on using http rather than ftp

On Thu, 2 Nov 2006 12:08:41 -0600, PAULK@marvin.com ("Paul King") wrote:

> The sample scripts you provided will be quite helpful in
>providing me with a place to start. I'm almost positive I won't be
able
>to provide the vendor with my own script to be invoked on their server.
>Consequently, will I be able to call a script written in some arbitrary
>language from the POST function provided they are using CGI?

Sure. They will have to provide you with the URL of the cgi script
that accepts the uploads. After that, it dosn't matter if they
are using Perl, they will have to follow the CGI protocol, so your
lwp-upload script will work.

There are some possible pitfalls though, like they may require
you send in a couple of form fields along with the file, or they
may want you to allow them to set a cookie first, or have javascript
enabled, or some other stupid Microsoft trick to force you to
use a Microsoft browser or system.

But you won't know that until they give you they URL. A decent
company would have a test URL setup for you to practice with,
maybe ask them.

If it gets too complex of an interaction, you will probably have to
switch to WWW::Mechanize from LWP.=20
Mechanize, will let you get around any obstacle, by exactly simulating
the behavior of a real browser.

Mechanize comes with a cookbook of examples, you can google
for example scripts, or go to http://perlmonks.org and search
their site for WWW::Mechanize.

Such as:
http://perlmonks.org?node_id=3D469979
http://perlmonks.org?node_id=3D521497
http://perlmonks.org?node_id=3D437928

If you do run into a problem, and need help with
Mechanize, you will get better answers at
http://perlmonks.org signup, it's free

See also this tutorial:
http://www.developer.com/lang/other...10942_3454041_2

Goodluck, zentara

--=20
I'm not really a human, but I play one on earth.
http://zentara.net/japh.html

--=20
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
<http://learn.perl.org/> <http://learn.perl.org/first-response>

Sponsored Links