Home > Archive > PERL Modules > May 2004 > LWP UserAgent form post problem
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
LWP UserAgent form post problem
|
|
| Wayne S 2004-05-17, 11:33 am |
| Hello,
I am probably missing something obvious right in front of me, but
hopefully another point of view will help me. When I paste the
following link in my browser:
http://www.tad.org/Datasearch/reloc...Any+Street+Type
I get the data I want. But when I use the following PERL code, the
server returns no matches:
#!C:\perl\bin\perl.exe -w
use strict;
use LWP;
open (OUT, ">tadout.txt");
my $browser = LWP::UserAgent->new();
my $response = $browser->post(
'http://www.tad.org/Datasearch/relocation.cfm',
[
"StNum" => "1017",
"StName" => "lindstrom",
"Styp" => "Any+Street+Type",
]
);
print OUT $response->content();
close (OUT);
What am I missing?
Thanks,
Wayne
| |
|
| Wayne S wrote:
> Hello,
Hi,
> "Styp" => "Any+Street+Type",
Styp should be "Any Street Type"
vic
| |
| Charles DeRykus 2004-05-17, 2:30 pm |
| In article <8c8696f2.0405170633.e2d3cb9@posting.google.com>,
Wayne S <wayne_naomi@yahoo.com> wrote:
>Hello,
>
>I am probably missing something obvious right in front of me, but
>hopefully another point of view will help me. When I paste the
>following link in my browser:
>
>http://www.tad.org/Datasearch/reloc...Any+Street+Type
>
>I get the data I want. But when I use the following PERL code, the
>server returns no matches:
>
>#!C:\perl\bin\perl.exe -w
>use strict;
>use LWP;
use LWP::Debug qw(+);
>
>open (OUT, ">tadout.txt");
or die $!;
>
>my $browser = LWP::UserAgent->new();
>
>my $response = $browser->post(
> 'http://www.tad.org/Datasearch/relocation.cfm',
> [
> "StNum" => "1017",
> "StName" => "lindstrom",
> "Styp" => "Any+Street+Type",
> ]
> );
if ( $response->is_success ) {
print OUT $response->content();
} else {
die "request failed: ", $response->status_line;
}
I've added a few lines that may help you figure out
what's going on. Cookies may not be a factor but LWP
won't automatically handle them as the browser will.
Check LWP for details. (WWW::Mechanize might be easier
to use if there're cookies).
hth,
--
Charles DeRykus
| |
| Eric Bohlman 2004-05-17, 4:30 pm |
| wayne_naomi@yahoo.com (Wayne S) wrote in
news:8c8696f2.0405170633.e2d3cb9@posting.google.com:
> I am probably missing something obvious right in front of me, but
> hopefully another point of view will help me. When I paste the
> following link in my browser:
>
> http://www.tad.org/Datasearch/reloc...StName=lindstro
> m&Styp=Any+Street+Type
You're using the GET method there (parameters sent as part of the URL).
> I get the data I want. But when I use the following PERL code, the
> server returns no matches:
>
> #!C:\perl\bin\perl.exe -w
> use strict;
> use LWP;
>
> open (OUT, ">tadout.txt");
>
> my $browser = LWP::UserAgent->new();
>
> my $response = $browser->post(
> 'http://www.tad.org/Datasearch/relocation.cfm',
> [
> "StNum" => "1017",
> "StName" => "lindstrom",
> "Styp" => "Any+Street+Type",
> ]
> );
You're using the POST method there (parameters sent as part of the request
body). It's very likely that the server-side process you're trying to
communicate can only handle GET requests.
| |
| Alan J. Flavell 2004-05-17, 5:30 pm |
| On Mon, 17 May 2004, Eric Bohlman wrote:
> wayne_naomi@yahoo.com (Wayne S) wrote in
>
> You're using the POST method there (parameters sent as part of the request
> body).
Yes, but also the hon Usenaut has tried to second-guess the procedure
by applying the formURLencoding rules to the value string
"Any+Street+Type". My understanding is that the module expects to get
the /actual/ user string there ("Any Street Type"), and will look
after the necessary encoding. So in this case the string is going to
land up by being doubly-encoded.
> It's very likely that the server-side process you're trying to
> communicate can only handle GET requests.
Well, with a decent CGI module it's trivial to implement forms
submission handlers which handle multiple submission modes. But
you're probably right to be sceptical. Far too many programmers of
server-side scripts are, for some obscure reason, keen to invest the
extra labour of producing one-off scripts with hand-knitted code
subject to unnecessary limitations and bugs, instead of taking the
easy way and using something that's both easy to apply and has years
of development effort behind it to ensure that it works. IYKWIM.
| |
| Wayne S 2004-05-17, 6:30 pm |
| vic <vic@rez.fr> wrote in message news:<c8almj$qan$1@netserv.univ-lille1.fr>...
> Wayne S wrote:
> Hi,
>
>
> Styp should be "Any Street Type"
>
> vic
Unfortunately I've already tried it with spaces, same result.
Thanks!
| |
| Wayne S 2004-05-17, 6:30 pm |
| Eric Bohlman <ebohlman@earthlink.net> wrote in message news:<Xns94EC8F643AE1Bebohlmanomsdevcom@130.133.1.4>...
>
> You're using the GET method there (parameters sent as part of the URL).
>
[color=darkred]
>
> You're using the POST method there (parameters sent as part of the request
> body). It's very likely that the server-side process you're trying to
> communicate can only handle GET requests.
The search form sent by the server at
http://www.tad.org/Datasearch/re_location_search.cfm uses a post form
like I'm doing in my script, however, when I give it the search
parameters in the url like a get operation, it takes them anyway.
Thanks!
| |
| Wayne S 2004-05-17, 6:30 pm |
| ced@bcstec.ca.boeing.com (Charles DeRykus) wrote in message news:<HxvBpI.L99@news.boeing.com>...
> if ( $response->is_success ) {
> print OUT $response->content();
> } else {
> die "request failed: ", $response->status_line;
> }
>
> I've added a few lines that may help you figure out
> what's going on. Cookies may not be a factor but LWP
> won't automatically handle them as the browser will.
> Check LWP for details. (WWW::Mechanize might be easier
> to use if there're cookies).
>
> hth,
I put those in my code and I got back a response from the server, the
response is '0 matches' so I don't think cookies are the problem. I
suspect it is somehow related to how I'm passing the Styp variable to
the server in my POST. This is how the server presents it in the form
(as a select option):
<select name="Styp">
<option>Any Street Type</option>
<option>DR - Drive</option>
<option>ST - Street</option>
<option>AVE - Avenue</option>
<option>RD - Road</option>
<option>LN - Lane</option>
<option>CT - Court</option>
</select>
I am FAR from being adept at HTML, but I want to pass 'Any Street
Type' to the server. I'm not sure how to handle the spaces (which
doesn't work in the script). When I use plus signs in the url it
works, when I use them in the script... it doesn't... ARGH
Thanks!
| |
| Wayne S 2004-05-18, 10:32 am |
| ced@bcstec.ca.boeing.com (Charles DeRykus) wrote in message news:<HxvBpI.L99@news.boeing.com>...
> use LWP::Debug qw(+);
Another side thought: setting debug helps (thanks for showing me that
one!)... part of the output I get from this is:
LWP::UserAgent::send_request: POST
http://www.tad.org/Datasearch/relocation.cfm
Is there any way to get LWP to show me what its POSTing?
|
|
|
|
|