PERL Miscellaneous - Following a redirect

Author

Following a redirect

Nospam

2006-10-31, 7:02 pm

I am looking at lwp::useragent to follow a redirect, so far I have :

#! Perl/bin/perl
use strict;
use warnings;

use Data::Dumper;
use LWP::UserAgent;

my $ua = LWP::UserAgent->new;
$ua->timeout(100);
$ua->max_redirect(100);

#my $url= 'http://epa.gov/npdes/';

my $url= 'http://www.google.com';

my $response = $ua->get($url);

print "$url\n";

printing http://epa.gov/npdes/ should redirect to
http://cfpub.epa.gov/npdes/ but nothing happens.

yankeeinexile@gmail.com

2006-10-31, 7:02 pm

"Nospam" <nospam@home.com> writes:
> I am looking at lwp::useragent to follow a redirect, so far I have :
>
> #! Perl/bin/perl
> use strict;
> use warnings;
>
> use Data::Dumper;
> use LWP::UserAgent;
>
> my $ua = LWP::UserAgent->new;
> $ua->timeout(100);
> $ua->max_redirect(100);
>
>
>
> #my $url= 'http://epa.gov/npdes/';
>
> my $url= 'http://www.google.com';
>
> my $response = $ua->get($url);
>
>
>
> print "$url\n";
>
>
> printing http://epa.gov/npdes/ should redirect to
> http://cfpub.epa.gov/npdes/ but nothing happens.

It isn't clear from reading the documentation, but LWP::UserAgent only
handles redirection at the HTTP layer, not the HTML-layer with
http-equiv.

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Lawrence Statton - lawrenabae@abaluon.abaom s/aba/c/g
Computer software consists of only two components: ones and
zeros, in roughly equal proportions. All that is required is to
sort them into the correct order.

Nospam

2006-10-31, 7:02 pm

<yankeeinexile@gmail.com> wrote in message news:87y7qwo5wd.fsf@gmail.com...
> "Nospam" <nospam@home.com> writes:
>
>
> It isn't clear from reading the documentation, but LWP::UserAgent only
> handles redirection at the HTTP layer, not the HTML-layer with
> http-equiv.

I noticed this, but I also tried the script with 'http://www.google.com'
which normally redirects to google.co.uk, but in the script just stays as
google.com

#! Perl/bin/perl
use strict;
use warnings;

use Data::Dumper;
use LWP::UserAgent;

my $ua = LWP::UserAgent->new;

$ua->timeout(100);
$ua->max_redirect(100);

my $url = 'http://www.google.com';

my $response = $ua->get($url);

print "$url \n";

J. Gleixner

2006-10-31, 7:02 pm

Nospam wrote:

> I noticed this, but I also tried the script with 'http://www.google.com'
> which normally redirects to google.co.uk, but in the script just stays as
> google.com
[...]
> my $url = 'http://www.google.com';
> my $response = $ua->get($url);
> print "$url \n";

You expect get() to overwrite your $url variable? Or are you saying that
the content in $response isn't from google.co.uk?

Possibly, 'google.co.uk' might be found in the $response.

Nospam

2006-10-31, 7:02 pm

"J. Gleixner" <glex_no-spam@qwest-spam-no.invalid> wrote in message
news:4547a0c0$0$506$815e3792@news.qwest.net...
> Nospam wrote:
>
as[color=darkred]
> [...]
>
> You expect get() to overwrite your $url variable? Or are you saying that
> the content in $response isn't from google.co.uk?
>
> Possibly, 'google.co.uk' might be found in the $response.

My problem is how to do I show that the response has
'http://www.google.co.uk' ?

how do I get response; to show the url it has redirected to? as a print
statement?

DJ Stunks

2006-10-31, 7:02 pm

Nospam wrote:
> "J. Gleixner" <glex_no-spam@qwest-spam-no.invalid> wrote:
> as
>
> My problem is how to do I show that the response has
> 'http://www.google.co.uk' ?
>
> how do I get response; to show the url it has redirected to? as a print
> statement?

the get() method in LWP::UserAgent returns an HTTP::Response object.

HTTP::Response has a base() method which will return "the base URI for
this response" and which, as the docs clearly state, "might not be the
original URI that was passed to $ua->request() method, because we might
have received some redirect responses first."

-jp

Peter J. Holzer

2006-10-31, 7:02 pm

On 2006-10-31 20:29, Nospam <nospam@home.com> wrote:
> "J. Gleixner" <glex_no-spam@qwest-spam-no.invalid> wrote in message
> news:4547a0c0$0$506$815e3792@news.qwest.net...
[...][color=darkred]
>
> My problem is how to do I show that the response has
> 'http://www.google.co.uk' ?

Look at the content: http://www.google.co.uk has lots of links to other
sites at google.co.uk, it has the text "pages from the UK", etc. should
be pretty easy to distinguish from www.google.com.

Alternatively you can turn off redirect handling by setting
requests_redirectable to [], by overriding redirect_ok, or maybe by
setting max_redirect to 0.

hp

--
_ | Peter J. Holzer | > Wieso sollte man etwas erfinden was nicht
|_|_) | Sy

Sponsored Links