PERL Beginners - keeping the file name while checking w/ LWP

Author

keeping the file name while checking w/ LWP

Brian Volk

2004-07-16, 8:56 pm

Hi All,

Can someone please point me in the right direction... maybe a module..?

I have a file that contains the file name and the url. I need to separate
the url so I can use LWP::Simple; to check the link, however, I need to keep
the file name in tact so I can trace it back to my website. Anyone have any
suggestions?

Thanks for your help.

Below is what I am using to check the links.

#!/usr/local/bin/perl -w

use strict;
use LWP::Simple;

my $file = "/Program Files/OptiPerl/results.txt";
open (LINKS, $file) or die "Can't open $file: $!";

# ------------ slurp the file -------------------------
my @urls = <LINKS>;
# ----------- remove line endings ---------------------
chomp @urls;
# --------- check each url in file --------------------
foreach my $url (@urls)
{
print "$url\n";
my ($type) = head($url);
# ---------- broken links go into bad.txt -------------
unless (defined $type) {
open (BAD, ">>bad.txt");
print (BAD "$url\n");
# print "Couldn't get $url\n";
next;
}
# --------- good links go into good.txt ---------------
if ($type) {
open (GOOD, ">>good.txt");
print (GOOD "$url\n");
# print "Got it! \n$url\n";
}
}
# --------- close all file handles --------------------
close LINKS;
close GOOD;
close BAD;

Brian Volk

Gunnar Hjalmarsson

2004-07-16, 8:56 pm

Brian Volk wrote:
> I have a file that contains the file name and the url. I need to
> separate the url so I can use LWP::Simple; to check the link,
> however, I need to keep the file name in tact so I can trace it
> back to my website. Anyone have any suggestions?

Use map() together with split() to populate a hash.

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

Brian Volk

2004-07-16, 8:56 pm

thank you... I was just reading about Grep vs. Loops :-)

Thanks again!

Brian

-----Original Message-----
From: Gunnar Hjalmarsson [mailto:noreply@gunnar.cc]
Sent: Friday, July 16, 2004 4:39 PM
To: beginners@perl.org
Subject: Re: keeping the file name while checking w/ LWP

Brian Volk wrote:
> I have a file that contains the file name and the url. I need to
> separate the url so I can use LWP::Simple; to check the link,
> however, I need to keep the file name in tact so I can trace it
> back to my website. Anyone have any suggestions?

Use map() together with split() to populate a hash.

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
<http://learn.perl.org/> <http://learn.perl.org/first-response>

Brian Volk

2004-07-28, 8:56 pm

Gunnar Hjalmarsson wrote:
>Use map() together with split() to populate a hash.

Like this...
my %urls = map { split(' ', $_) } <LINKS>;

if this is correct, can someone hint to me how I am supposed to incorporate
this into my foreach statement.

use strict;
use LWP::Simple;

my $file = "/Program Files/OptiPerl/links.out";
open (LINKS, $file) or die "Can't open $file: $!";

# ------------ slurp the file -------------------------
# my @urls = <LINKS>;

my %urls = map { split(' ', $_) } <LINKS>;

# ----------- remove line endings ---------------------
chomp %urls;
# --------- check each url in file --------------------

foreach my $url (@urls)
{
print "$url\n";
my ($type) = head($url);
# ---------- broken links go into bad.txt -------------
unless (defined $type) {
open (BAD, ">>bad.txt");
print (BAD "$url\n");
# print "Couldn't get $url\n";
next;
}
# --------- good links go into good.txt ---------------
if ($type) {
open (GOOD, ">>good.txt");
print (GOOD "$url\n");
# print "Got it! \n$url\n";
}
}
# --------- close all file handles --------------------
close LINKS;
close GOOD;
close BAD;

I changed @urls to %urls and the script actually gave me what I was trying
to accomplish... but I would rather know the correct way. Thank you for any
help.

Brian Volk

-----Original Message-----
From: Gunnar Hjalmarsson [mailto:noreply@gunnar.cc]
Sent: Friday, July 16, 2004 4:39 PM
To: beginners@perl.org
Subject: Re: keeping the file name while checking w/ LWP

Brian Volk wrote:
> I have a file that contains the file name and the url. I need to
> separate the url so I can use LWP::Simple; to check the link,
> however, I need to keep the file name in tact so I can trace it
> back to my website. Anyone have any suggestions?

Use map() together with split() to populate a hash.

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

--
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
<http://learn.perl.org/> <http://learn.perl.org/first-response>

Gunnar Hjalmarsson

2004-07-28, 8:56 pm

Brian Volk wrote:
> Gunnar Hjalmarsson wrote:
>
> Like this...
> my %urls = map { split(' ', $_) } <LINKS>;

Yep, that's what I had in mind.

> if this is correct, can someone hint to me how I am supposed to
> incorporate this into my foreach statement.

<code snipped>

> I changed @urls to %urls and the script actually gave me what I was
> trying to accomplish... but I would rather know the correct way.

Well, your description of what the file looks like exactly, and of
what it is you are trying to accomplish, isn't that clear to me, so I
don't know of any "Correct Way". :) This is my interpretation:

#!/usr/bin/perl
use strict;
use warnings;
use LWP::Simple;

my $file = '/Program Files/OptiPerl/results.txt';

open my $LINKS, $file or die "Can't open $file: $!";
open my $BAD, '>> bad.txt' or die $!;
open my $GOOD, '>> good.txt' or die $!;

my %urls = map { split ' ', $_ } <$LINKS>;

for my $name ( keys %urls ) {
print "$name $urls{$name}\n";
my ($type) = head( $urls{$name} );
print { defined $type ? $GOOD : $BAD } "$name $urls{$name}\n";
}

close $LINKS;
close $BAD;
close $GOOD;

--
Gunnar Hjalmarsson
Email: http://www.gunnar.cc/cgi-bin/contact.pl

Sponsored Links