PERL Beginners - Problem with my code

Author

Problem with my code

Mihir Kamdar

2007-08-02, 7:59 am

Hi,

I have a requirement. There are files in the parent directory and they are
touched in the child directory. So in child directory there are 0 byte files
having the same name as the ones in the parent directory. I need to read
files in the child directory, and for each file present in the child
directory, I need to take that file from the parent directory, process that
file(remove duplicate lines from that file), and then write the output for
each of the input files in the output directory.

I have written the code as below but it gives 0 byte files as output. Please
suggest what is wrong with the below code.
I have a fear that instead of processing files from parent directory, it is
processing files from the child directory.

I am a newbie at perl. So i might have made silly mistakes. Request you to
correct them.

#!/usr/bin/perl

my $child_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
;
my $parent_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
;
my $write_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
;
my $line;
my %hash;
my @file;
my $key ;
my $value ;

while () {

opendir my $dh, $child_path or die $!;

while (my $file = readdir $dh) {

my $fname = "$child_path/$file";
next unless -f $fname;
unless (exists $times{$file}){

opendir my $dh1,$parent_path or die $!;

while (my $file = readdir $dh1) {

while ($line=readline($file))
{
my @cdr=split (/,/, $line) ;

$hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
if u want.
}
close $file ;
}
closedir $dh1 ;

open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
while (($key, $value) = each %hash)
{
print $OUT_FILE $value;
}
close $OUT_FILE;
}
unlink("$parent_path/$file") ;
unlink("$child_path/$file") ;

}
closedir $dh;
}

Mihir Kamdar

2007-08-02, 7:59 am

On 8/2/07, Mihir Kamdar <kamdarmihir06@gmail.com> wrote:
>
> Hi,
>
> I have a requirement. There are files in the parent directory and they are
> touched in the child directory. So in child directory there are 0 byte files
> having the same name as the ones in the parent directory. I need to read
> files in the child directory, and for each file present in the child
> directory, I need to take that file from the parent directory, process that
> file(remove duplicate lines from that file), and then write the output for
> each of the input files in the output directory.
>
> I have written the code as below but it gives 0 byte files as output.
> Please suggest what is wrong with the below code.
> I have a fear that instead of processing files from parent directory, it
> is processing files from the child directory.
>
> I am a newbie at perl. So i might have made silly mistakes. Request you to
> correct them.
>
>
>
> #!/usr/bin/perl
>
> my $child_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
> ;
> my $parent_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
> ;
> my $write_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
> ;
> my $line;
> my %hash;
> my @file;
> my $key ;
> my $value ;
>
> while () {
>
> opendir my $dh, $child_path or die $!;
>
> while (my $file = readdir $dh) {
>
> my $fname = "$child_path/$file";
> next unless -f $fname;
> unless (exists $times{$file}){
>
> opendir my $dh1,$parent_path or die $!;
>
> while (my $file = readdir $dh1) {
>
> while ($line=readline($file))
> {
> my @cdr=split (/,/, $line) ;
>
> $hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
> if u want.
> }
> close $file ;
> }
> closedir $dh1 ;
>
> open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
>
> while (($key, $value) = each %hash)
> {
> print $OUT_FILE $value;
> }
> close $OUT_FILE;
> }
> unlink("$parent_path/$file") ;
> unlink("$child_path/$file") ;
>
>
> }
> closedir $dh;
> }
>
>
a little correction in the above code:
#!/usr/bin/perl

my $child_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
;
my $parent_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
;
my $write_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
;
my $line;
my %hash;
my @file;
my $key ;
my $value ;
my %times ;
while () {

opendir my $dh, $child_path or die $!;

while (my $file = readdir $dh) {

my $fname = "$child_path/$file";
next unless -f $fname;
unless (exists $times{$file}){

opendir my $dh1,$parent_path or die $!;

while (my $file = readdir $dh1) {

while ($line=readline($file))
{
my @cdr=split (/,/, $line) ;

$hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
if u want.
}
close $file ;
}
closedir $dh1 ;

open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
while (($key, $value) = each %hash)
{
print $OUT_FILE $value;
}
close $OUT_FILE;
}
unlink("$parent_path/$file") ;
unlink("$child_path/$file") ;

}
closedir $dh;
}

Mihir Kamdar

2007-08-02, 7:59 am

On 8/2/07, Mihir Kamdar <kamdarmihir06@gmail.com> wrote:
>
>
>
> On 8/2/07, Mihir Kamdar <kamdarmihir06@gmail.com> wrote:
> a little correction in the above code:
> #!/usr/bin/perl
>
> my $child_path = '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
> ;
> my $parent_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
> ;
> my $write_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
> ;
> my $line;
> my %hash;
> my @file;
> my $key ;
> my $value ;
> my %times ;
> while () {
>
> opendir my $dh, $child_path or die $!;
>
> while (my $file = readdir $dh) {
>
> my $fname = "$child_path/$file";
> next unless -f $fname;
> unless (exists $times{$file}){
>
> opendir my $dh1,$parent_path or die $!;
>
> while (my $file = readdir $dh1) {
>
> while ($line=readline($file))
> {
> my @cdr=split (/,/, $line) ;
>
> $hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
> if u want.
> }
> close $file ;
> }
> closedir $dh1 ;
>
> open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
>
> while (($key, $value) = each %hash)
> {
> print $OUT_FILE $value;
> }
> close $OUT_FILE;
> }
> unlink("$parent_path/$file") ;
> unlink("$child_path/$file") ;
>
>
> }
> closedir $dh;
> }
>
> Hi,
No replies from anybody. Can anybody plz help me with the above code?

Thanks,
Mihir

Edwin van den Oetelaar

2007-08-02, 7:00 pm

Mihir Kamdar wrote:
[color=darkred]
Yeah right,

I suggest you place some print statements at strategic places and start debugging.
Or you buy Komodo and singlestep to find out your problem.

Good luck,
Edwin

Bob McConnell

2007-08-02, 7:00 pm

> -----Original Message-----
> From: Mihir Kamdar
> Sent: Thursday, August 02, 2007 8:15 AM
> To: beginners
> Subject: Re: Problem with my code
>=20
> On 8/2/07, Mihir Kamdar wrote:
> directory and they
> there are 0 byte
> directory. I need to
> present in the child
> directory, process that
> write the output for
> files as output.
> parent directory, it
> mistakes. Request you
> No replies from anybody. Can anybody plz help me with the above code?
>=20
> Thanks,
> Mihir

First of all, I think you need to fix your description of the
requirements. This section doesn't make a lot of sense to me. If the
file is in your parent directory, it can't be touched in another
directory unless there is a symlink there.

If I understand what you are trying to say, you have a data file in one
directory with a matching tag file in another. Each time the data file
is updated, the tag file is touched to indicate the update is completed.
Then a second process must read and clean up that data file, either
deleting it when done, or renaming or moving it before it starts?

But, is it possible to overwrite or append data to an unprocessed file
in the data directory, or must the external process wait until it is
deleted, moved or renamed then create a new empty file before it can
save new data? Is there any way that both processes might try to access
the data file simultaneously, with one writing while the other is still
reading?

Does the tag file get deleted, moved or renamed along with the data
file?

When a new data file is added, does the tag file also get created by
that process, or is there a manual step to create it?

How frequently does each data file get updated?

How frequently does a new file name get added to those directories?

Is it necessary to work on more than one file at a time, or is a single
task loop adequate to keep up with the input?

Once you have a complete description, the usefulness of the advice
should improve.

Bob McConnell

Ken Foskey

2007-08-02, 7:00 pm

On Thu, 2007-08-02 at 17:44 +0530, Mihir Kamdar wrote:
....
Have you run it in a debugger and seen what happens?

perl -d script

The snippet does not actually assign to %hash, I assume it is in the
missing code but you should show a sample line from it.

[color=darkred]
> while (($key, $value) = each %hash)

I find this style more 'normal'.

foreach my $key (keys %hash) {
print $OUT_FILE $hash{$key};
}

Or even this to make code predictable:

foreach my $key (sort keys %hash) {
print $OUT_FILE $hash{$key};
}

--
Ken Foskey
FOSS developer

Mr. Shawn H. Corey

2007-08-02, 7:00 pm

Ken Foskey wrote:
> I find this style more 'normal'.
>
> foreach my $key (keys %hash) {
> print $OUT_FILE $hash{$key};
> }
>
> Or even this to make code predictable:
>
> foreach my $key (sort keys %hash) {
> print $OUT_FILE $hash{$key};
> }
>
>

If you're only interested in the values and the ordering is not important:

for ( values %hash ){
print;
}

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Mihir Kamdar

2007-08-02, 7:00 pm

On 8/2/07, Mr. Shawn H. Corey <shawnhcorey@magma.ca> wrote:
>
> Ken Foskey wrote:
>
> If you're only interested in the values and the ordering is not important:
>
> for ( values %hash ){
> print;
> }
>
>
> --
> Just my 0.00000002 million dollars worth,
> Shawn
>
> "For the things we have to learn before we can do them, we learn by doing
> them."
> Aristotle
>
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> http://learn.perl.org/
>
>
> Hi,

@Bob:

As soon as the data file is updated completely, the tag file is touched with
the same name in the child directory.

This process is automatic. No manual touching of tag files.

As per my requirement, in my code, I am trying to open the child directory,
read the tag name, open the parent directory,and corresponding to that tag,
pick up the data file, process it and write the output to another directory.

After processing, both the tag file as well as data file needs to be
deleted. We should only have the output file in the output directory.

There is no possibility of updating or appending any data file in the parent
directory once a tag file is created for it in the child directory.

The files keep on coming. As soon as the new files come, they need to get
processed, outputted into the output directory and the tag file as well as
the processed file should be removed.
So our perl script should run infinitely.

At one time, we need to process only one file.

Hope this helps and u r able to help me out.

Thanks,
Mihir

Mr. Shawn H. Corey

2007-08-02, 7:00 pm

Mihir Kamdar wrote:
> As soon as the data file is updated completely, the tag file is touched with
> the same name in the child directory.
>
> This process is automatic. No manual touching of tag files.
>
> As per my requirement, in my code, I am trying to open the child directory,
> read the tag name, open the parent directory,and corresponding to that tag,
> pick up the data file, process it and write the output to another directory.
>
> After processing, both the tag file as well as data file needs to be
> deleted. We should only have the output file in the output directory.
>
> There is no possibility of updating or appending any data file in the parent
> directory once a tag file is created for it in the child directory.
>
> The files keep on coming. As soon as the new files come, they need to get
> processed, outputted into the output directory and the tag file as well as
> the processed file should be removed.
> So our perl script should run infinitely.
>
> At one time, we need to process only one file.
>
> Hope this helps and u r able to help me out.

Just a rough version:

#!/usr/bin/perl

use strict;
use warnings;

use File::Basename;
use File::Copy;

my $TagDir = 'TBD';
my $FilesDir = 'TBD';
my $TmpDir = 'TBD';
my $OutputDir = 'TBD';

for(;;) {
my @tag_files = glob( "$TagDir/*" );
my %files = map { basename( $_ ), $FilesDir . basname( $_ ) } @tag_files;
for my $file ( keys %files ){
move( $files{$file}, "$TmpDir/$file" ) or die "cannot move file $file: $!";
$files{$file} = "$TmpDir/$file";
}
for my $file ( keys %files ){
my $output_file = "$OutputDir/$file";
open my $in_fh, '<', $files{$file} or die "cannot open $file: $!";
open my $out_fh, '>', $output_file or die "cannot open $output_file: $!";
while( <$in_fh> ){
# process file
# print output
}
close $in_fh;
close $out_fh;
unlink $files{$file}, "$TagDir/$file";
}
sleep 4;
}

__END__

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Nobull67@Gmail.Com

2007-08-02, 7:00 pm

On Aug 2, 10:48 am, kamdarmihi...@gmail.com (Mihir Kamdar) wrote:

> my $line;
> my %hash;
> my @file;
> my $key ;
> my $value ;
> my %times ;

You appear to be suffering from a nasty case of premature declaration.

Looking at your code it appears that three of those are harmlessly
being declared in the wrong scope. One is never used.

And most significantly, one is harmfully declared in the wrong scope -
i.e. the fact that you've declared it in the wrong scope will cause
your program to malfunction (assuming I've correctly understood what
you are trying to do).

The remaining one of them may be in the right scope because although
it is mentioned again in your script you never put anything in it so I
can't deduce its intent.

I won't say which is which because you should try to get into the
habit of _always_ declaring _all_ variables in the right scope even on
those occasions where getting it wrong would not actually stop your
program working.
[color=darkred]
>

The argument to readline() should be a file handle not a filename.

Note: You should enable warnings and strictures at the top of your
scripts. I'm note sure, but I suspect, Perl would have found your
mistake for you if you'd let it.

Mihir Kamdar

2007-08-02, 7:00 pm

On 8/2/07, nobull67@gmail.com <nobull67@gmail.com> wrote:
>
> On Aug 2, 10:48 am, kamdarmihi...@gmail.com (Mihir Kamdar) wrote:
>
>
> You appear to be suffering from a nasty case of premature declaration.
>
> Looking at your code it appears that three of those are harmlessly
> being declared in the wrong scope. One is never used.
>
> And most significantly, one is harmfully declared in the wrong scope -
> i.e. the fact that you've declared it in the wrong scope will cause
> your program to malfunction (assuming I've correctly understood what
> you are trying to do).
>
> The remaining one of them may be in the right scope because although
> it is mentioned again in your script you never put anything in it so I
> can't deduce its intent.
>
> I won't say which is which because you should try to get into the
> habit of _always_ declaring _all_ variables in the right scope even on
> those occasions where getting it wrong would not actually stop your
> program working.
>
>
> The argument to readline() should be a file handle not a filename.
>
> Note: You should enable warnings and strictures at the top of your
> scripts. I'm note sure, but I suspect, Perl would have found your
> mistake for you if you'd let it.
>
>
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> http://learn.perl.org/
>
>
> Hi,

The script is working now. In my earlier code, I guess,I was reading from
the parent directory all the files at once without comparing each one from
the child directory. With a small modification, my code is working which is
as follows:-

#!/usr/bin/perl

my $child_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
;
my $parent_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
;
my $write_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
;
my $line;
my %hash;
my @file;
my $key ;
my $value ;
my %times ;
while () {
opendir my $dh, $child_path or die $!;
while (my $file = readdir $dh) {
my $fname = "$child_path/$file";
next unless -f $fname;
unless (exists $times{$file}){
opendir my $dh1,$parent_path or die $!;
open (my $IN_FILE,"<","$parent_path/$file")
or die $!." Parent file not found for $parent_path/$file";
while ($line=readline($IN_FILE))
{
my @cdr=split (/,/, $line) ;

$hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
if u want.
}
close $IN_FILE ;
closedir $dh1 ;
open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
while (($key, $value) = each %hash)
{
print $OUT_FILE $value;
}
close $OUT_FILE;
}
unlink("$parent_path/$file") ;
unlink("$child_path/$file") ;
}
closedir $dh;
}

Please comment on the above code and tell me if it has any potential
pitfalls. Right now I am able to test this for just few sample files. But in
real environment, it has to pick up millions of files, process them and
write the output.

Your comments will be highly appreciated.

Thanks,
Mihir

Chas Owens

2007-08-02, 7:00 pm

On 8/2/07, Mihir Kamdar <kamdarmihir06@gmail.com> wrote:

> #!/usr/bin/perl
>
> my $child_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
> ;
> my $parent_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
> ;
> my $write_path =
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
> ;
> my $line;

This declaration is being done too soon. Variables should have the
smallest possible scope. $line is being used only with a while loop
below. It should be declared then.

> my %hash;
> my @file;

@file is never used

> my $key ;
> my $value ;

These variables are also being declared too soon.

> my %times ;
> while () {

Infinite loops are a bad idea. It is better to give it an exit
condition that can be triggered by a signal. The following code will
exit if the program receives either SIGINT or SIGTERM (control-c or
kill -15):

my $continue = 1;
$SIG{INT} = $SIG{TERM} = sub { $continue = 0 };
while ($continue) {
print localtime() . "\n";
sleep 1;
}

This also brings up another problem with your code: you don't sleep
inside the loop. This will cause your code to consume all available
CPU cycles. This is generally considered not a good idea.

> opendir my $dh, $child_path or die $!;
> while (my $file = readdir $dh) {
> my $fname = "$child_path/$file";
> next unless -f $fname;
> unless (exists $times{$file}){
> opendir my $dh1,$parent_path or die $!;
> open (my $IN_FILE,"<","$parent_path/$file")
> or die $!." Parent file not found for $parent_path/$file";
> while ($line=readline($IN_FILE))
> {
> my @cdr=split (/,/, $line) ;
>
> $hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
> if u want.

This is not a good idea. The correct way to do this is either
$hash{$cdr[2]}{$cdr[3]}{$cdr[6]}{$cdr[7]
} = $line;
or
$hash{"@cdr[2,3,6,7]"} = $line;

> }
> close $IN_FILE ;
> closedir $dh1 ;
> open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
> while (($key, $value) = each %hash)
> {
> print $OUT_FILE $value;
> }
> close $OUT_FILE;
> }
> unlink("$parent_path/$file") ;
> unlink("$child_path/$file") ;
> }
> closedir $dh;
> }
>
> Please comment on the above code and tell me if it has any potential
> pitfalls. Right now I am able to test this for just few sample files. But in
> real environment, it has to pick up millions of files, process them and
> write the output.
>
> Your comments will be highly appreciated.
>
> Thanks,
> Mihir
>

Mihir Kamdar

2007-08-03, 3:59 am

On 8/2/07, Chas Owens <chas.owens@gmail.com> wrote:
>
> On 8/2/07, Mihir Kamdar <kamdarmihir06@gmail.com> wrote:
>
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
>
> This declaration is being done too soon. Variables should have the
> smallest possible scope. $line is being used only with a while loop
> below. It should be declared then.
>
>
> @file is never used
>
>
> These variables are also being declared too soon.
>
>
> Infinite loops are a bad idea. It is better to give it an exit
> condition that can be triggered by a signal. The following code will
> exit if the program receives either SIGINT or SIGTERM (control-c or
> kill -15):
>
> my $continue = 1;
> $SIG{INT} = $SIG{TERM} = sub { $continue = 0 };
> while ($continue) {
> print localtime() . "\n";
> sleep 1;
> }
>
> This also brings up another problem with your code: you don't sleep
> inside the loop. This will cause your code to consume all available
> CPU cycles. This is generally considered not a good idea.
>
> $IN_FILE,"<","$parent_path/$file")
> fields
>
> This is not a good idea. The correct way to do this is either
> $hash{$cdr[2]}{$cdr[3]}{$cdr[6]}{$cdr[7]
} = $line;
> or
> $hash{"@cdr[2,3,6,7]"} = $line;
>
> $!;
> But in
>
Hi,

I appreciate your comments and have tried to make changes as per your
suggestions. Plz have a look at the below and comment if any further changes
are required. This is turning out to be a good learning for me.

#!/usr/bin/perl -w

my $child_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
;
my $parent_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
;
my $write_path =
'/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
;

my %times ;
my $continue = 1;

$SIG{INT} = $SIG{TERM} = sub { $continue = 0 };

while ($continue) {
opendir my $dh, $child_path or die $!;
while (my $file = readdir $dh) {
my $fname = "$child_path/$file";
next unless -f $fname;
unless (exists $times{$file}){
my $line;
my %hash;
opendir my $dh1,$parent_path or die $!;
open (my $IN_FILE,"<","$parent_path/$file")
or die $!." Parent file not found for $parent_path/$file";
while ($line=readline($IN_FILE))
{
my @cdr=split (/,/, $line) ;

$hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
if u want.
}
close $IN_FILE ;
closedir $dh1 ;
open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
while (my($key, $value) = each %hash)
{
print $OUT_FILE $value;
}
close $OUT_FILE;
}
unlink("$parent_path/$file") ;
unlink("$child_path/$file") ;
}
closedir $dh;
sleep(2) ;
}

Jeff Pang

2007-08-03, 3:59 am

-----Original Message-----
>From: Mihir Kamdar <kamdarmihir06@gmail.com>

> $hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields if u want.

There are (maybe) two problems above.
1. when using hash slice,the form is @hash{'key1','key2'...},not $hash{'key1','key2'...}
2. when you say @hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line,only the first key ($cdr[2]) has got value,the other keys would get undef as their values,since $line is a scalar,but the statement expect a list on the right of '=' I think.

--
Jeff Pang <pangj@earthlink.net>
http://home.arcor.de/jeffpang/

Chas Owens

2007-08-03, 3:59 am

On 8/3/07, Jeff Pang <pangj@earthlink.net> wrote:
>
>
> -----Original Message-----
>
>
> There are (maybe) two problems above.
> 1. when using hash slice,the form is @hash{'key1','key2'...},not $hash{'key1','key2'...}
> 2. when you say @hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line,only the first key ($cdr[2])
> has got value,the other keys would get undef as their values,since $line is a scalar,
> but the statement expect a list on the right of '=' I think.

I don't think he is trying to use the slice notation. He is using the
multidimensional array emulation. This not really a good practice
because it is easy to confuse it with slices. A better way to get
the same functionality is

$hash{"@cdr[2,3,6,7]"} = $line;

The quotes act as a signal that you aren't looking for a slice.

from perldoc perlvar
$; The subscript separator for multidimensional array emulation.
If you refer to a hash element as

$foo{$a,$b,$c}

it really means

$foo{join($;, $a, $b, $c)}

Jeff Pang

2007-08-03, 3:59 am

-----Original Message-----
>From: Chas Owens <chas.owens@gmail.com>
>Sent: Aug 3, 2007 3:21 AM
>To: Jeff Pang <pangj@earthlink.net>
>Cc: beginners@perl.org
>Subject: Re: Problem with my code
>
>On 8/3/07, Jeff Pang <pangj@earthlink.net> wrote:
>
>I don't think he is trying to use the slice notation. He is using the
>multidimensional array emulation. This not really a good practice
>because it is easy to confuse it with slices. A better way to get
>the same functionality is
>
>$hash{"@cdr[2,3,6,7]"} = $line;
>
>The quotes act as a signal that you aren't looking for a slice.
>

Yes I know this expression too.But why one should write something like this?It's too out of date.

--
Jeff Pang <pangj@earthlink.net>
http://home.arcor.de/jeffpang/

Mr. Shawn H. Corey

2007-08-03, 7:59 am

Chas Owens wrote:
> On 8/3/07, Jeff Pang <pangj@earthlink.net> wrote:
>
> I don't think he is trying to use the slice notation. He is using the
> multidimensional array emulation. This not really a good practice
> because it is easy to confuse it with slices. A better way to get
> the same functionality is
>
> $hash{"@cdr[2,3,6,7]"} = $line;
>
> The quotes act as a signal that you aren't looking for a slice.
>
> from perldoc perlvar
> $; The subscript separator for multidimensional array emulation.
> If you refer to a hash element as
>
> $foo{$a,$b,$c}
>
> it really means
>
> $foo{join($;, $a, $b, $c)}
>

But the expression $hash{"@cdr[2,3,6,7]"} is the same as $hash{join($",@cdr[2,3,6,7])} You have just replaced one special variable with another and $" is a bad choice since it's default is a space character, which may easily appear in one (or all) of @cd
r[2,3,6,7]

A better choice is $hash{$cdr[2]}{$cdr[3]}{$cdr[6]}{$cdr[7]
} but you would need four for-loops to get to the value.

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Chas Owens

2007-08-03, 7:02 pm

On 8/3/07, Mr. Shawn H. Corey <shawnhcorey@magma.ca> wrote:
snip
> But the expression $hash{"@cdr[2,3,6,7]"} is the same as $hash{join($",@cdr[2,3,6,7])} You have just replaced one special variable with another and $" is a bad choice since it's default is a space character, which may easily appear in one (or all) of @
cdr[2,3,6,7]
>
> A better choice is $hash{$cdr[2]}{$cdr[3]}{$cdr[6]}{$cdr[7]
} but you would
> need four for-loops to get to the value.
snip

The use of the special variable $; is not why
$foo{$a[1],$a[2],$a[3],$a[4]} is a bad choice; it is a bad choice
because it is hard to tell if they meant to use
@foo{$a[1],$a[2],$a[3],$a[4]} but screwed up the sigil. The spaces
are only important if you want to recover the individual values from
the key, and, since you are storing the entire record, you don't even
need the key to get those values. Why would you create a nested hash?
This does not appear to be a tree data. If you want to do it long
hand because you don't want the spaces that using an array slice will
add then simply say

$hash{"{$cdr[2]$cdr[3]$cdr[6]$cdr[7]"}

Mr. Shawn H. Corey

2007-08-03, 7:02 pm

Chas Owens wrote:
> The use of the special variable $; is not why
> $foo{$a[1],$a[2],$a[3],$a[4]} is a bad choice; it is a bad choice
> because it is hard to tell if they meant to use
> @foo{$a[1],$a[2],$a[3],$a[4]} but screwed up the sigil. The spaces
> are only important if you want to recover the individual values from
> the key, and, since you are storing the entire record, you don't even
> need the key to get those values. Why would you create a nested hash?
> This does not appear to be a tree data. If you want to do it long
> hand because you don't want the spaces that using an array slice will
> add then simply say
>
> $hash{"{$cdr[2]$cdr[3]$cdr[6]$cdr[7]"}
>
>

#!/usr/bin/perl

use strict;
use warnings;

use Data::Dumper;

my %hash1 = ();
my %hash2 = ();

my $key1 = 'ab';
my $key2 = 'c';
$hash1{"$key1$key2"} = 'some value';
$hash2{$key1}{$key2} = 'some value';
print "\%hash1 = ", Dumper \%hash1;
print "\%hash2 = ", Dumper \%hash2;

my $key3 = 'a';
my $key4 = 'bc';
$hash1{"$key3$key4"} = 'some other value';
$hash2{$key3}{$key4} = 'some other value';
print "\%hash1 = ", Dumper \%hash1;
print "\%hash2 = ", Dumper \%hash2;

__END__

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Xavier Noria

2007-08-03, 7:02 pm

El Aug 3, 2007, a las 1:45 PM, Mr. Shawn H. Corey escribi=F3:

> But the expression $hash{"@cdr[2,3,6,7]"} is the same as $hash{join=20
> ($",@cdr[2,3,6,7])} You have just replaced one special variable =20
> with another and $" is a bad choice since it's default is a space =20
> character, which may easily appear in one (or all) of @cdr[2,3,6,7]
>
> A better choice is $hash{$cdr[2]}{$cdr[3]}{$cdr[6]}{$cdr[7]
} but =20
> you would need four for-loops to get to the value.

To be able to use arrays as keys there are a couple of modules out =20
there. One is mine, based on pack/unpack :

http://search.cpan.org/~fxn/Hash-Mu....06/MultiKey.pm

and there's another one that is implemented as nested hashes, but I =20
can't remember its name right now, does anybody know it?

-- fxn

Chas Owens

2007-08-03, 7:02 pm

On 8/3/07, Mr. Shawn H. Corey <shawnhcorey@magma.ca> wrote:
> Chas Owens wrote:
>
> #!/usr/bin/perl
>
> use strict;
> use warnings;
>
> use Data::Dumper;
>
> my %hash1 = ();
> my %hash2 = ();
>
> my $key1 = 'ab';
> my $key2 = 'c';
> $hash1{"$key1$key2"} = 'some value';
> $hash2{$key1}{$key2} = 'some value';
> print "\%hash1 = ", Dumper \%hash1;
> print "\%hash2 = ", Dumper \%hash2;
>
> my $key3 = 'a';
> my $key4 = 'bc';
> $hash1{"$key3$key4"} = 'some other value';
> $hash2{$key3}{$key4} = 'some other value';
> print "\%hash1 = ", Dumper \%hash1;
> print "\%hash2 = ", Dumper \%hash2;
snip

Note that my advice is to use

$hash3{"@key"}

Which does not suffer from that problem and that I specifically said
"you don't want the spaces that using an array slice will add".

Mr. Shawn H. Corey

2007-08-03, 7:02 pm

Chas Owens wrote:
> Note that my advice is to use
>
> $hash3{"@key"}
>
> Which does not suffer from that problem and that I specifically said
> "you don't want the spaces that using an array slice will add".
>
>

Whnever you use a composite key, you have the possibility of a collision.

#!/usr/bin/perl

use strict;
use warnings;

use Data::Dumper;

my %hash1 = ();
my %hash2 = ();

my @key1 = ( 'a b', 'c' );
my @key2 = ( 'a', 'b c' );

$hash1{"@key1"} = 'some value';
$hash1{"@key2"} = 'some other value';

$hash2{$key1[0]}{$key1[1]} = 'some value';
$hash2{$key2[0]}{$key2[1]} = 'some other value';

print "\%hash1 = ", Dumper \%hash1;
print "\%hash2 = ", Dumper \%hash2;

__END__

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Chas Owens

2007-08-03, 7:02 pm

On 8/3/07, Mr. Shawn H. Corey <shawnhcorey@magma.ca> wrote:
snip
> Whnever you use a composite key, you have the possibility of a collision.
snip

Okay, I will concede that, but there is generally a character that is
safe (usually the character the line was split on). In this case
$hash{join ',', @cdr[2,3,6,7]} = $line;
is safe due to the fact that @cdr was created using split /,/, so I
would recommend that over a multidimensional hash.

Mr. Shawn H. Corey

2007-08-03, 7:02 pm

Chas Owens wrote:
> Okay, I will concede that, but there is generally a character that is
> safe (usually the character the line was split on). In this case
> $hash{join ',', @cdr[2,3,6,7]} = $line;
> is safe due to the fact that @cdr was created using split /,/, so I
> would recommend that over a multidimensional hash.
>

Safe until the format of the input changes. If, for example, you tried this on a CSV (comma-separated-values) file, it would fail since commas can be escaped. That is why $; is set to the default value of ASCII \034, a non-printable control character th
at is unlikely to appear in text. But if you have your heart set on a composite key:

{
local( $" ) = $;;
$hash{"@cdr[2,3,6,7]"} = $line;
}

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Chas Owens

2007-08-03, 7:02 pm

On 8/3/07, Mr. Shawn H. Corey <shawnhcorey@magma.ca> wrote:
> Chas Owens wrote:
>
> Safe until the format of the input changes. If, for example, you tried
> this on a CSV (comma-separated-values) file, it would fail since
> commas can be escaped. That is why $; is set to the default value
> of ASCII \034, a non-printable control character that is unlikely to
> appear in text. But if you have your heart set on a composite key:
>
> {
> local( $" ) = $;;
> $hash{"@cdr[2,3,6,7]"} = $line;
> }
snip

They composite key breaks at the same time as the split, so both would
need to be modified and \034 isn't any safer. It all depends on your
data. Also, the short cut of using "@cdr[2,3,6,7]" is pointless if
you aren't going to use the default value of $", it is shorter to type

$hash{join "\034", @cdr[2,3,6,7]} = $line;

Nobull67@Gmail.Com

2007-08-03, 7:02 pm

On Aug 3, 6:54 am, kamdarmihi...@gmail.com (Mihir Kamdar) wrote:

> I appreciate your comments and have tried to make changes as per your
> suggestions. Plz have a look at the below and comment if any further changes
> are required. This is turning out to be a good learning for me.

Most of the things I would point out have already been mentioned in
this thread but here are a couple I haven't noticed.

> #!/usr/bin/perl -w

The -w switch has now been replaced with

use warnings;

Unless compatibility with old versions of Perl is an issue you should
use the new version.

You should also

use strict;

> unless (exists $times{$file}){

Nothing ever sets anything into %times so that condition is always
false.

Unless you are actually expecting %times to contain false values then
it's more idiomatic to omit the exists()

> my $line;

You are still have a very minor case of premature declaration.

> my %hash;
> opendir my $dh1,$parent_path or die $!;
> open (my $IN_FILE,"<","$parent_path/$file")
> or die $!." Parent file not found for $parent_path/$file";
> while ($line=readline($IN_FILE))
> {
> my @cdr=split (/,/, $line) ;
>
> $hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key fields
> if u want.
> }
> close $IN_FILE ;
> closedir $dh1 ;
> open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
> while (my($key, $value) = each %hash)
> {
> print $OUT_FILE $value;
> }
> close $OUT_FILE;

If you are not bothering to checking the return value of close() then
you may as well simply omit the explicit close() if the handle is
about to go out of scope anyhow. This applies to all close() and
closedir() in your script.

> }

BTW: Your indentation is a bit random. In small programs like this
it's not much of an issue.

Perl Pra

2007-08-04, 7:58 am

Perl Pra

2007-08-04, 7:58 am

hi Mihir Kamdar,

Please the following highlighted.

I have tested it. this excatly does what you need.

configure the parent_path,child_path and out_path to your corresponding
paths.

_____________________ BEGIN

use strict;
use File::Path;
use File::Copy;
use warnings;

my $parent_path="C:/parent";
my $child_path="C:/child";
my $out_path="C:/output";
my @parent_files=<$parent_path/*>;

######### THIS SECTION ACTS AS the Touch Command Check the Files in Parent
Directory and Created the zero bye files in the Child directory#############

foreach my $file (@parent_files)
{
my @arr=$file=~ /(\w+\.\w+)$/;
if (-f "$child_path/$arr[0]" )
{
}
else
{

open my $W_H, '>', "$child_path/$arr[0]" || die "Could not Create
the file $child_path/$arr[0]\n";
close $W_H;
}

}

########################################
########################################
########################################
##############

############################## this section Checks the files in the child
directory, Takes the same file from the parent directory removes duplicates
from the file and creates the file in output directory
########################################
########################################
#####3

my @child_files=<$child_path/*>;

foreach my $file (@child_files)
{
my @arr=$file=~ /(\w+\.\w+)$/;
my %uniques;
open my $R_H, '<', "$parent_path/$arr[0]";
my @file_content=<$R_H>;
close $R_H;
foreach my $line (@file_content) { $uniques{$line}++; }
open my $W_H, '>', "$out_path/$arr[0]";
for(keys(%uniques)) { print $W_H "$_\n"; }
close $W_H;
}

########################################
########################################
########################################
##############

_________________________________ END

Regards,
Siva

On 8/2/07, Mihir Kamdar <kamdarmihir06@gmail.com> wrote:
>
> Hi,
>
> I have a requirement. There are files in the parent directory and they are
> touched in the child directory. So in child directory there are 0 byte
> files
> having the same name as the ones in the parent directory. I need to read
> files in the child directory, and for each file present in the child
> directory, I need to take that file from the parent directory, process
> that
> file(remove duplicate lines from that file), and then write the output for
> each of the input files in the output directory.
>
> I have written the code as below but it gives 0 byte files as output.
> Please
> suggest what is wrong with the below code.
> I have a fear that instead of processing files from parent directory, it
> is
> processing files from the child directory.
>
> I am a newbie at perl. So i might have made silly mistakes. Request you to
> correct them.
>
>
>
> #!/usr/bin/perl
>
> my $child_path =
>
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/child'
> ;
> my $parent_path =
>
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files'
> ;
> my $write_path =
>
> '/home/user71/RangerDatasource/Customization/TelekomMalaysia/Scripts/Tests/cprogs/files/output'
> ;
> my $line;
> my %hash;
> my @file;
> my $key ;
> my $value ;
>
> while () {
>
> opendir my $dh, $child_path or die $!;
>
> while (my $file = readdir $dh) {
>
> my $fname = "$child_path/$file";
> next unless -f $fname;
> unless (exists $times{$file}){
>
> opendir my $dh1,$parent_path or die $!;
>
> while (my $file = readdir $dh1) {
>
> while ($line=readline($file))
> {
> my @cdr=split (/,/, $line) ;
>
> $hash{$cdr[2],$cdr[3],$cdr[6],$cdr[7]}=$
line; #Add some more cdr key
> fields
> if u want.
> }
> close $file ;
> }
> closedir $dh1 ;
>
> open (my $OUT_FILE,">","$write_path/$file.out") or die $!;
> while (($key, $value) = each %hash)
> {
> print $OUT_FILE $value;
> }
> close $OUT_FILE;
> }
> unlink("$parent_path/$file") ;
> unlink("$child_path/$file") ;
>
>
> }
> closedir $dh;
> }
>

John W. Krahn

2007-08-04, 7:01 pm

perl pra wrote:
> hi Mihir Kamdar,
>
> Please the following highlighted.
>
> I have tested it. this excatly does what you need.
>
> configure the parent_path,child_path and out_path to your corresponding
> paths.
>
> _____________________ BEGIN
>
>
>
>
> use strict;
> use File::Path;
> use File::Copy;
> use warnings;
>
> my $parent_path="C:/parent";
> my $child_path="C:/child";
> my $out_path="C:/output";
> my @parent_files=<$parent_path/*>;
>
>
> ######### THIS SECTION ACTS AS the Touch Command Check the Files in Parent
> Directory and Created the zero bye files in the Child directory#############
>
> foreach my $file (@parent_files)
> {
> my @arr=$file=~ /(\w+\.\w+)$/;

Why are you doing this? What if the file name contains characters that don't
match \w? You should use File::Basename::basename to extract the file name
from the path or use opendir/readdir so that you don't have to separate the
file name from the path.

> if (-f "$child_path/$arr[0]" )
> {
> }
> else
> {
>
> open my $W_H, '>', "$child_path/$arr[0]" || die "Could not Create
> the file $child_path/$arr[0]\n";

Because of the high precedence of the '||' operator the die() will *never*
execute! You need to either use parentheses:

open( my $W_H, '>', "$child_path/$arr[0]" ) || die "Could not
Create the file '$child_path/$arr[0]' $!";

or use the low precedence 'or' operator:

open my $W_H, '>', "$child_path/$arr[0]" or die "Could not Create
the file '$child_path/$arr[0]' $!";

> close $W_H;
> }
>
> }
>
> ########################################
########################################
########################################
##############
>
>
> ############################## this section Checks the files in the child
> directory, Takes the same file from the parent directory removes duplicates
> from the file and creates the file in output directory
> ########################################
########################################
#####3
>
> my @child_files=<$child_path/*>;
>
>
> foreach my $file (@child_files)
> {
> my @arr=$file=~ /(\w+\.\w+)$/;

See above.

> my %uniques;
> open my $R_H, '<', "$parent_path/$arr[0]";

You should *always* verify that the file opened correctly.

> my @file_content=<$R_H>;
> close $R_H;
> foreach my $line (@file_content) { $uniques{$line}++; }
> open my $W_H, '>', "$out_path/$arr[0]";

You should *always* verify that the file opened correctly.

> for(keys(%uniques)) { print $W_H "$_\n"; }
> close $W_H;
> }
>
> ########################################
########################################
########################################
##############
>
>
> _________________________________ END

John
--
Perl isn't a toolbox, but a small machine shop where you
can special-order certain sorts of tools at low cost and
in short order. -- Larry Wall

Perl Pra

2007-08-06, 3:59 am

hi jhon,

If I use use opendir/readdir I am getting the default directories . and ..
into the directoy handle.

Is there any way tha i could eliminate this.

For example if i want to copy all contentents of the directory(including
sub directories)
I open a directory Handle using opendir and read it usign readdir, Now my
dirhandle contains default directories . and .. which i cannot copy.

Is there any way that i can elimate the default directories . and .. while
reading the direcotry from a directory handle.

Thanks
Siva

Jeff Pang

2007-08-06, 3:59 am

-----Original Message-----
>From: Jeff Pang <pangj@earthlink.net>
>
>while(<DIR> ) {
> next if /^\.+$/; #or
> next if $_ eq '.' or $_ eq '..';
>}
>

Sorry for the mistake,it's,

while(my $obj = readdir DIR) {
next if $obj =~ /^\.+$/; #or
next if $obj eq '.' or $obj eq '..';
}

--
Jeff Pang <pangj@earthlink.net>
http://home.arcor.de/jeffpang/

Jeff Pang

2007-08-06, 3:59 am

-----Original Message-----
>From: perl pra <perlpra@gmail.com>
>Sent: Aug 6, 2007 12:29 PM
>To: "John W. Krahn" <krahnj@telus.net>
>Cc: Perl beginners <beginners@perl.org>
>Subject: Re: Problem with my code
>
>hi jhon,
>
>If I use use opendir/readdir I am getting the default directories . and ..
>into the directoy handle.
>
>Is there any way tha i could eliminate this.
>
>
>For example if i want to copy all contentents of the directory(including
>sub directories)
>I open a directory Handle using opendir and read it usign readdir, Now my
>dirhandle contains default directories . and .. which i cannot copy.
>
>Is there any way that i can elimate the default directories . and .. while
>reading the direcotry from a directory handle.
>

Hello,

I just think there're some ways to skip them.like:

while(<DIR> ) {
next if /^\.+$/; #or
next if $_ eq '.' or $_ eq '..';
}

or using File::Copy it would do this thing automatically for you?

--
Jeff Pang <pangj@earthlink.net>
http://home.arcor.de/jeffpang/

Mumia W.

2007-08-06, 3:59 am

On 08/06/2007 12:32 AM, Jeff Pang wrote:
> [...]
> while(my $obj = readdir DIR) {
> next if $obj =~ /^\.+$/; #or [...]

More, precisely, you might use this:

next if $obj =~ /\A\.\.?\z/;

Perl Pra

2007-08-06, 3:59 am

Thanks a lot

Regards,
Siva

On 8/6/07, Mumia W. <mumia.w.18.spam+nospam@earthlink.net> wrote:
>
> On 08/06/2007 12:32 AM, Jeff Pang wrote:
>
> More, precisely, you might use this:
>
> next if $obj =~ /\A\.\.?\z/;
>
>
>
>
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> http://learn.perl.org/
>
>
>

Mr. Shawn H. Corey

2007-08-06, 7:58 am

perl pra wrote:
> hi jhon,
>
> If I use use opendir/readdir I am getting the default directories . and ..
> into the directoy handle.
>
> Is there any way tha i could eliminate this.
>
>
> For example if i want to copy all contentents of the directory(including
> sub directories)
> I open a directory Handle using opendir and read it usign readdir, Now my
> dirhandle contains default directories . and .. which i cannot copy.
>
> Is there any way that i can elimate the default directories . and .. while
> reading the direcotry from a directory handle.
>
> Thanks
> Siva
>

If you want the contents of a directory and all its sub-directories, you should use File::Find It's simple to use and it takes care of some problems like cyclic symbolic links. (And I hope you're using File::Copy to copy the files.)

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Dr.Ruud

2007-08-08, 7:00 pm

"Mumia W." schreef:
> Jeff Pang wrote:

>
> More, precisely, you might use this:
>
> next if $obj =~ /\A\.\.?\z/;

Please see `perldoc -f -f`.

--
Affijn, Ruud

"Gewoon is een tijger."

Sponsored Links