PERL Beginners - Re: Passing each of the files in a directory as parameter to the

Author

Re: Passing each of the files in a directory as parameter to the

Rob Dixon

2007-08-01, 7:00 pm

Mihir Kamdar wrote:
>
> I have a requirement to read each of the files from a directory and pass
> each one of them as a parameter to my perl script one by one. The
> requirement is such that my perl script should keep running in the
> background and it should take each of the files as they enter into the
> target directory, pass that as a parameter to my perl script, process it and
> write the output to a different directory.
>
> Ex:- $cd success
> success>$ls
> success>file1 file2 file3 file4 file5
>
> $cd
> home>perl test.pl /home/success/file1
> home>perl test.pl /home/success/file2
> ...
> ...
>
> Please suggest how to go about this. Are there any perl modules to scan
> through a directory and pick each of the files to process.
>
> Once the file is taken from the input directory and processed and output
> written to a different output directory, I can delete that file from the
> input directory.

A Perl program can easily find what files are in a directory and process and
delete each of them. I think that is the way to go rather than having an
external program calling a Perl script to process the files one by one. Is
there any reason why you can't do this?

Rob

Mr. Shawn H. Corey

2007-08-01, 7:00 pm

Mihir Kamdar wrote:
> Also, these input files will keep on coming from some source. So, my perl
> file has to run continuously and pick each of the files as they come.

How quickly does the Perl program have to process these files? ASAP, once an hour, once a day?

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Rob Dixon

2007-08-01, 7:00 pm

Mihir Kamdar wrote:
>
> On 8/1/07, Rob Dixon <rob.dixon@350.com> wrote:
>
> My requirement is like this:-
>
> There is a parent directory where the input files will be stored. These
> files will keep on coming into the parent directory. There is a child
> directory where these input files will just get touched(0 byte files).
>
> I have to read the name of the file from the child directory and then go to
> the parent directory to process it. Only those files in the parent directory
> will be processed which have been touched in the child directory.
>
> Also, these input files will keep on coming from some source. So, my perl
> file has to run continuously and pick each of the files as they come.

(Please bottom-post your responses to this list so that long threads remain
comprehensible. Thank you.)

So the child directory contains files with the same name as the main directory
but of zero size, and you must process files with the whose name matches those
that are either new or whose access time has changed. (Is this what touching the
file changes? My speciality is not Unix.) Is that right?

Something like the code below should do the trick.

HTH,

Rob

my $path = 'child_directory';

my %times;

while () {

opendir my $dh, $path or die $!;

while (my $file = readdir $dh) {

my $fname = "$path/$file";
next unless -f $fname;
my $access = (stat $fname)[8];

unless (exists $times{$file} and $times{$file} == $access) {
process($file);
$times{$file} = $access;
}
}

closedir $dh;

sleep 1;
}

Rob Dixon

2007-08-01, 7:00 pm

Mihir Kamdar wrote:
>
> Can you explain with reference to your code, what does this do:
>
> my $access = (stat $fname)[8];

In my last post I asked you to please bottom-post your responses to
this group. That means to put your reply after the text of the
message you are replying to.

If you look at

perldoc -f stat

you will see that stat() returns 13 items of information about a file.
The ninth (at index 8) is the time the file was last accessed , which I
am storing in $access.

HTH,

Rob

Mr. Shawn H. Corey

2007-08-01, 7:00 pm

Mihir Kamdar wrote:
> It has to be ASAP Shawn......but if that's not possible, as a workaround
> we may also look at every half hour or something like that...
>
> do you have any suggestions in mind?

Well, if I could change anything, I'd change my program to a deamon and have it listen on a socket. The program that writes these files would, in addition, request a session with my deamon and send it (via the socket) the entire file. I would also add t
o its startup a procedure to cover any lapses it had between when it last ran and now (optionally, you could turn this off).

If you can't do that, then a loop that sleeps every 4 seconds to check and update if anything new happens.

1 + 1 = 1, with rounding
1 + 1 = 2, with and without rounding
1 + 1 = 3, with rounding

The modification times (via touch) has a resolution of +/- 1 second. Any sleep period of less than 4 seconds have the potential of skipping some files or repeat some files (depending on how it's coded).

--
Just my 0.00000002 million dollars worth,
Shawn

"For the things we have to learn before we can do them, we learn by doing them."
Aristotle

Sponsored Links