AWK - awk instead of shell?

Author

awk instead of shell?

L. Kliemann

2007-03-12, 9:57 pm

Recently, I started programming in awk, and I like it. Many of my programs
are written in shell so far, some of them fairly complex. I like shell for
its ability to directly interact with the filesystem via the tools cp, cd,
rm, mv, etc. However, shell can be very slow (and awk, on the contrary, seems
very fast to me) and also, shell misses some idioms, like associative arrays.
My idea is to try to use awk wherever I used to use shell in the past. Maybe
it is a bad idea, but I'd like to give it a try. Are there any resources on
this topic? Most importantly, I need a clean way to iterate over some files
in a directory, or over a directory tree.

Regards,
Lasse

Janis

2007-03-12, 9:57 pm

On 12 Mrz., 16:13, "L. Kliemann" <stu33...@mail.uni-kiel.de> wrote:
> Recently, I started programming in awk, and I like it. Many of my
> programs are written in shell so far, some of them fairly complex.
> I like shell for its ability to directly interact with the
> filesystem via the tools cp, cd, rm, mv, etc.

Each language has it's advantages and application domain, shell and
awk.

> However, shell can be very slow

That depends on what shell you use; modern shells are quite fast in
what they do, in what they are supposed to do.

> (and awk, on the contrary, seems very fast to me) and also, shell
> misses some idioms, like associative arrays.

The modern shells support also associative arrays.

There's a point if you want to program portably, though, because the
associative arrays are standard in awk but a (non-POSIX) extension
in the shells.

> My idea is to try to use awk wherever I used to use shell in the
> past. Maybe it is a bad idea, but I'd like to give it a try.

I think this is indeed a bad idea since shells and awk don't cover
the same domain. You already noticed some differences. While you can
do some rudimentary data processing with shells and do some process
and file processing in awk (though often only through a shell
interface "system()") they have their strengths the other way round,
use awk for your data processing and shells to handle the OS
ressources.

Shell and awk are no real competitors, they complement each other.

> Are there any resources on this topic? Most importantly, I need a
> clean way to iterate over some files in a directory, or over a
> directory tree.

While possible in awk that's a typical shell feature; you usually
either let the shell expand the filenames to let awk operate on
them, or even use an external command to find the files.

Janis

>
> Regards, Lasse

Vassilis

2007-03-12, 9:57 pm

=CF/=C7 L. Kliemann =DD=E3=F1=E1=F8=E5:
> Recently, I started programming in awk, and I like it. Many of my programs
> are written in shell so far, some of them fairly complex. I like shell for
> its ability to directly interact with the filesystem via the tools cp, cd,
> rm, mv, etc. However, shell can be very slow (and awk, on the contrary, s=
eems
> very fast to me) and also, shell misses some idioms, like associative arr=
ays.
> My idea is to try to use awk wherever I used to use shell in the past. Ma=
ybe
> it is a bad idea, but I'd like to give it a try. Are there any resources =
on
> this topic? Most importantly, I need a clean way to iterate over some fil=
es
> in a directory, or over a directory tree.
>
>
> Regards,
> Lasse

Awk is a great programming language and a powerful small tool.
But its strengths shine in combination with the surrounding Unix/shell
environment.
As the shell without its small tools is (almost) merely a program
executing other programs, in the same way, awk (without a shell) is
another program processing data, like any other.
The best advice I can give, is not to think of individual programs (C
OR shell OR awk OR perl OR TCL OR python OR ...), but a combination
thereof. See, for example, if you haven't already, the "Unix
programming environment", by Kernighan & Pike.

You can hardcode filenames in awk programs, but that's hardly a viable
option (or good practice).
The common way to iterate over some files is shell metacharacters,
find -exec, or a find | xargs combination.

Vassilis

Jan Weber

2007-03-12, 9:57 pm

On Mon, 12 Mar 2007 16:13:26 +0100, L. Kliemann <stu33404@mail.uni-kiel.=
de> wrote:

> Recently, I started programming in awk, and I like it. Many of my prog=
rams
> are written in shell so far, some of them fairly complex. I like shell=
for
> its ability to directly interact with the filesystem via the tools cp,=
cd,
> rm, mv, etc. However, shell can be very slow (and awk, on the contrary=
, seems
> very fast to me) and also, shell misses some idioms, like associative =
arrays.
> My idea is to try to use awk wherever I used to use shell in the past.=
Maybe
> it is a bad idea, but I'd like to give it a try. Are there any resourc=
es on
> this topic? Most importantly, I need a clean way to iterate over some =
files
> in a directory, or over a directory tree.
>
>
> Regards,
> Lasse
>

There are no builtin means to iterate over directory trees, but a
combination of awk+find (or shell+find+awk) should work:

# process dir:
find=3D "find " startDir " -name \"" filePattern "\"";
while ( 0< (find |getline file ) ) {
# process file:
while ( (getline line <file)>0 ) {
# do something ...

}
close(file);
}
close(find);

Best regards,
Jan

Ed Morton

2007-03-13, 6:57 pm

L. Kliemann wrote:
> Recently, I started programming in awk, and I like it. Many of my programs
> are written in shell so far, some of them fairly complex. I like shell for
> its ability to directly interact with the filesystem via the tools cp, cd,
> rm, mv, etc. However, shell can be very slow (and awk, on the contrary, seems
> very fast to me) and also, shell misses some idioms, like associative arrays.
> My idea is to try to use awk wherever I used to use shell in the past. Maybe
> it is a bad idea, but I'd like to give it a try. Are there any resources on
> this topic? Most importantly, I need a clean way to iterate over some files
> in a directory, or over a directory tree.

shell is an environment for calling tools. awk is a tool for processing
text. Don't use shell for processing text and don't use awk for calling
other tools. For processing text, choose which tool to use based on what
needs to be done: For simple, one line transformations use sed. For
simple selection of patterns use grep. For simple file comparison use
diff or comm. For simple conversion of characters use tr or sed. For
most other text processing applications, use awk (or perl or ruby or
whatever scripting language/tool you prefer). For anything that's not
text processing, use whatever tool is appropriate to that job (e.g.
find, ls, ...). Use shell to sequence and provide an environment for the
calls to those tools.

Ed.

Sponsored Links