AWK - AWK - How to extarct fileds and put them below each other ...

Author

AWK - How to extarct fileds and put them below each other ...

vito

2004-08-20, 10:06 pm

Hi,

I will try to explain the issue I have with awk utility (Linux, ksh):

1. I have a input file in format:

XXX YYY COL_1 COL_2 COL_3 COL_4 COL_5 .. COL_X

2. I want to get output file that look like this (note - first two fields (XXX and YYY) are excluded):

COL_1
COL_2
COL_3
COL_4
..
COL_X

(so first two columns are excluded and others were rewriten in 'vertical' fashion - one record below the other)

I managed to do this (please see my solution) but it is a little akward but nonetheless it works. Is there any more efficient way to do the task?

awk '{ print }' input_file | grep XXX | grep YYY | awk '{ print $3 "\n" $4 "\n" $5 "\n" $6 "\n" $7 "\n" $8 "\n" $9 "\n" $10 }' > temp_output

while read task;
do
BACKUP_TASK=$task
print "BACKUP_TASK is $BACKUP_TASK"
${BACKUP_TASK} #this is read by shell script and specific
#shell function (COL_1, COL_2, ...) is invoked
done < temp_output

3. Finaly, is there a way in 'awk' to print col_1, all fileds between col_3 and col_10 and all fileds between col_20 and col_30 (something like {print $1, $3-10, $20-30}) - in short, how do I specify range of fields to be read without explicitly specifying each field?

Thanks

Chris F.A. Johnson

2004-08-21, 8:56 am

On 2004-08-21, vito wrote:
>
> Hi,
>
> I will try to explain the issue I have with awk utility (Linux, ksh):
>
> 1. I have a input file in format:
>
> XXX YYY COL_1 COL_2 COL_3 COL_4 COL_5 .. COL_X
>
> 2. I want to get output file that look like this (note - first two
> fields (XXX and YYY) are excluded):
>
> COL_1
> COL_2
> COL_3
> COL_4
> .
> COL_X
>
> (so first two columns are excluded and others were rewriten in
> 'vertical' fashion - one record below the other)
>
> I managed to do this (please see my solution) but it is a little akward
> but nonetheless it works. Is there any more efficient way to do the
> task?
>
> awk '{ print }' input_file | grep XXX | grep YYY | awk '{ print $3 "\n"
> $4 "\n" $5 "\n" $6 "\n" $7 "\n" $8 "\n" $9 "\n" $10 }' > temp_output
>
> while read task;
> do
> BACKUP_TASK=$task
> print "BACKUP_TASK is $BACKUP_TASK"
> ${BACKUP_TASK} #this is read by shell script and specific
> #shell function (COL_1, COL_2, ...) is invoked
> done < temp_output

You don't need awk to do that:

read a a a b < FILE
printf "%s\n" $a $b

But if you really want to use awk:

awk '{
n = 2
while ( n++ < NF ) print $n
}'

> 3. Finaly, is there a way in 'awk' to print col_1, all fileds between
> col_3 and col_10 and all fileds between col_20 and col_30 (something
> like {print $1, $3-10, $20-30}) - in short, how do I specify range of
> fields to be read without explicitly specifying each field?

awk '{
print $1
n = 3
while ( n++ <= 10 ) printf "%s " $n
print ""
n = 20
while ( n++ <= 30 ) print $n
}' FILE

--
Chris F.A. Johnson http://cfaj.freeshell.org/shell
========================================
===========================
My code (if any) in this post is copyright 2004, Chris F.A. Johnson
and may be copied under the terms of the GNU General Public License

Bob Harris

2004-08-21, 3:58 pm

In article <vito.1bbz5r@mail.codecomments.com>,
vito <vito.1bbz5r@mail.codecomments.com> wrote:

> Hi,
>
> I will try to explain the issue I have with awk utility (Linux, ksh):
>
> 1. I have a input file in format:
>
> XXX YYY COL_1 COL_2 COL_3 COL_4 COL_5 .. COL_X
>
> 2. I want to get output file that look like this (note - first two
> fields (XXX and YYY) are excluded):
>
> COL_1
> COL_2
> COL_3
> COL_4
> .
> COL_X
>
> (so first two columns are excluded and others were rewriten in
> 'vertical' fashion - one record below the other)
>
> I managed to do this (please see my solution) but it is a little akward
> but nonetheless it works. Is there any more efficient way to do the
> task?
>
> awk '{ print }' input_file | grep XXX | grep YYY | awk '{ print $3 "\n"
> $4 "\n" $5 "\n" $6 "\n" $7 "\n" $8 "\n" $9 "\n" $10 }' > temp_output
>
> while read task;
> do
> BACKUP_TASK=$task
> print "BACKUP_TASK is $BACKUP_TASK"
> ${BACKUP_TASK} #this is read by shell script and specific
> #shell function (COL_1, COL_2, ...) is invoked
> done < temp_output
>
> 3. Finaly, is there a way in 'awk' to print col_1, all fileds between
> col_3 and col_10 and all fileds between col_20 and col_30 (something
> like {print $1, $3-10, $20-30}) - in short, how do I specify range of
> fields to be read without explicitly specifying each field?
>
> Thanks
>
>
>
> --
> vito
> ------------------------------------------------------------------------
> Posted via http://www.codecomments.com
> ------------------------------------------------------------------------
>

awk '{for(j=3;j<=NF;j++)print $j}' input_file >output_file

Bob Harris

William Park

2004-08-21, 3:58 pm

vito <vito.1bbz5r@mail.codecomments.com> wrote:
>
> Hi,
>
> I will try to explain the issue I have with awk utility (Linux, ksh):
>
> 1. I have a input file in format:
>
> XXX YYY COL_1 COL_2 COL_3 COL_4 COL_5 .. COL_X
>
> 2. I want to get output file that look like this (note - first two
> fields (XXX and YYY) are excluded):
>
> COL_1
> COL_2
> COL_3
> COL_4
> .
> COL_X
>
> (so first two columns are excluded and others were rewriten in
> 'vertical' fashion - one record below the other)

In shell,
read a b c <<< "$line"
printf "%s\n" $c
In awk,
... { $1=""; $2=""; print $0 }

> 3. Finaly, is there a way in 'awk' to print col_1, all fileds between
> col_3 and col_10 and all fileds between col_20 and col_30 (something
> like {print $1, $3-10, $20-30}) - in short, how do I specify range of
> fields to be read without explicitly specifying each field?

No. If separator is a single space, then you can use
cut -d ' ' -f 1,3-10,20-30
But, inside awk, you need to loop manually.
--
William Park <opengeometry@yahoo.ca>
Open Geometry Consulting, Toronto, Canada

vito

2004-08-21, 7:29 pm

Gentlemen,

Thank you for your help and elegant solutions ...

Vito

Ed Morton

2004-08-22, 3:55 am

vito wrote:
> Gentlemen,
>
> Thank you for your help and elegant solutions ...

You got the right answers for your question, but your original post also
showed this:

awk '{ print }' input_file | grep XXX | grep YYY | awk '{ print $3
"\n" $4 "\n" $5 "\n" $6 "\n" $7 "\n" $8 "\n" $9 "\n" $10 }'

and I may have missed it, but I don't think anyone pointed out that you
don't need grep AND awk since awk can search for patterns, or that your
first "awk" statement is equivalent to "cat" or that instead of cat-ing
a file to grep or awk, you can just pass the file as the argument to the
command. So instead of:

awk '{ print }' input_file | grep XXX | grep YYY | awk '{stuff}'

you should just do:

awk '/XXX/ && /YYY/ {stuff}' input_file

and save yourself a slew of processes.

Regards,

Ed.

Brian Inglis

2004-08-26, 3:58 pm

On Fri, 20 Aug 2004 22:06:22 -0500 in comp.lang.awk, vito
<vito.1bbz5r@mail.codecomments.com> wrote:

>
>Hi,
>
>I will try to explain the issue I have with awk utility (Linux, ksh):
>
>1. I have a input file in format:
>
>XXX YYY COL_1 COL_2 COL_3 COL_4 COL_5 .. COL_X
>
>2. I want to get output file that look like this (note - first two
>fields (XXX and YYY) are excluded):
>
>COL_1
>COL_2
>COL_3
>COL_4
>.
>COL_X
>
>(so first two columns are excluded and others were rewriten in
>'vertical' fashion - one record below the other)
>
>I managed to do this (please see my solution) but it is a little akward
>but nonetheless it works. Is there any more efficient way to do the
>task?
>
>awk '{ print }' input_file | grep XXX | grep YYY | awk '{ print $3 "\n"
>$4 "\n" $5 "\n" $6 "\n" $7 "\n" $8 "\n" $9 "\n" $10 }' > temp_output

awk 'BEGIN { OFS = "\n" }
/XXX/ && /YYY/ { $1 = ""; $2 = ""; print $0 }' input_file >temp_output

--
Thanks. Take care, Brian Inglis Calgary, Alberta, Canada

Brian.Inglis@CSi.com (Brian[dot]Inglis{at}SystematicSW[dot]a
b[dot]ca)
fake address use address above to reply

Brian Inglis

2004-09-02, 8:55 am

Sponsored Links