AWK - join columns into single column

Author

join columns into single column

enda_ridge@yahoo.co.uk

2006-01-30, 7:55 am

Hi,
I am new to using awk. I would be very grateful for some help with the
following problem. I have access to awk and gawk using cygwin.

I have text files of the following format:

#******************
# COmments
# Headings for columns
#**************************
Col1_val Col2_val Col3_val Col4_val Col5_val

I would like to tell awk to do the following:
Skip commented lines (begin with #, only occur at head of file)
For the whole file, combine all data in columns X to Y into a single
column of data in a new file.
X to Y will always include all columns between X and Y without gaps,
e.g. column 4,5,6 and never 4,6
It does not matter whether the data is combined with columns end-to-end
or with columns line by line.

Thanks in advance.
Enda

enda_ridge@yahoo.co.uk

2006-01-30, 7:55 am

Is it possible to repeat this process for the last X lines of the file,
e.g. the last 500 lines?

Thanks,
Enda

Ed Morton

2006-01-30, 6:56 pm

enda_ridge@yahoo.co.uk wrote:
> Hi,
> I am new to using awk. I would be very grateful for some help with the
> following problem. I have access to awk and gawk using cygwin.
>
> I have text files of the following format:
>
> #******************
> # COmments
> # Headings for columns
> #**************************
> Col1_val Col2_val Col3_val Col4_val Col5_val
>
>
> I would like to tell awk to do the following:
> Skip commented lines (begin with #, only occur at head of file)

awk '/^#/{next}1' file

> For the whole file, combine all data in columns X to Y into a single
> column of data in a new file.

If by "combine" you mean "string concatenate":

awk '{print $X $Y}' file1 > file2

If you mean "sum":

awk '{print $X + $Y}' file1 > file2

> X to Y will always include all columns between X and Y without gaps,
> e.g. column 4,5,6 and never 4,6

Let's assume you mean "sum". Try this:

awk -v X=4 -v Y=6 '/^#/{next}{sum=0; for (i=X; i<=Y; i++) sum+=$i; print
sum}' file1 > file2

> It does not matter whether the data is combined with columns end-to-end
> or with columns line by line.

I have no idea what that means.

Ed.

Ed Morton

2006-01-30, 6:56 pm

enda_ridge@yahoo.co.uk wrote:

> Is it possible to repeat this process for the last X lines of the file,
> e.g. the last 500 lines?

Please read http://cfaj.freeshell.org/google before posting again as
you're falling foul of a flaw in google.

Now, to your question, I'm not sure what you mean by "repeat", whether
you mean add the last 500 lines again to the total, or just have a
script that only works on the last 500 lines. Assuming the latter, put
this in a file called "whatever.awk":

BEGIN{ ARGV[ARGC++] = ARGV[1] }
NR==FNR { start = FNR - 500; next }
/^#/ { next }
FNR >= start {sum=0; for (i=X; i<=Y; i++) sum+=$i; print sum}

and run it as:

awk -v X=4 -v Y=6 -f whatever.awk file1 > file2

Regards,

Ed.

enda_ridge@yahoo.co.uk

2006-01-30, 6:56 pm

Ed,
Sorry if the original posting wasn't clear. I am trying to do the
following:

Sample input file:

# Commented line
# Commented line
1000 2000 3000
1001 2001 3001
1002 2002 3002
1003 2003 3003

Desired input command:
join columns 2 to 3

desired output file
2000
2001
2002
2003
3000
3001
3002
3003

Second desired input command:
join collumns 2 to 3, last 2 lines

desired output file
2002
2003
3002
3003

hope this clears it up.

ENda

enda_ridge@yahoo.co.uk

2006-01-30, 6:56 pm

Ed,

The following modifications achieve what I need:

BEGIN{ ARGV[ARGC++] = ARGV[1] }
NR==FNR { start = FNR - 2; next }
/^#/ { next }
FNR >= start {for (i=X; i<=Y; i++)print $i; }

How would I pass in the number of lines from the end? In the above it
is hard coded to 2.

How would I adjust the script to do all lines?

Thanks,
Enda

Ed Morton

2006-01-30, 6:56 pm

enda_ridge@yahoo.co.uk wrote:
> Ed,
>
> The following modifications achieve what I need:

As I mentioned earlier, PLEASE read http://cfaj.freeshell.org/google
before posting again as you're falling foul of a flaw in google.

> BEGIN{ ARGV[ARGC++] = ARGV[1] }
> NR==FNR { start = FNR - 2; next }
> /^#/ { next }
> FNR >= start {for (i=X; i<=Y; i++)print $i; }
>
> How would I pass in the number of lines from the end? In the above it
> is hard coded to 2.

The same way you pass in values for X and Y.

> How would I adjust the script to do all lines?

Don't test for FNR >= start or set your "- whatever" value to zero.

Ed.

Sponsored Links