AWK - ending records reading when a condition is met

Author

ending records reading when a condition is met

z.entropic

2007-06-07, 9:57 pm

I have an awk script which I would like to [b]stop[\b] reading records
from a given file when a condition is met. Condition-less script is
as follows:

for i in *.out; do awk 'FNR == 1 { print FILENAME } FNR > 33 { printf
"%7.4f\n", $1 }' $i > $i.Q; done

but its modification

for i in *.out; do awk 'FNR == 1 { print FILENAME } FNR > 33 [b]&& $2
> .5[\b] { printf "%7.4f\n", $1 }' $i > $i.Q; done

only [b]eliminates[\b] records meeting the $2>0.5 condition.

It must be an easy problem, but... please help.

z.entropic

Vassilis

2007-06-07, 9:57 pm

/ z.entropic :
> I have an awk script which I would like to [b]stop[\b] reading records
> from a given file when a condition is met. Condition-less script is
> as follows:
>
> for i in *.out; do awk 'FNR == 1 { print FILENAME } FNR > 33 { printf
> "%7.4f\n", $1 }' $i > $i.Q; done
>
> but its modification
>
> for i in *.out; do awk 'FNR == 1 { print FILENAME } FNR > 33 [b]&& $2
>
> only [b]eliminates[\b] records meeting the $2>0.5 condition.
>
> It must be an easy problem, but... please help.
>
> z.entropic

Just put "nextfile" after your printf. nextfile stops processing
current file and skips to the next one.

Vasssilis

Vassilis

2007-06-07, 9:57 pm

Patrick TJ McPhee

2007-06-08, 6:57 pm

In article <1181252243.512994.71520@q66g2000hsg.googlegroups.com>,
Vassilis <F.H.Novalis@gmail.com> wrote:

% Just put "nextfile" after your printf. nextfile stops processing
% current file and skips to the next one.

Keeping in mind that nextfile is a gawk-specific extension.

If you're not using gawk, then if you have only one file, you can get the
same effect with exit. If you want to skip to a second file, you
should probably be using gawk.
--

Patrick TJ McPhee
North York Canada
ptjm@interlog.com

Kenny McCormack

2007-06-08, 6:57 pm

In article <136hjdth7tc8f39@corp.supernews.com>,
Patrick TJ McPhee <ptjm@interlog.com> wrote:
>In article <1181252243.512994.71520@q66g2000hsg.googlegroups.com>,
>Vassilis <F.H.Novalis@gmail.com> wrote:
>
>% Just put "nextfile" after your printf. nextfile stops processing
>% current file and skips to the next one.
>
>Keeping in mind that nextfile is a gawk-specific extension.
>
>If you're not using gawk, then if you have only one file, you can get the
>same effect with exit. If you want to skip to a second file, you
>should probably be using gawk.

s/If you want to skip to a second file,/But/
s/probably/definitely/p

z.entropic

2007-06-08, 6:57 pm

Kenny McCormack

2007-06-08, 6:57 pm

In article <1181318468.549001.238580@q69g2000hsb.googlegroups.com>,
z.entropic <subPlanck@excite.com> wrote:
>On Jun 7, 11:32 pm, p...@interlog.com (Patrick TJ McPhee) wrote:
>
>I'm using MKS ToolKit's awk.

This implies DOS/Windows - in which case, you can certainly and easily
switch to gawk. You'll be glad you did.

z.entropic

2007-06-08, 6:57 pm

On Jun 7, 11:32 pm, p...@interlog.com (Patrick TJ McPhee) wrote:
> In article <1181252243.512994.71...@q66g2000hsg.googlegroups.com>,
>
> Vassilis <F.H.Nova...@gmail.com> wrote:
>
> % Just put "nextfile" after your printf. nextfile stops processing
> % current file and skips to the next one.
>
> Keeping in mind that nextfile is a gawk-specific extension.
>
> If you're not using gawk, then if you have only one file, you can get the
> same effect with exit. If you want to skip to a second file, you
> should probably be using gawk.
> --
>
> Patrick TJ McPhee
> North York Canada
> p...@interlog.com

Thanks; poking around, I found on
http://uw713doc.sco.com/cgi-bin/info2html?(gawk.info)Nextfile%2520Function&lang=en
an awk function that simulates the action of 'nextifile':

# nextfile --- skip remaining records in current file
# correctly handle successive occurrences of the same file
# Arnold Robbins, arnold@gnu.org, Public Domain
# May, 1993

# this should be read in before the "main" awk program

function nextfile() { _abandon_ = FILENAME; next }

_abandon_ == FILENAME {
if (FNR == 1)
_abandon_ = ""
else
next
}

but now I'm stuck on how to implement it in my one-liner... I
understand that the function should be defined right at the start of
the awk program, but have no idea how to call it further down the line
when both conditions are met.

for i in *.out; do awk 'FNR == 1 { print FILENAME } (FNR > 33) && ($2
< 0.5) ??? { printf "%7.4f\n", $1 }' $i > $i.Q; done

Do I use a logic statement (x?:)? In what form? You see, I'm not a
programmer, just a two-bit user...

z.entropic

Ed Morton

2007-06-08, 6:57 pm

z.entropic wrote:
> On Jun 7, 11:32 pm, p...@interlog.com (Patrick TJ McPhee) wrote:
>
>
>
> Thanks; poking around, I found on
> http://uw713doc.sco.com/cgi-bin/info2html?(gawk.info)Nextfile%2520Function&lang=en
> an awk function that simulates the action of 'nextifile':
>
> # nextfile --- skip remaining records in current file
> # correctly handle successive occurrences of the same file
> # Arnold Robbins, arnold@gnu.org, Public Domain
> # May, 1993
>
> # this should be read in before the "main" awk program
>
> function nextfile() { _abandon_ = FILENAME; next }
>
> _abandon_ == FILENAME {
> if (FNR == 1)
> _abandon_ = ""
> else
> next
> }
>
> but now I'm stuck on how to implement it in my one-liner... I
> understand that the function should be defined right at the start of
> the awk program, but have no idea how to call it further down the line
> when both conditions are met.
>
> for i in *.out; do awk 'FNR == 1 { print FILENAME } (FNR > 33) && ($2
> < 0.5) ??? { printf "%7.4f\n", $1 }' $i > $i.Q; done
>
> Do I use a logic statement (x?:)? In what form? You see, I'm not a
> programmer, just a two-bit user...
>
> z.entropic
>
>

Hang on, you have this script (cleaned up to be readable and fix the
shell variable quoting):

for i in *.out; do
awk '
FNR == 1 {print FILENAME}
(FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1}
' "$i" > "$i.Q"
done

and you just want to avoid processing records in the current file after
the second condition is ture, and you don't have "netxfile" in your awk,
right? Then all you do is add a "found" flag:

for i in *.out; do
awk '
FNR == 1 {print FILENAME; found=0}
found { next }
(FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1; found=1}
' "$i" > "$i.Q"
done

or test the current FILENAME against the name of the file in which the
condition was found to be true:

for i in *.out; do
awk '
FNR == 1 {print FILENAME}
(FNR > 33) && ($2 < 0.5) && (fname != FILENAME) {
printf "%7.4f\n", $1
fname=FILENAME
}
' "$i" > "$i.Q"
done

Regards,

Ed.

z.entropic

2007-06-08, 6:57 pm

On Jun 8, 12:45 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> z.entropic wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hang on, you have this script (cleaned up to be readable and fix the
> shell variable quoting):
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME}
> (FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1}
> ' "$i" > "$i.Q"
> done
>
> and you just want to avoid processing records in the current file after
> the second condition is ture, and you don't have "netxfile" in your awk,
> right? Then all you do is add a "found" flag:
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME; found=0}
> found { next }
> (FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1; found=1}
> ' "$i" > "$i.Q"
> done
>
> or test the current FILENAME against the name of the file in which the
> condition was found to be true:
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME}
> (FNR > 33) && ($2 < 0.5) && (fname != FILENAME) {
> printf "%7.4f\n", $1
> fname=FILENAME
> }
> ' "$i" > "$i.Q"
> done
>
> Regards,
>
> Ed.- Hide quoted text -
>
> - Show quoted text -

Thanks, Ed, it's your version #1. I want to process multiple files,
and when both conditions are met for any of them, I want the
processing to quite the current file and go to the next.

I've tried your version, but somehow on my system (MKS ToolKit in the
DOS box under XP) I can't get it to work... it dosn't stop if $2 < 0.5

z.entropic

z.entropic

2007-06-08, 6:57 pm

On Jun 8, 12:45 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> z.entropic wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hang on, you have this script (cleaned up to be readable and fix the
> shell variable quoting):
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME}
> (FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1}
> ' "$i" > "$i.Q"
> done
>
> and you just want to avoid processing records in the current file after
> the second condition is ture, and you don't have "netxfile" in your awk,
> right? Then all you do is add a "found" flag:
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME; found=0}
> found { next }
> (FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1; found=1}
> ' "$i" > "$i.Q"
> done
>
> or test the current FILENAME against the name of the file in which the
> condition was found to be true:
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME}
> (FNR > 33) && ($2 < 0.5) && (fname != FILENAME) {
> printf "%7.4f\n", $1
> fname=FILENAME
> }
> ' "$i" > "$i.Q"
> done
>
> Regards,
>
> Ed.- Hide quoted text -
>
> - Show quoted text -

Ed,

The first script (which is my case) prints only the first value for
FNR > 33 && $2 < 0.5, but I need all the values until this condition
is met.

z.entropic

Vassilis

2007-06-08, 6:57 pm

/ z.entropic :
>
> The first script (which is my case) prints only the first value for
> FNR > 33 && $2 < 0.5, but I need all the values until this condition
> is met.
>
> z.entropic

After found, add a next, and then a print command:

for i in *.out; do
awk '
FNR == 1 {print FILENAME; found=0}
found { next }
(FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1; found=1;
next} # here
{ print } # and here
' "$i" > "$i.Q"
done

z.entropic

2007-06-08, 6:57 pm

On Jun 8, 1:52 pm, Vassilis <F.H.Nova...@gmail.com> wrote:
> / z.entropic :
>
>
>
>
>
> After found, add a next, and then a print command:
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME; found=0}
> found { next }
> (FNR > 33) && ($2 < 0.5) {printf "%7.4f\n", $1; found=1;
> next} # here
> { print } # and here
> ' "$i" > "$i.Q"
> done

Vassilis,

Almost there as the printing ends when $2 < 0.5--but the output
contains records 1-..., instead of only those for FNR > 33...
Somehow, the first condition in this double-condition statement is
disregarded...

Thanks for your patience.

z.entropic

Vassilis

2007-06-08, 6:57 pm

/ z.entropic :
> On Jun 8, 1:52 pm, Vassilis <F.H.Nova...@gmail.com> wrote:
>
> Vassilis,
>
> Almost there as the printing ends when $2 < 0.5--but the output
> contains records 1-..., instead of only those for FNR > 33...
> Somehow, the first condition in this double-condition statement is
> disregarded...
>
> Thanks for your patience.
>
> z.entropic

I see.
The following script should do just right:

for i in *.out; do
awk '
FNR == 1 { print FILENAME }
FNR <= 33 { next }
$2 >= 0.5 { next }
{ printf "%7.4f\n", $1 }
' "$i" > "$i.Q"
done

This way, we're inverting conditions, ignoring any other lines we
don't care about.

Ed Morton

2007-06-08, 6:57 pm

z.entropic wrote:
> On Jun 8, 1:52 pm, Vassilis <F.H.Nova...@gmail.com> wrote:
>
>
>
> Vassilis,
>
> Almost there as the printing ends when $2 < 0.5--but the output
> contains records 1-...,

That's what you said you wanted: "I need all the values until this
condition is met".

instead of only those for FNR > 33...
> Somehow, the first condition in this double-condition statement is
> disregarded...

So, are you saying that what you really want is the formatted version of
$1 for those records from record number 33 up to and including the first
record where $2 is less than 0.5? If so, this should do it:

for i in *.out; do
awk '
FNR == 1 {print FILENAME}
(FNR <= 33) || (fname == FILENAME) { next }
{ printf "%7.4f\n",$1 }
$2 < 0.5 {fname = FILENAME}
' "$i" > "$i.Q"
done

or use a "found" flag as demonstrated earlier if you prefer.

Regards,

Ed.

z.entropic

2007-06-08, 6:57 pm

On Jun 8, 3:37 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> z.entropic wrote:
>
>
>
>
>
>
>
>
> That's what you said you wanted: "I need all the values until this
> condition is met".
>
> instead of only those for FNR > 33...
>
>
> So, are you saying that what you really want is the formatted version of
> $1 for those records from record number 33 up to and including the first
> record where $2 is less than 0.5? If so, this should do it:
>
> for i in *.out; do
> awk '
> FNR == 1 {print FILENAME}
> (FNR <= 33) || (fname == FILENAME) { next }
> { printf "%7.4f\n",$1 }
> $2 < 0.5 {fname = FILENAME}
> ' "$i" > "$i.Q"
> done
>
> or use a "found" flag as demonstrated earlier if you prefer.
>
> Regards,
>
> Ed.- Hide quoted text -
>
> - Show quoted text -

Sorry for having been 'a bit' imprecise; your latest version is what I
really wanted, all the records from #33 until the second condition is
met.

Sincere thanks to you and others for great and immediate help,

z.entropic

Sponsored Links