AWK - Exception to single quoting for awk patterns with single quotes?

Author

Exception to single quoting for awk patterns with single quotes?

Anand Hariharan

2007-04-26, 9:56 pm

I had to write a quick awk one-liner that had to remove multi-line
records containing '2' (a single quote followed by 2 followed by another
single quote).

I initially tried

gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /\'2\'/ { print }' InputFile

and the shell (bash) balked at me with:

gawk: cmd. line:1: fatal: cannot open file `{' for reading (No such file
or directory)

Then I tried -

gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile

(note that this time I didn't escape the single quote), and it worked
quite alright.

So, the question is, is there some kind of exception made to the single
quote when it is within a pattern for awk? Is this a
feature/idiosyncracy of bash in particular?

thank you for listening,
- Anand Hariharan

--
ROT-13 the email address to reply.

Bill Marcum

2007-04-27, 3:56 am

["Followup-To:" header set to comp.unix.shell.]
On Thu, 26 Apr 2007 20:55:36 -0500, Anand Hariharan
<znvygb.nanaq.unevunena@tznvy.pbz> wrote:
>
>
> I had to write a quick awk one-liner that had to remove multi-line
> records containing '2' (a single quote followed by 2 followed by another
> single quote).
>
> I initially tried
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> and the shell (bash) balked at me with:
>
> gawk: cmd. line:1: fatal: cannot open file `{' for reading (No such file
> or directory)
>
> Then I tried -
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> (note that this time I didn't escape the single quote), and it worked
> quite alright.
>
> So, the question is, is there some kind of exception made to the single
> quote when it is within a pattern for awk? Is this a
> feature/idiosyncracy of bash in particular?
>
You will probably find that the $1 !~ /'2'/ {print} will not print
records containing the character 2, whether or not the 2 is quoted.

--
Are we on STRIKE yet?

Barry Margolin

2007-04-27, 3:56 am

In article <_JcYh.15407$Pq5.7120@bignews6.bellsouth.net>,
Anand Hariharan <znvygb.nanaq.unevunena@tznvy.pbz> wrote:

> I had to write a quick awk one-liner that had to remove multi-line
> records containing '2' (a single quote followed by 2 followed by another
> single quote).
>
> I initially tried
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> and the shell (bash) balked at me with:
>
> gawk: cmd. line:1: fatal: cannot open file `{' for reading (No such file
> or directory)

Don't ask me why, but Bourne (and similar) shell doesn't allow escaping
single quotes inside single quotes. You have to switch to double quotes
in order to quote the single quotes:

gawk 'BEGIN { RS=""; FS="\n" } $1 !~ '"/'2'/ { print }" InputFile

>
> Then I tried -
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> (note that this time I didn't escape the single quote), and it worked
> quite alright.
>
> So, the question is, is there some kind of exception made to the single
> quote when it is within a pattern for awk? Is this a
> feature/idiosyncracy of bash in particular?

Your second command is equivalent to:

gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /2/ { print }' InputFile

The first quote in the regexp is simply closing the first quoted string,
and the second one is starting a new quoted string. E.g. try:

echo 'BEGIN /'2'/ END'

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

Ed Morton

2007-04-27, 7:56 am

Anand Hariharan wrote:
> I had to write a quick awk one-liner that had to remove multi-line
> records containing '2' (a single quote followed by 2 followed by another
> single quote).
>
> I initially tried
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> and the shell (bash) balked at me with:
>
> gawk: cmd. line:1: fatal: cannot open file `{' for reading (No such file
> or directory)
>
> Then I tried -
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> (note that this time I didn't escape the single quote), and it worked
> quite alright.
>
> So, the question is, is there some kind of exception made to the single
> quote when it is within a pattern for awk? Is this a
> feature/idiosyncracy of bash in particular?

You can't use a single quote inside an awk script when called on the
command line from any shell. If you insead put your script in a file and
invoke it using awk -f script, then you can use single quotes.

If you really want to do it on the command line then use a variable that
contains the quotes, e.g.:

awk -v pat="'2'" -v RS= -F"\n" '$1 ~ pat' InputFile

Ed.

Kenny McCormack

2007-04-27, 6:56 pm

In article <j5OdnWr8KYEoeKzbnZ2dnUVZ_t-mnZ2d@comcast.com>,
Ed Morton <morton@lsupcaemnt.com> wrote:
....
>You can't use a single quote inside an awk script when called on the
>command line from any shell.

Wrong. The hack is to switch to double quotes, include the single
quote, then switch back to single quote mode. This has the form of
inserting the following 5 character string in your command line whenver
you want a single quote to appear: '"'"'

>If you insead put your script in a file and
> invoke it using awk -f script, then you can use single quotes.

Yes. This is the best advice. The above hack is definitely not for
beginners.

>If you really want to do it on the command line then use a variable that
>contains the quotes, e.g.:
>
>awk -v pat="'2'" -v RS= -F"\n" '$1 ~ pat' InputFile

Not necessary.

Note that the above "hack" is useful in other shell contexts, e.g., when
you want to run a gawk command via su:

su -c 'gawk ...'

Or, equivalently, with: sh -c
(this comes up because I use tcsh as my interactive shell, but program
in sh[-ish], so if I need to test something from the command line, I do
the above)

Anand Hariharan

2007-04-30, 6:57 pm

On Apr 26, 8:55 pm, Anand Hariharan <znvygb.nanaq.unevun...@tznvy.pbz>
wrote:
> I had to write a quick awk one-liner that had to remove multi-line
> records containing '2' (a single quote followed by 2 followed by another
> single quote).
>
> I initially tried
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> and the shell (bash) balked at me with:
>
> gawk: cmd. line:1: fatal: cannot open file `{' for reading (No such file
> or directory)
>
> Then I tried -
>
> gawk 'BEGIN { RS=""; FS="\n" } $1 !~ /'2'/ { print }' InputFile
>
> (note that this time I didn't escape the single quote), and it worked
> quite alright.
>
> So, the question is, is there some kind of exception made to the single
> quote when it is within a pattern for awk? Is this a
> feature/idiosyncracy of bash in particular?
>
> thank you for listening,
> - Anand Hariharan
>

Bill and Barry -

Thank you for responding and hinting/explaining what the real problem
was.

sincerely,
- Anand

Ed Morton

2007-04-30, 6:57 pm

Anand Hariharan wrote:
> On Apr 26, 8:55 pm, Anand Hariharan <znvygb.nanaq.unevun...@tznvy.pbz>
> wrote:
>
>
>
> Bill and Barry -
>
> Thank you for responding and hinting/explaining what the real problem
> was.

I assume that means you didn't like the answer I provided. If you do it
other than the ways I described, you're setting a trend for your
programs that will get you into trouble down the road as the other
solutions posted cause you to jump back and forth between awk and shell
interpretations of your script which can cause all sorts of seemingly
bizarre behavior. For example:

$ echo "a b" | awk '{print $1}'
a
$ echo "a b" | awk "{print $1}"
a b

or:

$ shellvar="hello world"
$ awk -v awkvar="$shellvar" 'BEGIN{ print awkvar; exit }'
hello world
$ awk 'BEGIN{ print "'"$shellvar"'"; exit }'
hello world
$ shellvar="hello
world"
$ awk -v awkvar="$shellvar" 'BEGIN{ print awkvar; exit }'
hello
world
$ awk 'BEGIN{ print "'"$shellvar"'"; exit }'
awk: BEGIN{ print "hello
awk: ^ unterminated string

Regards,

Ed.

Sponsored Links