PERL Beginners - regex one liner

Author

regex one liner

Stu Meacham

2006-03-20, 9:55 pm

I would like to modify a file 'in place' at the command line with regexes. =
The file changes daily and is messy. Can one negate a regex itself as opp=
osed to a class of regular expressions? If I could remove everything but t=
hat selected by m/\d{2}\t\d{2}\t\d{2}/g life would be better as I know it. =
Thanx

--=20
________________________________________
_______

Search for businesses by name, location, or phone number. -Lycos Yellow Pa=
ges

http://r.lycos.com/r/yp_emailfooter...com/default.as=
p?SRC=3Dlycos10

Timothy Johnson

2006-03-20, 9:55 pm

What have you tried? Have you seen the -i option?

-----Original Message-----
From: stu meacham [mailto:stu21774@lycos.com]=20
Sent: Monday, March 20, 2006 2:33 PM
To: beginners@perl.org
Subject: regex one liner

I would like to modify a file 'in place' at the command line with
regexes. The file changes daily and is messy. Can one negate a regex
itself as opposed to a class of regular expressions? If I could remove
everything but that selected by m/\d{2}\t\d{2}\t\d{2}/g life would be
better as I know it. Thanx

--=20
________________________________________
_______

Search for businesses by name, location, or phone number. -Lycos Yellow
Pages

http://r.lycos.com/r/yp_emailfooter...cos.com/default
..asp?SRC=3Dlycos10

--=20
To unsubscribe, e-mail: beginners-unsubscribe@perl.org
For additional commands, e-mail: beginners-help@perl.org
<http://learn.perl.org/> <http://learn.perl.org/first-response>

DJ Stunks

2006-03-20, 9:55 pm

Stu Meacham wrote:
> I would like to modify a file 'in place' at the command line with regexes. The file changes
> daily and is messy. Can one negate a regex itself as opposed to a class of regular expressions?
> If I could remove everything but that selected by m/\d{2}\t\d{2}\t\d{2}/g life would be better
> as I know it. Thanx

why does it have to be in-place? just grab that info out of your
"messy" file and pipe it into another:

perl -lne'print for /\d{2}\t\d{2}\t\d{2}/g' messy.file > tidy.file

if you really want, rm your messy.file afterwards.

-jp

Stu Meacham

2006-03-21, 7:55 am

> perl -i -p -e 's/^(\d{2}\t\d{2}\t\d{2})/g' This was the 1st thing that I=
tried; it doesn't work. It was initially easy but different things kept a=
ppearing that forced me to use > 1=20
statements on the command line. Negating what I want seems like it ought t=
o be simple.
>=20
>=20
> What have you tried? Have you seen the -i option?
>=20
> -----Original Message-----
>=20
>=20
> I would like to modify a file 'in place' at the command line with
> regexes. The file changes daily and is messy. Can one negate a regex
> itself as opposed to a class of regular expressions? If I could remove
> everything but that selected by m/\d{2}\t\d{2}\t\d{2}/g life would be
> better as I know it. Thanx
>=20
> --
> ________________________________________
_______
>=20
> Search for businesses by name, location, or phone number. -Lycos Yellow
> Pages
>=20
> http://r.lycos.com/r/yp_emailfooter...cos.com/default
> .asp?SRC=3Dlycos10
>=20
>=20
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> <http://learn.perl.org/> <http://learn.perl.org/first-response>

>

--=20
________________________________________
_______

Search for businesses by name, location, or phone number. -Lycos Yellow Pa=
ges

http://r.lycos.com/r/yp_emailfooter...com/default.as=
p?SRC=3Dlycos10

Stu Meacham

2006-03-21, 9:58 pm

I tried one final time with non-capturing parentheses i.e. (?: to no avai=
l.=20=20
This works just fine however:

perl -i -p -e '@matches =3D m/\d{2}\t\d{2}\t\d{2}/g; s/.*//g; print"@matche=
s\n"'=20

Retrieve, delete what's left, and rewrite what's to be kept. It should now
work everytime all the time. Crude but effective. My (result)/(time spent=
)=20
ratio would have been lower without your help. I stumbled over the ; insi=
de
of the ''s for one.
Thanks very much.

Stu
>=20
> There are a couple of things here:
>=20
> 1) s/// requires a string to replace the match with. you haven't
> supplied one. This should produce a syntax error.
>=20
> 2) You've anchored your patttern at the beginning of the string. This
> makes /g a no-op; you're pattern can't possibly match more than once,
> because '^' can only appear once.
>=20
> 3) s/// replaces whatever is matched on the left side with whatever
> you put on the right side, and leaves the rest of the string
> untouched. You need to specify what you don't want, as well.
>=20
> If you know that your will always have a single occurance at the
> beginning of the line, use something like:
>=20
> s/^(\d\d\t\d\d\t\d\d).*/$1/
>=20
> I think we're about at the end of how much we can help based on what
> you've given us. Show us a couple of the complete command lines you've
> tried (so we can see what your ultimate goal is) and some sample data.
>=20
> -- jay

--=20
________________________________________
_______

Search for businesses by name, location, or phone number. -Lycos Yellow Pa=
ges

http://r.lycos.com/r/yp_emailfooter...com/default.as=
p?SRC=3Dlycos10

Timothy Johnson

2006-03-21, 9:58 pm

We run into one of these "How do I do this in a one-liner?" questions
pretty frequently, and I for one have to ask, what exactly makes the
one-liner so compelling, especially when you are using it for something
that will be run repeatedly?

IMHO it would have been much more practical to just create a three line
Perl script and run it from the command-line. It makes it easier to
read, doesn't discourage commenting, and avoids the contortions
sometimes needed to cram it all into one line.

-----Original Message-----
From: stu meacham [mailto:stu21774@lycos.com]=20
Sent: Tuesday, March 21, 2006 8:25 AM
To: Jay Savage; beginners perl
Subject: Re: regex one liner

I tried one final time with non-capturing parentheses i.e. (?: to no
avail. =20
This works just fine however:

perl -i -p -e '@matches =3D m/\d{2}\t\d{2}\t\d{2}/g; s/.*//g;
print"@matches\n"'=20

Retrieve, delete what's left, and rewrite what's to be kept. It should
now
work everytime all the time. Crude but effective. My (result)/(time
spent)=20
ratio would have been lower without your help. I stumbled over the ;
inside
of the ''s for one.
Thanks very much.

<snip>

Mr. Shawn H. Corey

2006-03-21, 9:58 pm

Timothy Johnson wrote:
> We run into one of these "How do I do this in a one-liner?" questions
> pretty frequently, and I for one have to ask, what exactly makes the
> one-liner so compelling, especially when you are using it for something
> that will be run repeatedly?

Because you can use them in aliases:

alias pcalc='perl -ple '''BEGIN{use Data::Dumper}$_=eval''''

See `man alias` for details.

--

Just my 0.00000002 million dollars worth,
--- Shawn

"For the things we have to learn before we can do them,
we learn by doing them."
Aristotle

"The man who sets out to carry a cat by its tail learns something that
will always be useful and which will never grow dim or doubtful."
Mark Twain

"Believe in the Divine, but paddle away from the rocks."
Hindu Proverb

* Perl tutorials at http://perlmonks.org/?node=Tutorials
* A searchable perldoc is at http://perldoc.perl.org/

Timothy Johnson

2006-03-21, 9:58 pm

>
>-----Original Message-----
>From: Mr. Shawn H. Corey [mailto:shawnhcorey@magma.ca]
>Sent: Tuesday, March 21, 2006 1:36 PM
>To: beginners perl
>Subject: Re: regex one liner
>
>Timothy Johnson wrote:
something[color=darkred]
>
>Because you can use them in aliases:
>
> alias pcalc=3D'perl -ple '''BEGIN{use Data::Dumper}$_=3Deval''''
>
>See `man alias` for details.

And you can't do this?

alias pcalc=3D'perl ~/pcalc.pl'

And as for the issue of slightly varying regexes as arguments to a
script (different email), it may seem easier to you, but not necessarily
to the next guy that comes along. Obviously you aren't required to take
other people into account when designing your scripts, but I always try
to. By simply moving the part that always changes to an argument you
give yourself the same flexibility and ability to use the history while
at the same time leaving a trail in case you get hit by a truck and
someone else needs to step into your shoes temporarily. And you never
know; the person who needs to figure out what you did may end up being
you.

Obviously I'm not saying that it's wrong to do it, but IMHO it's worth
the small bit of extra time up front, and Perl gives you all the tools
needed to meet your requirements without having to cram it all into one
line.

That being said, if I have the time and anyone has a question, I'll be
happy to try to help. I don't want to discourage anyone from expanding
their knowledge.

Uri Guttman

2006-03-21, 9:58 pm

>>>>> "SHC" == Shawn H Corey <shawnhcorey@magma.ca> writes:

SHC> alias pcalc='perl -ple '''BEGIN{use Data::Dumper}$_=eval''''

you might as well clean that up. the module load can be done with -M and
the \ and quoting is annoying.

alias pcalc='perl -MData::Dumper -ple "\$_=eval"'

and you could swap the " and ' but you would still need to escape the
$_.

ly shells don't offer alternate delimiters like perl does.

uri

--
Uri Guttman ------ uri@stemsystems.com -------- http://www.stemsystems.com
--Perl Consulting, Stem Development, Systems Architecture, Design and Coding-
Search or Offer Perl Jobs ---------------------------- http://jobs.perl.org

Mr. Shawn H. Corey

2006-03-21, 9:58 pm

Timothy Johnson wrote:
> And you can't do this?
>
> alias pcalc='perl ~/pcalc.pl'

No. With alias, I can create an alias file that works with sh (and ksh,
bash) and csh (and tcsh). This would be dot'ed in your .profile as:

. ~/.alias

or source'd in your .cshrc as:

source ~/.alias

If you want the same as a script:

mv ~/pcalc.pl ~/bin/pcalc

BTW, out here in the real world (that would be UNIX), *.pl stands for
Perl Library file, not a script.

--

Just my 0.00000002 million dollars worth,
--- Shawn

"For the things we have to learn before we can do them,
we learn by doing them."
Aristotle

"The man who sets out to carry a cat by its tail learns something that
will always be useful and which will never grow dim or doubtful."
Mark Twain

"Believe in the Divine, but paddle away from the rocks."
Hindu Proverb

* Perl tutorials at http://perlmonks.org/?node=Tutorials
* A searchable perldoc is at http://perldoc.perl.org/

Kevin Viel

2006-03-21, 9:58 pm

> BTW, out here in the real world (that would be UNIX), *.pl stands for
> Perl Library file, not a script.

What extension do you suggest using, if any, in the real world?

Thanks,

Kevin

----------------------------------------------------------------------
Kevin Viel
Department of Genetics e-mail: kviel@darwin.sfbr.org
Southwest Foundation phone: (210)258-9884
P.O. Box 760549 fax: (210)258-9444
San Antonio, TX 78245-0549

Dave Gray

2006-03-21, 9:58 pm

On 3/21/06, Kevin Viel <kviel@darwin.sfbr.org> wrote:
>
> What extension do you suggest using, if any, in the real world?

..ps for perl script

/snicker

Timothy Johnson

2006-03-21, 9:58 pm

>
>
>-----Original Message-----
>From: Mr. Shawn H. Corey [mailto:shawnhcorey@magma.ca]
>Sent: Tuesday, March 21, 2006 2:29 PM
>To: beginners perl
>Subject: Re: regex one liner
>
>Timothy Johnson wrote:
>
>No. With alias, I can create an alias file that works with sh (and ksh,
>bash) and csh (and tcsh). This would be dot'ed in your .profile as:
>
> . ~/.alias
>
>or source'd in your .cshrc as:
>
> source ~/.alias
>
>If you want the same as a script:
>
> mv ~/pcalc.pl ~/bin/pcalc
>

It works on my SunOS box using bash. I can set an alias to the
command-line for launching my script without moving it into any special
directories, and yes, I am much more familiar with Windows than any
flavor of UNIX, so I'm aware that I might not be following best
practices, but that's not the point. I'm not completely sure you didn't
get my point, however.
=20
>BTW, out here in the real world (that would be UNIX), <snip>
>

Beyond being rude and immature, this is off-topic. Please don't try to
start a "my OS is better than your OS" war.

Timothy Johnson

2006-03-21, 9:58 pm

>
>-----Original Message-----
>From: Jay Savage [mailto:daggerquill@gmail.com]
>Sent: Tuesday, March 21, 2006 3:06 PM
>To: Timothy Johnson
>Subject: Re: regex one liner

<snip>

necessarily[color=darkred]

<snip>
[color=darkred]
>
>You're making some assumptions here that may not be valid.=20

Quite possibly.

>It's not
>always woth my time--or anyone else's--to sit down and write a tool to
>accomplish a relatively simple task, especially if it's a task that
>needs to be performed a finite number of times. Or for that matter, a
>task that needs to be performed an infinite number of times with
>infinite variations. No one is talking about "scripts". At least I'm
>not. Nothing that needs to be maintained. Nothing, for the most part,
>that another human being is ever going to see. The stuff that we all
>kepp hidden away in our .profiles and heads. The basic stuff that
>makes a sy

min or advanced user's day possible.
>

<snip >

>As far as I'm concerned, thats what the command line switches are
>there for. I look at it this way: if it's something I'd write a shell
>script for, I'll probably write a Perl script and save it to a file.
>If it's something I'd do on the cammand line with another tool, I'll
>probably do it on the command line with Perl.
>
>It's partly a matter of translation, too. If I'm turning to perl to
>clean up some variation on
>
> %sed -e 's/something/something/g' -e'something else' -e'something
else' file
>
>the Perl replacement will probably end up being one line, too.
>
>-- jay

Excellent points. I concede that there are times where it will save you
effort. I'll still maintain my prejudice against trying to do
complicated things in one-liners, but that's just because I'm stubborn
like that.

Omega -1911

2006-03-21, 9:58 pm

With all the debate going on, let me just say that a perl script was THE
ONLY method that allowed me to change paths in 13,161 files and scripts when
I moved from one provider to another. Try a *one-liner" on multiple files
(13,000+) located in different directories...

With a Perl script, I can make the needed changes, make a copy of the
original for back-up purposes, and move files accordingly. Will I always
remember a *one-liner* that will do all that? Certainly not, but I will
always have that script. Honestly, was there a better way then using a
script for the above purpose?

I like this list, but sometimes I think authors forget from wince they came
making the "barrier to entry" difficult as if job security was an issue.

~Feel free to allow knowledge freedom~

On 3/21/06, Timothy Johnson <tjohnson@zonelabs.com> wrote:
>
>
> It works on my SunOS box using bash. I can set an alias to the
> command-line for launching my script without moving it into any special
> directories, and yes, I am much more familiar with Windows than any
> flavor of UNIX, so I'm aware that I might not be following best
> practices, but that's not the point. I'm not completely sure you didn't
> get my point, however.
>
>
> Beyond being rude and immature, this is off-topic. Please don't try to
> start a "my OS is better than your OS" war.
>
>
> --
> To unsubscribe, e-mail: beginners-unsubscribe@perl.org
> For additional commands, e-mail: beginners-help@perl.org
> <http://learn.perl.org/> <http://learn.perl.org/first-response>
>
>
>

Chad Perrin

2006-03-21, 9:58 pm

On Tue, Mar 21, 2006 at 09:27:28AM -0500, Jay Savage wrote:
> On 3/20/06, stu meacham <stu21774@lycos.com> wrote:
>
> 2) You've anchored your patttern at the beginning of the string. This
> makes /g a no-op; you're pattern can't possibly match more than once,
> because '^' can only appear once.

I think that was in large part my fault. I sent an email with a
proposed solution while I was half-asleep, and ended up making two
errors:

1. I accidentally sent it to the individual rather than the list.

2. I screwed up the regex pretty badly.

If I'd been fully conscious and thinking clearly, what I would have sent
would have been something like this:

$foo !~ s/\d{2}\t\d{2}\t\d{2}//g

(where $foo is what you want to modify)

I just saw the initial regex example and thought "negation", then
misused the carat. Mea culpa, Jay. I didn't mean to lead you astray.

--
Chad Perrin [ CCD CopyWrite | http://ccd.apotheon.org ]
"The measure on a man's real character is what he would do
if he knew he would never be found out." - Thomas McCauley

Chad Perrin

2006-03-21, 9:58 pm

On Tue, Mar 21, 2006 at 04:32:21PM -0700, Chad Perrin wrote:
>
> misused the carat. Mea culpa, Jay. I didn't mean to lead you astray.

.. . . and here I go with the stupid mistakes again. I meant Stu, not
Jay.

--
Chad Perrin [ CCD CopyWrite | http://ccd.apotheon.org ]
"Real ugliness is not harsh-looking syntax, but having to
build programs out of the wrong concepts." - Paul Graham

Chad Perrin

2006-03-21, 9:58 pm

On Tue, Mar 21, 2006 at 04:05:17PM -0700, Dave Gray wrote:
> On 3/21/06, Kevin Viel <kviel@darwin.sfbr.org> wrote:
>
> .ps for perl script

Funny.

Actually, I tend to either use no file extension or .plx as the file
extension for a non-library script. I'm pretty sure ActivePerl
recognizes .plx (not entirely sure, but pretty sure), though I generally
do all my scripting on *nix, so it's not really an issue here.

--
Chad Perrin [ CCD CopyWrite | http://ccd.apotheon.org ]
"The ability to quote is a serviceable
substitute for wit." - W. Somerset Maugham

Mr. Shawn H. Corey

2006-03-21, 9:58 pm

Timothy Johnson wrote:
> Beyond being rude and immature, this is off-topic. Please don't try to
> start a "my OS is better than your OS" war.

I wasn't trying to be rude but Perl was developed and evolved in UNIX;
something most people don't know. In UNIX, all scripts, whether there
are Perl, sh, sed, awk, or anything else, do not have extensions. In
fact, in UNIX, extensions are conventions, not necessities.

I apologize if my post caused any bad feelings.

--

Just my 0.00000002 million dollars worth,
--- Shawn

"For the things we have to learn before we can do them,
we learn by doing them."
Aristotle

"The man who sets out to carry a cat by its tail learns something that
will always be useful and which will never grow dim or doubtful."
Mark Twain

"Believe in the Divine, but paddle away from the rocks."
Hindu Proverb

* Perl tutorials at http://perlmonks.org/?node=Tutorials
* A searchable perldoc is at http://perldoc.perl.org/

Mr. Shawn H. Corey

2006-03-21, 9:58 pm

Chad Perrin wrote:
> Actually, I tend to either use no file extension or .plx as the file
> extension for a non-library script. I'm pretty sure ActivePerl
> recognizes .plx (not entirely sure, but pretty sure), though I generally
> do all my scripting on *nix, so it's not really an issue here.
>

In UNIX, you should use file(1) to determine what the file is.

--

Just my 0.00000002 million dollars worth,
--- Shawn

"For the things we have to learn before we can do them,
we learn by doing them."
Aristotle

"The man who sets out to carry a cat by its tail learns something that
will always be useful and which will never grow dim or doubtful."
Mark Twain

"Believe in the Divine, but paddle away from the rocks."
Hindu Proverb

* Perl tutorials at http://perlmonks.org/?node=Tutorials
* A searchable perldoc is at http://perldoc.perl.org/

Stu Meacham

2006-03-22, 3:57 am

Good question. I went to a script right away after the difficulty with the=
one-liner and returned to the command line syntax problem more out of curi=
osity than anything else. I'm a fan of the command line because once I get=
one that works all I need is the 'history' command and an up arrow key to =
retrieve it--one less file. You're right though, cleverness is overated.
Stu

> --
>=20
> We run into one of these "How do I do this in a one-liner?" questions
> pretty frequently, and I for one have to ask, what exactly makes the
> one-liner so compelling, especially when you are using it for something
> that will be run repeatedly?
>=20
> IMHO it would have been much more practical to just create a three line
> Perl script and run it from the command-line. It makes it easier to
> read, doesn't discourage commenting, and avoids the contortions
> sometimes needed to cram it all into one line.
>=20
>=20

--=20
________________________________________
_______

Search for businesses by name, location, or phone number. -Lycos Yellow Pa=
ges

http://r.lycos.com/r/yp_emailfooter...com/default.as=
p?SRC=3Dlycos10

Sponsored Links