Unix Shell Programming - find with delete error! Beginner!

Author

find with delete error! Beginner!

Marek Stepanek

2006-11-26, 4:12 am

Hello all!

I did a terrible mistake in a friend's computer. There were many duplicates
in his computer of music files of the type:

name.mp3
name 1.mp3
name 2.mp3
name 3.mp3
....
name 7.mp3

I made a find, finding all files with a name, a space, a digit, a dot and
the extension mp3:

find . -iregex ".* [[:digit:]]\.mp3" -print

which found every wanted file, but not the original of the type:

name.mp3

So I made

find . -delete -iregex ".* [[:digit:]]\.mp3" -print

BUT all subfolders were removed! Including all "name.mp3". All his music
folder is empty now! And it was my mistake! :-(

I tried it now on my computer with a test. Same result!!!

[markslap:~/Desktop/test] mareklap% ls -r */
test02/:
total 64
8 test.txt 8 test 7.txt 8 test 6.txt 8 test 5.txt 8 test 4.txt 8 test
3.txt 8 test 2.txt 8 test 1.txt

test01/:
total 64
8 test.txt 8 test 7.txt 8 test 6.txt 8 test 5.txt 8 test 4.txt 8 test
3.txt 8 test 2.txt 8 test 1.txt
[markslap:~/Desktop/test] mareklap% find . -iregex ".* [[:digit:]]\.txt"
-print
../test01/test 1.txt
../test01/test 2.txt
../test01/test 3.txt
../test01/test 4.txt
../test01/test 5.txt
../test01/test 6.txt
../test01/test 7.txt
../test02/test 1.txt
../test02/test 2.txt
../test02/test 3.txt
../test02/test 4.txt
../test02/test 5.txt
../test02/test 6.txt
../test02/test 7.txt
[markslap:~/Desktop/test] mareklap% find . -delete -iregex ".*
[[:digit:]]\.txt" -print
../test01/test 1.txt
../test01/test 2.txt
../test01/test 3.txt
../test01/test 4.txt
../test01/test 5.txt
../test01/test 6.txt
../test01/test 7.txt
../test02/test 1.txt
../test02/test 2.txt
../test02/test 3.txt
../test02/test 4.txt
../test02/test 5.txt
../test02/test 6.txt
../test02/test 7.txt
[markslap:~/Desktop/test] mareklap% ls
[markslap:~/Desktop/test] mareklap% # <-- everything removed!

First question: what did I wrong? (okok! I should have done the test first!
But no comments on this! It is hard enough, the damage I caused!)
Second question: there is no "undo" from shell errors like this, isn't it?

greetings to all

marek

Michael Heiming

2006-11-26, 4:12 am

In comp.unix.shell Marek Stepanek <mstep@podiuminternational.org>:

> Hello all!

> I did a terrible mistake in a friend's computer. There were many duplicates
> in his computer of music files of the type:

[..]

> find . -delete -iregex ".* [[:digit:]]\.mp3" -print

> BUT all subfolders were removed! Including all "name.mp3". All his music
> folder is empty now! And it was my mistake! :-(

[..]

> First question: what did I wrong? (okok! I should have done the test first!

You should have used "-type f" in your find to exclude
directories and perhaps used "-exec ..." or pipe to xargs and
obviously put commands in 'echo ...' to see what they'd do before
really executing them, if unsure about the consequences. Never
heard about the "-delete" switch at all, it isn't in the man page
of GNU find, so I am not sure what it really does.

> But no comments on this! It is hard enough, the damage I caused!)
> Second question: there is no "undo" from shell errors like this, isn't it?

Of course there is, it is called "Backup" if you have no it is
now time to rethink this hazardous backup strategy seriously.

Good luck

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | perl -pe 'y/a-z/n-za-m/'
#bofh excuse 9: doppler effect

Dave Gibson

2006-11-26, 7:02 pm

Marek Stepanek <mstep@podiuminternational.org> wrote:
>
>
>
> Hello all!
>
> I did a terrible mistake in a friend's computer. There were many
> duplicates in his computer of music files of the type:
>
> name.mp3
> name 1.mp3
> name 2.mp3
> name 3.mp3
> ...
> name 7.mp3
>
> I made a find, finding all files with a name, a space, a digit, a dot
> and the extension mp3:
[...]
> find . -delete -iregex ".* [[:digit:]]\.mp3" -print
>
> BUT all subfolders were removed! Including all "name.mp3". All his
> music folder is empty now! And it was my mistake! :-(
[...]
> First question: what did I wrong?

The -delete is misplaced. Expressions are evaluated in the order they
appear on the command line.

find . -delete -iregex '.* [[:digit:]]\.mp3' -print

Means, "Search the current directory; if anything was found, delete it;
if anything was deleted, match its name against the regular expression;
if the name matched, print it." Try this instead:

find . -type f -iregex '.* [[:digit:]]\.mp3$' -delete -print

> (okok! I should have done the test first!
> But no comments on this! It is hard enough, the damage I caused!)
> Second question: there is no "undo" from shell errors like this,
> isn't it?

It depends on the filesystem and whether it has been used since the
files were disastered.

mstep@podiuminternational.org

2006-11-26, 7:02 pm

Dave Gibson schrieb:

>
> The -delete is misplaced. Expressions are evaluated in the order they
> appear on the command line.
>
> find . -delete -iregex '.* [[:digit:]]\.mp3' -print
>
> Means, "Search the current directory; if anything was found, delete it;
> if anything was deleted, match its name against the regular expression;
> if the name matched, print it." Try this instead:
>
> find . -type f -iregex '.* [[:digit:]]\.mp3$' -delete -print
>

Thank you Dave, this was a very good explanation!

>
> It depends on the filesystem and whether it has been used since the
> files were disastered.

But you not able to "undo" things with a command in the shell, isn't
it? On Macintosh we use "Data Rescue" but it takes long time, to bring
things back.

Thank you all who replied

marek

Michael Paoli

2006-11-26, 7:02 pm

Marek Stepanek wrote:
> find . -delete -iregex ".* [[:digit:]]\.mp3" -print
> BUT all subfolders were removed! Including all "name.mp3". All his music

> First question: what did I wrong? (okok! I should have done the test first!

One thing to keep in mind, is find(1) works (literally) logically,
proceeding left to right and using the logical truth value along the
way. Since you start with:
find . -delete
At the point the -delete is encountered, the logical value is true,
so the -delete action will be done (or at least attempted) for every
pathame find examines. Anything you put after that on that find
command, short of something which terminates find's operation, will
not prevent find from making those -delete attempts.

For better or worse (generally better ... much better :-)) UNIX
generally presumes one knows what one is doing, and does what is
commanded of it. Some other operating systems take, uhm,
"alternative" approaches, ... such as asking "Are you sure?" every
time you tell it to do something - which can result in having to
reply "Yes" 5,378 or more times per day.

> Second question: there is no "undo" from shell errors like this, isn't it?

Correct, shell has no general undo capability (just some command line
editing undo capabilities for some shells ... that's probably about
it in the "undo" category), rm(1), unlink(2) and rmdir(2) have no
undo capabilities. On UNIX, it is typically infeasible to recover
removed (rm(1), etc.) items from a UNIX filesystem, particularly an
active one, once they've been removed (some or much of the data may
still be there, but typically finding all the pieces and putting them
back together is generally more time and trouble than it's worth).
It's usually much more feasible and less time consuming to restore
from backup. You *do* have backups, correct? And off-site backups
also, for localized/regional "disaster" scenarios?

Other random observation(s): Documentation isn't always 100%
complete, correct, and accurate. I don't have a find(1) that
supports -delete at my fingertips to test against, but what seems to
be implied by some of the man pages for find(1) that support -delete,
would seem to imply something which may not precisely be the case.
E.g., it states, in part:
"It will not attempt to delete a filename with a ``/'' character in
its pathname relative to ``.'' for security reasons."
That would seem to imply, for example, that:
find ./dir -delete
wouldn't delete dir and also wouldn't delete dir/dir or dir/file,
but I'd guestimate from "for security reasons" the "feature" is
probably intended to prevent -delete from removing items above the
current directory, and perhaps also the current directory, and
perhaps any pathnames that contain .. as one of the directory
components. It would also appear there may be, or at least fairly
recently have been, additional bug(s)/issue(s) with the -delete
option. That -delete option hasn't exactly been in find(1) for 20
years or even made it into all currently supported UNIX flavors, so
it may still have "issues" (portability and standards compliance
among them).

references/excerpts::
find(1)
http://www.google.com/search?hl=en&...G=Google+Search
http://developer.apple.com/document...an1/find.1.html
http://lists.freebsd.org/pipermail/...ber/016206.html
-delete
Delete found files and/or directories. Always returns true.
This executes from the current working directory as find recurses
down the tree. It will not attempt to delete a filename with a
``/'' character in its pathname relative to ``.'' for security
reasons. Depth-first traversal processing is implied by this
option.

mstep@podiuminternational.org

2006-11-28, 4:01 am

Michael Paoli schrieb:

> snip

wow! what an extensive answer! Thank you Michael !

greetings from Munich

marek

Kaz Kylheku

2006-11-28, 4:01 am

Marek Stepanek wrote:
> I made a find, finding all files with a name, a space, a digit, a dot and
> the extension mp3:
>
> find . -iregex ".* [[:digit:]]\.mp3" -print

This can be handled with -name '* [0-9].mp3". Or -iname if the mp3
suffix varies in case; note that this is a GNU extension.

> So I made
>
> find . -delete -iregex ".* [[:digit:]]\.mp3" -print

Bad idea. Unless you're a wizard, never ask find to take any
destructive action. Get the list of files out of it, verify that it's
correct and then couple that list into the destructive action.

In 17 years of Unix use, I've never used the -delete predicate of find.

The other potential culprit is -exec, which can run something harmful.
-exec an "echo" command first to do a dry run. Then replace the echo
and run the command again.

> BUT all subfolders were removed! Including all "name.mp3". All his music
> folder is empty now! And it was my mistake! :-(

Measure twice, cut once, tailor.

> I tried it now on my computer with a test. Same result!!!

And keep backups. You can make a quick and dirty backup of a folder
using hard links:

cp -rl source-dir backup-dir

The -l option means to create only hard links to files, and not
actually move data, so you're getting a virtual copy. The operation
only has to create directories and directory entries.

If you modify any of the files in the original directory, or truncate
them to zero, the virtual copies are affected. However, you are
protected against accidental deletion.

When you are happy with the result of the cleanup, then just

rm -rf backup-dir

mstep@podiuminternational.org

2006-11-30, 7:59 am

Kaz Kylheku schrieb:

> snip

thank you too, Kaz, very good idea with the backup and hard links!

Greetings from Munich

marek

Sponsored Links