Friday, May 3, 2024
 Popular · Latest · Hot · Upcoming
5
rated 0 times [  5] [ 0]  / answers: 1 / hits: 3325  / 3 Years ago, fri, october 15, 2021, 2:11:41

In question How to rename all files in folder with name ending with "_backup" @Radu Rădeanu gave a good answer that was useful to me as well:



find . -type f -name '*.jpg_backup' -print0 
| while IFS= read -r -d '' file ; do mv -- "$file"
"$(echo $file | sed 's/_backup//g')"; done


However, I would like to fully understand his one-liner. Precisely, the part that I don't understand is:



while IFS= read -r -d '' file


I get that IFS is "internal field separator" and I suppose this is removing or ignoring the whitespace, but I don't understand the syntax and the options.



I would also like to understand why the -- is necessary after mv.



Could someone help? Thanks.


More From » bash

 Answers
7

read word1 word2 … rest does the following:




  1. Read a line.

  2. While the current input ends in a backslash, read another line and append it to the first one, minus the backslash-newline.

  3. Break the input into separate words, where any character in the value of the IFS variable is considered a word separator.

  4. Assign the first word to the variable word, the second word to the variable word2, etc.

  5. If there are words left over, they are assigned to the variable rest, with the original word separators inside preserved.



The option -r deactivates step 2, so a at the end of a line is not considered a word separator.



With IFS set to the empty string, there are no word separators, so the whole line is one big word: step 3 does nothing, and step 5 ends up assigning the original line to the specified variable. Regarding why IFS is set in this location, see Why is while IFS= read used so often, instead of IFS=; while read..?.



The option -d changes the notion of line: normally a line ends in a newline character; with -d '' (empty argument to -d), bash reads input records delimited by a null byte instead.



The upshot is that find … -print0 | while IFS= read -r -d '' file; do … executes the loop body for each match that find prints, regardless of any special characters that may occur in the file name.






As for the -- in mv -- "$file", it's there in case the value of file begins with a dash: mv would interpret it as an option. In this particular case, it isn't necessary, because the ouptut of this find command always starts with ./. Using -- systematically in scripts is arguably good hygiene.






There is a defect in this snippet: it still won't cope with file names containing whitespace or [?*, because of the "$(echo $file | sed 's/_backup//g')" bit. A variable expansion like $file doesn't just substitute the value of the variable: it splits the variable into words using the value of IFS (just like read does), and treats each word as a wildcard pattern that is replaced by the list of matching files if it matches any. To avoid this behavior, write "$file". This is a general shell programming rule: always put double quotes around variable substitutions (and command substitutions $(…) too), unless you know why you need to leave them out. (If you want the nitty-gritty, see When is double-quoting necessary?; $VAR vs ${VAR} and to quote or not to quote and What is the significance of single and double quotes in environment variables? may also be of interest).



The quick fix is to add the double quotes:



mv -- "$file" "$(echo "$file" | sed 's/_backup//g')"


This happens to work here because of the way the input is produced, but for general values of file it fails in a few rare cases:




  • If the value of file consists of the character - followed by one or more letters among Een, then echo will treat it as an option and output nothing.

  • Command substitution eats up the last newlines of the output, so if $file ends with one or more newline characters, they will be truncated.



Bash has a string substitution which can be used instead of sed here. It's more robust (you don't need to worry about subtleties with special characters) and faster.



mv -- "$file" "${file//_backup/}"


Though given what the question asked for, this is the wrong operation: it removes _backup anywhere in the file name instead of only at the end. Here doing it right would be easier.



mv -- "$file" "${file%_backup}"

[#27748] Saturday, October 16, 2021, 3 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
bblerest

Total Points: 240
Total Questions: 119
Total Answers: 113

Location: Wallis and Futuna
Member since Mon, May 18, 2020
4 Years ago
;