Question

3

How do I move thousands of files from a list?

rated 0 times [ 3] [ 0] / answers: 1 / hits: 1540 / 2 Years ago, mon, february 14, 2022, 6:40:04

So, I have a list of files in a text file. I believe it's about 100,000 files.

The files in said list are spread across many directories, have different sizes, filenames, extensions, ages, etc.

I am trying to find a way to move those files, and just those, to another drive.

Complicating factor: some of the files have the same name, but are not the same file. They can't just be moved into one folder with an overwriting or ignoring policy towards multiples.

Preferably, I would like them to retain their directory structure, but only have the files that I want inside the destination directory. (the destination drive isn't big enough to simply copy everything).

Below is an example of some lines in the text file that has the names of the source files:

media/dave/xdd/cruzer/F#(NTFS 1)/Raw Files/Portable Network Graphic file/3601-3900/FILE3776.PNG/Windows/winsxs/amd64_microsoft-windows-o..disc-style-memories_31bf3856ad364e35_6.1.7600.16385_none_51190840a935f980/Title_mainImage-mask.png

media/dave/xdd/d1/other/hd1/Program Files/DVD Maker/Shared/DvdStyles/Memories/Title_content-background.png

I have tried to use

rsync -av `cat /sourcefile.txt` /media/destinationhdd

which complains that there are too many arguments.

Also

rsync -a --files-from=/sourcefile.txt / /media/destinationhdd

and

cat /sourcefile.txt | xargs -n 200 rsync -av % /media/destinationhdd

However, this just tries to copy my root directory to the destination.

How do I just copy the specific files that I want to?

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

scrubuz

Add To Favorites

Follow

Total Points: 194

Total Questions: 96

Total Answers: 127

Location: Monaco

Member since Fri, Sep 24, 2021

3 Years ago

answered 2 Years ago homerurhyth · Accepted Answer

Here is a little shell script for you:

#!/bin/sh



while read line

do

    DIR=`dirname "$line"`

    mkdir -p "$2/$DIR"

    mv "$line" "$2/$DIR"

done < $1

Usage (assuming you saved the script as script.sh and made it executable with chmod +x script.sh):

./script.sh input.txt output_directory

It will move all files listed in input.txt to output_directory, using their original paths, for example, for an input.txt has the following list:

test.txt

dir1/test.txt

Another Test/something_else.txt

The files will be moved to:

output_directory/test.txt

output_directory/dir1/test.txt

output_directory/Another Test/something_else.txt

I did some testing before posting this answer, but please make sure you try it on a smaller sample first to confirm it's working as expected!