Question

1

Extract all files after wget download

rated 0 times [ 1] [ 0] / answers: 1 / hits: 3757 / 3 Years ago, tue, october 26, 2021, 3:51:50

I'm using wget to download a list of files from a URL, here is my command:

wget --user='username' --password='mypassword' -i  url.txt

This downloads a list of .gz files

What I need to do is:

Before downloading, delete all files in directory

After downloading, unzip all files and delete zipped ones

Is there a command or list of commands which might accomplish this which I can insert into a cronjob? I need to run this once a day.

Thanks

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

ousear

Add To Favorites

Follow

Total Points: 395

Total Questions: 114

Total Answers: 89

Location: Jordan

Member since Thu, Aug 5, 2021

3 Years ago

answered 3 Years ago ndeecru · Accepted Answer

Let's try a one-liner :

rm -rfi * && wget -q -O - --user='username' --password='mypassword' -i  url.txt | wget - && find . -name "*.zip" -exec "unzip && rm" {} ;

Don't hesitate to edit/adapt : this could be really wrong as I'm not always very at ease with the - pseudo-descriptor and how each application handles it.

Some details

Clear current directory.

rm -rfi *

Get the list, but instead of sending the contents to a file, redirect it to stdout (using -q, -O and the pseudo-descriptor -).

wget -q -O - --user='username' --password='mypassword' -i  url.txt

Download files names of which are being sent from the previous wget call. Again, - makes refers to the standard input, which is also the previous command's output.

wget -

Find all zip files in the directory, and execute unzip and rm on each of them.

find . -name "*.zip" -exec "unzip && rm" {} ;