Question

3

cat and Pipe to Run Command Won't Take Two Arguments

rated 0 times [ 3] [ 0] / answers: 1 / hits: 688 / 2 Years ago, mon, february 28, 2022, 4:12:09

I have been using this command:

cat urls.txt | xargs -n 1 -P 10 wget -q

to download text files of URLs. This worked fine when my URL file was like:

http://domain1.com/page.html

http://domain2.com/page.html

http://domain3.com/page.html

However I now need to download text files of URLs and post data, like:

--post-data '1=1' http://domain1.com/page.html

--post-data '1=1' http://domain2.com/page.html

--post-data '1=1' http://domain3.com/page.html

When using the above cat command it tries to download the URL and then the post data as a URL. e.g. in the above example, it would download http://domain1.com/page.html and then try and download --post-data 1=1, then http://domain2.com/page.html and so on.

Is there anyway to get cat to send each line of the URLs files only?

Update: I've found that by adding an escape to the space like:

--post-data '1=1' http://domain1.com/page.html

is making it be treated as one url, but the -- appear to be stripped from the --post-data argument.

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

saucdisr

Add To Favorites

Follow

Total Points: 4

Total Questions: 102

Total Answers: 117

Location: Ecuador

Member since Thu, Jun 4, 2020

4 Years ago

answered 2 Years ago ellter · Accepted Answer

To make xargs run your command once for each line of input, give it the -L 1 option (and remove the -n 1 option, since they are mutually exclusive). The standards document for xargs says this:

-L number

The utility shall be executed for each non-empty number lines of arguments

from standard input. A line is considered to end with the first <newline>

unless the last character of the line is a <blank>; a trailing <blank>

signals continuation to the next non-empty line, inclusive.