Monday, May 20, 2024
 Popular · Latest · Hot · Upcoming
32
rated 0 times [  32] [ 0]  / answers: 1 / hits: 7573  / 2 Years ago, wed, may 4, 2022, 6:48:04

How can I block a list of about 1.4 million IP addresses? I've already tried to do it with iptables PREROUTING, like:


-A PREROUTING -d IP_HERE/32 -j DROP


But with this many records, my bandwidth goes down like crazy when I do a speedtest.


Without blocked IPs in iptables:


1 Gb/s


With blocked IPs in iptables:


3 Mb/s at peak.


I want to use XDP_DROP like here (last step):
https://blog.cloudflare.com/how-to-drop-10-million-packets/


But I don't have an idea how to use this. :/ (I'm really bad at programing)


Are there alternatives to this approach?


More From » networking

 Answers
2

You should have a look into ipset.


From the official website:



Ipset may be the proper tool for you [...] to store multiple IP addresses or port numbers and match against the collection by iptables.


[...] (Ipset) may store IP addresses, networks, (TCP/UDP) port numbers, MAC addresses, interface names or combinations of them in a way, which ensures lightning speed when matching an entry against a set.



To use it, you need to create an ipset, add the IPs and create an iptables rule to match with the ipset:


ipset create blacklist hash:ip hashsize 1400000
ipset add blacklist <IP-ADDRESS>
iptables -I INPUT -m set --match-set blacklist src -j DROP

A real life example of usage can be found here. Notice that it uses ipset restore instead of going through each IP in a loop because it’s much more faster.


If your list of IPs has overlaps, you may want to preprocess it to convert to IP ranges where possible. Here is an example of a tool to do it. It won't get you better performances with ipset but it will reduce the size of your list.




On a side note, in term of performances, it is very fast and scale without penalty. As the Cloudflare's blog mention, there are faster low level approaches; but it's much more complex and only adds a few bytes per seconds, which, unless you have the scale and ambition of a cloud provider, are not worth the effort.


[#1253] Thursday, May 5, 2022, 2 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
ionodest

Total Points: 252
Total Questions: 122
Total Answers: 100

Location: Liechtenstein
Member since Tue, Apr 27, 2021
3 Years ago
ionodest questions
Sat, Jan 1, 22, 06:31, 2 Years ago
Wed, May 12, 21, 03:21, 3 Years ago
Tue, Feb 22, 22, 09:46, 2 Years ago
Thu, Jun 30, 22, 12:12, 2 Years ago
;