As I was reading the first paragraph, I was thinking about fail2ban.

The biggest issue here is detecting a bad user. If you do that manually, skip fail2ban and use sudo ufw deny from 1.2.3.4. That will be a permanent block but there you go.

fail2ban works best when your system (any service, including a dynamic website) sends things to the logs (syslog or service specific). fail2ban then has a stack of things to look for and then what to do if it finds things.

For example, I'm currently employing a fail2ban plugin for Wordpress that sends events to syslog. fail2ban detects three incorrect tries and then blocks the IP for five minutes. It's genius stuff that has pretty much entirely blocked brute force attacks. I mention that plugin as it's a good example of a simple custom-written ruleset. It's easy to see how it works and adapt it for your own needs.

Email notification is pretty simple but you can go further and email nmap scans back. Thinking about it, it might be worth running the IP through a whois, extracting the abuse email and automatically sending an abuse report when you ban a user (explaining why).