Thanks for all the ideas and comments so far. I have not (yet) got firejail to work with the VPN active (@Anonymous suggestion). But I followed up on the suggestion linked by @ofirule specifically this answer to use control groups.

That answer has a link to a complete shell script to do everything automatically but I wanted to do things manually at least to start with so I could follow along and make sure I knew exactly what I was changing. So, following the Manual HowTo there I will post below exactly what I did for anyone else.

Note that my network interface is named enp0s31f6 and my gateway IP is 192.168.0.1 I think those are the only two specific things that may need changing for others following this:

# Install required tools

sudo apt-get install cgroup-lite cgroup-tools



sudo su



# Define a control group named novpn with a classid of 11:11

cd /sys/fs/cgroup/net_cls

mkdir novpn

cd novpn

echo 0x00110011 > net_cls.classid



# Add mark 11 on packets of classid 0x00110011

iptables -t mangle -A OUTPUT -m cgroup --cgroup 0x00110011 -j MARK --set-mark 11



# Force the packets to exit through my interface (enp0s31f6) with NAT

iptables -t nat -A POSTROUTING -m cgroup --cgroup 0x00110011 -o enp0s31f6 -j MASQUERADE



# Define a new "novpn" routing table

echo 11 novpn >> /etc/iproute2/rt_tables



# Packets with mark 11 will use novpn

ip rule add fwmark 11 table novpn



# Add a route for default gateway for novpn

ip route add default via 192.168.0.1 table novpn



# Unset reverse path filtering for all interfaces

# My PREVIOUS VALUES: lo was 0, others (all, default, enp0s31f6) were 2 in case I need to revert back.

for i in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 > $i; done



exit



# Create control group    

sudo cgcreate -t $USER:$USER -a $USER:$USER -g net_cls:novpn



# Run app on specific novpn interface (close all browser windows first)

killall firefox

cgexec -g net_cls:novpn firefox

I don't (yet) fully understand all the above but it works! :-) Running firefox via the cgexec command I can check my IP in firefox with the VPN active and can see it reports my actual public IP not the VPN IP. I also proved this method works by running a radio streaming service which was having issues working with the VPN - it now works absolutely fine bypassing the VPN.