Static NAT causes duplicate firewall rules to be added

[richardlawley]

ISSUE TYPE

• Bug Report

COMPONENT NAME

VR

CLOUDSTACK VERSION

4.11.2

CONFIGURATION

Advanced Networking

OS / ENVIRONMENT

n/a

SUMMARY

When two static NAT IPs exist on a network, every time a firewall rule is added or removed, duplicate CONNMARK rules are added to mangle/PREROUTING. Over time and on a busy network, this causes a significant number of duplicate rules and an increase in processing time.

I believe this is caused by the rules being defined with -I PREROUTING instead of -A PREROUTING, as the code appears to be trying to match them to the output of iptables-save. This may have been introduced in 40d7746.

I've fixed this locally by changing in configure.py:

self.fw.append(["mangle", "",
 "-I PREROUTING -s %s/32 -m state --state NEW -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff" %
 rule["internal_ip"]])

to

self.fw.append(["mangle", "front",
 "-A PREROUTING -s %s/32 -m state --state NEW -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff" %
 rule["internal_ip"]])

However, there are more instances of adding rules with -I in the file related to VPC and VPNs, which I don't have a way of testing at the moment.

STEPS TO REPRODUCE

• Create advanced network
• Deploy two VMs
• Enable static NAT for both IPs
• Create one Firewall rule
• Log onto VR and run iptables -t mangle -nL PREROUTING --line-numbers
• Add a rule and repeat

For a simpler repeat repro, log onto the VR, then:

• Find a firewall rules file in /var/cache/cloud/processed/firewall_rules,json.XXXXX.gz and gunzip it
• iptables -t mangle -F PREROUTING --line-numbers
• /opt/cloud/bin/configure /var/cache/cloud/processed/firewall_rules,json.XXXXX
• iptables -t mangle -F PREROUTING --line-numbers

EXPECTED RESULTS

No increase in the number of rules in PREROUTING

# iptables -t mangle -nL PREROUTING --line-numbers
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
1 MARK all -- 10.1.1.167 0.0.0.0/0 state NEW MARK set 0x2
2 CONNMARK all -- 10.1.1.167 0.0.0.0/0 state NEW CONNMARK save
3 MARK all -- 10.1.1.75 0.0.0.0/0 state NEW MARK set 0x2
4 CONNMARK all -- 10.1.1.75 0.0.0.0/0 state NEW CONNMARK save
5 FIREWALL_192.168.23.137 all -- 0.0.0.0/0 192.168.23.137
6 VPN_192.168.23.137 all -- 0.0.0.0/0 192.168.23.137
7 FIREWALL_192.168.23.136 all -- 0.0.0.0/0 192.168.23.136
8 VPN_192.168.23.136 all -- 0.0.0.0/0 192.168.23.136
9 FIREWALL_192.168.23.135 all -- 0.0.0.0/0 192.168.23.135
10 VPN_192.168.23.135 all -- 0.0.0.0/0 192.168.23.135
11 CONNMARK all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED CONNMARK restore
12 CONNMARK all -- 0.0.0.0/0 0.0.0.0/0 state NEW CONNMARK set 0x2

ACTUAL RESULTS

PREROUTING rules were increased by N, where N is the number of static NAT rules.

Error messages when running configure.py

# /opt/cloud/bin/configure.py /var/cache/cloud/processed/firewall_rules.json.2c3c52b5-9888-4fe4-a9d7-8b559fe98622
iptables v1.6.2: CONNMARK target: No operation specified
Try `iptables -h' or 'iptables --help' for more information.
iptables v1.6.2: CONNMARK target: No operation specified
Try `iptables -h' or 'iptables --help' for more information.

Duplicate firewall rules added (lines 1 & 2 here)

# iptables -t mangle -nL PREROUTING --line-numbers
Chain PREROUTING (policy ACCEPT)
num target prot opt source destination
1 CONNMARK all -- 10.1.1.167 0.0.0.0/0 state NEW CONNMARK save
2 CONNMARK all -- 10.1.1.75 0.0.0.0/0 state NEW CONNMARK save
3 MARK all -- 10.1.1.167 0.0.0.0/0 state NEW MARK set 0x2
4 CONNMARK all -- 10.1.1.167 0.0.0.0/0 state NEW CONNMARK save
5 MARK all -- 10.1.1.75 0.0.0.0/0 state NEW MARK set 0x2
6 CONNMARK all -- 10.1.1.75 0.0.0.0/0 state NEW CONNMARK save
7 FIREWALL_192.168.23.137 all -- 0.0.0.0/0 192.168.23.137
8 VPN_192.168.23.137 all -- 0.0.0.0/0 192.168.23.137
9 FIREWALL_192.168.23.136 all -- 0.0.0.0/0 192.168.23.136
10 VPN_192.168.23.136 all -- 0.0.0.0/0 192.168.23.136
11 FIREWALL_192.168.23.135 all -- 0.0.0.0/0 192.168.23.135
12 VPN_192.168.23.135 all -- 0.0.0.0/0 192.168.23.135
13 CONNMARK all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED CONNMARK restore
14 CONNMARK all -- 0.0.0.0/0 0.0.0.0/0 state NEW CONNMARK set 0x2

[yadvr]

Merged in #3366