Allow traffic from private gateway to internet by ravening · Pull Request #5397 · apache/cloudstack

ravening · 2021-09-01T11:45:13Z

Description

When private gateway is configured, the external server
which are connected through private gateway cannot send
traffic to outside world. Add a source iptable rule so
that the servers which are connected through private gateway
can ping the public IP's

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Configure the private gateway in a vpc
Configure an external dedicated server to use the private gateway to communicate with the outside world as well as with the instances in the cloudstack
by default we can ping the dedicated server from the outside world and we cannot ping from dedicated server to the outside world. However we can ping only from dedicated server to the virtual router
after this change we are able to ping to the outside world

weizhouapache · 2021-09-01T11:57:34Z

@ravening you can set acl on private gateway.

ravening · 2021-09-01T12:42:26Z

@ravening you can set acl on private gateway.

@weizhouapache we tried and it didn't work.
If I add ACL rule, it creates a new rule under ACL_* chain but that didn't help

weizhouapache · 2021-09-01T13:07:41Z

@ravening you can set acl on private gateway.
@weizhouapache we tried and it didn't work.
If I add ACL rule, it creates a new rule under ACL_* chain but that didn't help

@ravening
it is risky to allow all traffic from private gateway to internet via vpc vr.
therefore I prefer to achieve the goal via ACL. if ACl does not work, fix it then.

ravening · 2021-09-01T13:35:26Z

@ravening you can set acl on private gateway.
@weizhouapache we tried and it didn't work.
If I add ACL rule, it creates a new rule under ACL_* chain but that didn't help
@ravening
it is risky to allow all traffic from private gateway to internet via vpc vr.
therefore I prefer to achieve the goal via ACL. if ACl does not work, fix it then.

@weizhouapache I couldn't find the rule which adds the -s option which is needed for source ip subnet

ravening · 2021-09-01T23:33:58Z

@weizhouapache please review it again.

weizhouapache · 2021-09-02T08:21:12Z

@ravening
is the private gateway IP same as the gateway ?

this issue should has been fixed by a8c9b45

ravening · 2021-09-02T08:37:32Z

@ravening
is the private gateway IP same as the gateway ?
this issue should has been fixed by a8c9b45

@weizhouapache no they aren't same

weizhouapache · 2021-09-02T09:01:22Z

@ravening
is the private gateway IP same as the gateway ?
this issue should has been fixed by a8c9b45
@weizhouapache no they aren't same

vpc vr acts as the gateway of private network, right ?

ravening · 2021-09-02T09:02:32Z

@ravening
is the private gateway IP same as the gateway ?
this issue should has been fixed by a8c9b45
@weizhouapache no they aren't same
vpc vr acts as the gateway of private network, right ?

@weizhouapache yes but gateway ends with .126 whereas up ends with .125

weizhouapache · 2021-09-02T09:18:11Z

yes but gateway ends with .126 whereas up ends with .125

@ravening I think you should set the gateway to IP address when you add the private gateway.

ravening · 2021-09-02T09:31:15Z

yes but gateway ends with .126 whereas up ends with .125
@ravening I think you should set the gateway to IP address when you add the private gateway.

@weizhouapache we tried changing "private gateway" to ip address but then it breaks ssh from vm's to dedicated server

ravening · 2021-09-02T10:16:38Z

yes but gateway ends with .126 whereas up ends with .125
@ravening I think you should set the gateway to IP address when you add the private gateway.
@weizhouapache we tried changing "private gateway" to ip address but then it breaks ssh from vm's to dedicated server

@weizhouapache this is what we found

if "private gateway" is same as public ip then ping from dedicated server to internet works but ssh from vm's to dedicated server doesnt work with all proper network acl's

if they are different then ping from dedicated server to internet doesnt work but ssh from vm's to dedicated server works

so after making this code change, both scenarios works

weizhouapache · 2021-09-02T10:36:38Z

yes but gateway ends with .126 whereas up ends with .125
@ravening I think you should set the gateway to IP address when you add the private gateway.
@weizhouapache we tried changing "private gateway" to ip address but then it breaks ssh from vm's to dedicated server
@weizhouapache this is what we found
if "private gateway" is same as public ip then ping from dedicated server to internet works but ssh from vm's to dedicated server doesnt work with all proper network acl's
if they are different then ping from dedicated server to internet doesnt work but ssh from vm's to dedicated server works
so after making this code change, both scenarios works

@ravening I guess your dedicated servers are still using .126 as gateway.

ravening · 2021-09-02T11:12:02Z

yes but gateway ends with .126 whereas up ends with .125
@ravening I think you should set the gateway to IP address when you add the private gateway.
@weizhouapache we tried changing "private gateway" to ip address but then it breaks ssh from vm's to dedicated server
@weizhouapache this is what we found
if "private gateway" is same as public ip then ping from dedicated server to internet works but ssh from vm's to dedicated server doesnt work with all proper network acl's
if they are different then ping from dedicated server to internet doesnt work but ssh from vm's to dedicated server works
so after making this code change, both scenarios works
@ravening I guess your dedicated servers are still using .126 as gateway.

@weizhouapache 125 is configured as gateway in his routes on dedicated server

weizhouapache · 2021-09-02T12:21:38Z

yes but gateway ends with .126 whereas up ends with .125
@ravening I think you should set the gateway to IP address when you add the private gateway.
@weizhouapache we tried changing "private gateway" to ip address but then it breaks ssh from vm's to dedicated server
@weizhouapache this is what we found
if "private gateway" is same as public ip then ping from dedicated server to internet works but ssh from vm's to dedicated server doesnt work with all proper network acl's
if they are different then ping from dedicated server to internet doesnt work but ssh from vm's to dedicated server works
so after making this code change, both scenarios works
@ravening I guess your dedicated servers are still using .126 as gateway.
@weizhouapache 125 is configured as gateway in his routes on dedicated server

@ravening have you restarted the vpc or vr ?

ravening · 2021-09-02T12:55:39Z

yes but gateway ends with .126 whereas up ends with .125
@ravening I think you should set the gateway to IP address when you add the private gateway.
@weizhouapache we tried changing "private gateway" to ip address but then it breaks ssh from vm's to dedicated server
@weizhouapache this is what we found
if "private gateway" is same as public ip then ping from dedicated server to internet works but ssh from vm's to dedicated server doesnt work with all proper network acl's
if they are different then ping from dedicated server to internet doesnt work but ssh from vm's to dedicated server works
so after making this code change, both scenarios works
@ravening I guess your dedicated servers are still using .126 as gateway.
@weizhouapache 125 is configured as gateway in his routes on dedicated server
@ravening have you restarted the vpc or vr ?

@weizhouapache yes tried all those things... Nothing helped

yadvr · 2021-09-07T19:36:50Z

Assuming this issue would have existed for years and staying conservative towards cutting the RC I did not included in the 4.15.2 milestone. Therefore pl discuss and if it's the case change base branch to main. One concern is what if allowing public traffic via pg causes some regression or security issue?

weizhouapache · 2021-09-10T10:21:34Z

yes but gateway ends with .126 whereas up ends with .125
@ravening I think you should set the gateway to IP address when you add the private gateway.
@weizhouapache we tried changing "private gateway" to ip address but then it breaks ssh from vm's to dedicated server
@weizhouapache this is what we found
if "private gateway" is same as public ip then ping from dedicated server to internet works but ssh from vm's to dedicated server doesnt work with all proper network acl's

@ravening
let's focus on investigating the issue (ssh from vm to dedicated server does not work) in this scenario.

can you share the iptables rules which contains the device for private gateway in VR ?

iptables-save |grep ethx

if they are different then ping from dedicated server to internet doesnt work but ssh from vm's to dedicated server works
so after making this code change, both scenarios works

weizhouapache · 2021-09-13T13:32:45Z

let's focus on investigating the issue (ssh from vm to dedicated server does not work) in this scenario.
can you share the iptables rules which contains the device for private gateway in VR ?

Hi @ravening
could you share the iptables rules and route table in VR and dedicated server?

ip a
iptables-save
ip route

ravening · 2021-09-14T07:45:37Z

@weizhouapache below are the details

# ip a
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 1e:00:8f:00:8d:76 brd ff:ff:ff:ff:ff:ff
inet 10.32.22.125/27 brd 10.32.22.127 scope global eth2
valid_lft forever preferred_lft forever
# ip route
default via 5.79.116.62 dev eth1
5.79.116.32/27 dev eth1 proto kernel scope link src 5.79.116.33
10.32.22.96/27 dev eth2 proto kernel scope link src 10.32.22.125
10.69.0.0/24 dev eth3 proto kernel scope link src 10.69.0.252
10.69.1.0/24 dev eth4 proto kernel scope link src 10.69.1.252
10.69.2.0/24 dev eth5 proto kernel scope link src 10.69.2.251
10.69.3.0/24 dev eth6 proto kernel scope link src 10.69.3.252
169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.0.255
# iptables-save | grep eth2
:ACL_INBOUND_eth2 - [0:0]
:NETWORK_STATS_eth2 - [0:0]
-A FORWARD -j NETWORK_STATS_eth2
-A FORWARD -d 10.32.22.96/27 -o eth2 -j ACL_INBOUND_eth2
-A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth2 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth2 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth2 -j DROP
-A NETWORK_STATS_eth2 -d 10.69.0.0/16 -i eth2
-A NETWORK_STATS_eth2 -s 10.69.0.0/16 -o eth2
-A NETWORK_STATS -i eth0 -o eth2 -p tcp
-A NETWORK_STATS -i eth2 -o eth0 -p tcp
-A NETWORK_STATS ! -i eth0 -o eth2 -p tcp
-A NETWORK_STATS -i eth2 ! -o eth0 -p tcp
:VPN_STATS_eth2 - [0:0]
-A PREROUTING -i eth2 -m state --state NEW -j CONNMARK --set-xmark 0x66/0xffffffff
-A FORWARD -j VPN_STATS_eth2
-A VPN_STATS_eth2 -o eth2 -m mark --mark 0x525
-A VPN_STATS_eth2 -i eth2 -m mark --mark 0x524

weizhouapache · 2021-09-14T08:10:51Z

@ravening

I want to be clear about
(1) is 10.32.22.125 configured as gateway in dedicated servers (10.32.22.96/27)
(2) does ping from vm to dedicated server work ?
I notice there are many rules in ACL for private gateway, ssh from vm to dedicated server might not work, but ping should work.

-A FORWARD -d 10.32.22.96/27 -o eth2 -j ACL_INBOUND_eth2
-A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT

@weizhouapache below are the details

# ip a
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 1e:00:8f:00:8d:76 brd ff:ff:ff:ff:ff:ff
inet 10.32.22.125/27 brd 10.32.22.127 scope global eth2
valid_lft forever preferred_lft forever
# ip route
default via 5.79.116.62 dev eth1
5.79.116.32/27 dev eth1 proto kernel scope link src 5.79.116.33
10.32.22.96/27 dev eth2 proto kernel scope link src 10.32.22.125
10.69.0.0/24 dev eth3 proto kernel scope link src 10.69.0.252
10.69.1.0/24 dev eth4 proto kernel scope link src 10.69.1.252
10.69.2.0/24 dev eth5 proto kernel scope link src 10.69.2.251
10.69.3.0/24 dev eth6 proto kernel scope link src 10.69.3.252
169.254.0.0/16 dev eth0 proto kernel scope link src 169.254.0.255
# iptables-save | grep eth2
:ACL_INBOUND_eth2 - [0:0]
:NETWORK_STATS_eth2 - [0:0]
-A FORWARD -j NETWORK_STATS_eth2
-A FORWARD -d 10.32.22.96/27 -o eth2 -j ACL_INBOUND_eth2
-A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth2 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth2 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth2 -j DROP
-A NETWORK_STATS_eth2 -d 10.69.0.0/16 -i eth2
-A NETWORK_STATS_eth2 -s 10.69.0.0/16 -o eth2
-A NETWORK_STATS -i eth0 -o eth2 -p tcp
-A NETWORK_STATS -i eth2 -o eth0 -p tcp
-A NETWORK_STATS ! -i eth0 -o eth2 -p tcp
-A NETWORK_STATS -i eth2 ! -o eth0 -p tcp
:VPN_STATS_eth2 - [0:0]
-A PREROUTING -i eth2 -m state --state NEW -j CONNMARK --set-xmark 0x66/0xffffffff
-A FORWARD -j VPN_STATS_eth2
-A VPN_STATS_eth2 -o eth2 -m mark --mark 0x525
-A VPN_STATS_eth2 -i eth2 -m mark --mark 0x524

ravening · 2021-09-14T08:19:24Z

@weizhouapache

yes 10.32.22.125 is configured as gateway in dedicated servers but cloudstack says 10.32.22.126 as the gateway
if he tries to configure 10.32.22.126 as the gateway in dedicated server then ping wont work

without any code change, the ping from vm to dedicated server works but ping from dedicated server to internet doesnt work

if i make both public ip and gateway 10.32.22.125 in cloudstack then ping from decdicated server to internet works but ssh from vm to dedicated server doesnt work (ping still works)

with this code change both scenarios works

weizhouapache · 2021-09-14T08:28:14Z

@weizhouapache
yes 10.32.22.125 is configured as gateway in dedicated servers but cloudstack says 10.32.22.126 as the gateway
if he tries to configure 10.32.22.126 as the gateway in dedicated server then ping wont work
without any code change, the ping from vm to dedicated server works but ping from dedicated server to internet doesnt work
if i make both public ip and gateway 10.32.22.125 in cloudstack then ping from decdicated server to internet works but ssh from vm to dedicated server doesnt work (ping still works)
with this code change both scenarios works

@ravening
since ping works but ssh does not work (from vm to dedicated server), ip configuration, route should be ok.

it should be caused by ACL rules.
As I pointed out in my previous comment, ssh works only from 10.69.3.0/24 and 10.0.0.0/16.
I suggest to add ACL rule to allow ssh (port 22) from 10.69.0.0/24, 10.69.1.0/24, 10.69.2.0/24

ravening · 2021-09-14T08:42:11Z

@weizhouapache
yes 10.32.22.125 is configured as gateway in dedicated servers but cloudstack says 10.32.22.126 as the gateway
if he tries to configure 10.32.22.126 as the gateway in dedicated server then ping wont work
without any code change, the ping from vm to dedicated server works but ping from dedicated server to internet doesnt work
if i make both public ip and gateway 10.32.22.125 in cloudstack then ping from decdicated server to internet works but ssh from vm to dedicated server doesnt work (ping still works)
with this code change both scenarios works
@ravening
since ping works but ssh does not work (from vm to dedicated server), ip configuration, route should be ok.
it should be caused by ACL rules.
As I pointed out in my previous comment, ssh works only from 10.69.3.0/24 and 10.0.0.0/16.
I suggest to add ACL rule to allow ssh (port 22) from 10.69.0.0/24, 10.69.1.0/24, 10.69.2.0/24

@weizhouapache

should I add this to all tiers?

I tested the ssh connection from the "admin tier" which has the below acl rule

100 : cidr 0.0.0.0/0 icmp -1 -1 allow all direction: ingress
101: cidr 0.0.0.0/0 port 22 tcp allow all direction- ingress

1000: cidr 0.0.0.0/0 action : deny protocol: all direction: ingress

weizhouapache · 2021-09-14T08:44:45Z

@weizhouapache
yes 10.32.22.125 is configured as gateway in dedicated servers but cloudstack says 10.32.22.126 as the gateway
if he tries to configure 10.32.22.126 as the gateway in dedicated server then ping wont work
without any code change, the ping from vm to dedicated server works but ping from dedicated server to internet doesnt work
if i make both public ip and gateway 10.32.22.125 in cloudstack then ping from decdicated server to internet works but ssh from vm to dedicated server doesnt work (ping still works)
with this code change both scenarios works
@ravening
since ping works but ssh does not work (from vm to dedicated server), ip configuration, route should be ok.
it should be caused by ACL rules.
As I pointed out in my previous comment, ssh works only from 10.69.3.0/24 and 10.0.0.0/16.
I suggest to add ACL rule to allow ssh (port 22) from 10.69.0.0/24, 10.69.1.0/24, 10.69.2.0/24
@weizhouapache
should I add this to all tiers?
I tested the ssh connection from the "admin tier" which has the below acl rule
100 : cidr 0.0.0.0/0 icmp -1 -1 allow all direction: ingress
101: cidr 0.0.0.0/0 port 22 tcp allow all direction- ingress
1000: cidr 0.0.0.0/0 action : deny protocol: all direction: ingress

@ravening I think you need to change the ACL rules of private gateway, not VPC tiers.

ravening · 2021-09-14T08:55:46Z

@weizhouapache
yes 10.32.22.125 is configured as gateway in dedicated servers but cloudstack says 10.32.22.126 as the gateway
if he tries to configure 10.32.22.126 as the gateway in dedicated server then ping wont work
without any code change, the ping from vm to dedicated server works but ping from dedicated server to internet doesnt work
if i make both public ip and gateway 10.32.22.125 in cloudstack then ping from decdicated server to internet works but ssh from vm to dedicated server doesnt work (ping still works)
with this code change both scenarios works
@ravening
since ping works but ssh does not work (from vm to dedicated server), ip configuration, route should be ok.
it should be caused by ACL rules.
As I pointed out in my previous comment, ssh works only from 10.69.3.0/24 and 10.0.0.0/16.
I suggest to add ACL rule to allow ssh (port 22) from 10.69.0.0/24, 10.69.1.0/24, 10.69.2.0/24
@weizhouapache
should I add this to all tiers?
I tested the ssh connection from the "admin tier" which has the below acl rule
100 : cidr 0.0.0.0/0 icmp -1 -1 allow all direction: ingress
101: cidr 0.0.0.0/0 port 22 tcp allow all direction- ingress
1000: cidr 0.0.0.0/0 action : deny protocol: all direction: ingress
@ravening I think you need to change the ACL rules of private gateway, not VPC tiers.

@weizhouapache you mean to say add static routes under the private gateay?

weizhouapache · 2021-09-14T09:33:34Z

@weizhouapache you mean to say add static routes under the private gateay?

@ravening no. when you add a private gateway, you can specify a network ACL. you need to add rules in ACL.

weizhouapache · 2021-09-16T09:24:07Z

his is my current setup now
private gateway ends with .125 and public ip ends with .126
I reverted the code change
ping/ssh from vm to dedicated server works fine but ping from dedicated server to internet doesnt work
should I make both private gateway and public ip same and add the acl rules?

@ravening yes. if ping works (from vm to dedicated server and dedicated server to internet), VPC VR should be ok, then you need to check ACL rules.

ravening · 2021-09-16T09:25:56Z

his is my current setup now
private gateway ends with .125 and public ip ends with .126
I reverted the code change
ping/ssh from vm to dedicated server works fine but ping from dedicated server to internet doesnt work
should I make both private gateway and public ip same and add the acl rules?
@ravening yes. if ping works (from vm to dedicated server and dedicated server to internet), VPC VR should be ok, then you need to check ACL rules.

@weizhouapache This is the test I did

made both private gateway and public ip end with .125
reverted the code changes
added the 3 cidr list you provided to the network acl

result:

ping from dedicated server to internet works
ssh from vm to dedicated server DOES NOT work. ping works though

weizhouapache · 2021-09-16T09:32:29Z

his is my current setup now
private gateway ends with .125 and public ip ends with .126
I reverted the code change
ping/ssh from vm to dedicated server works fine but ping from dedicated server to internet doesnt work
should I make both private gateway and public ip same and add the acl rules?
@ravening yes. if ping works (from vm to dedicated server and dedicated server to internet), VPC VR should be ok, then you need to check ACL rules.
@weizhouapache This is the test I did
made both private gateway and public ip end with .125
reverted the code changes
added the 3 cidr list you provided to the network acl
result:
ping from dedicated server to internet works
ssh from vm to dedicated server DOES NOT work. ping works though

@ravening can you share the iptables rules (ACL_INBOUND and ACL_OUTBOUND) ?

ravening · 2021-09-16T09:35:11Z

@weizhouapache

inbound

# iptables-save | grep ACL_INBOUND
:ACL_INBOUND_eth2 - [0:0]
:ACL_INBOUND_eth3 - [0:0]
:ACL_INBOUND_eth4 - [0:0]
:ACL_INBOUND_eth5 - [0:0]
:ACL_INBOUND_eth6 - [0:0]
-A FORWARD -s 10.32.22.96/27 -d 10.69.0.0/24 -j ACL_INBOUND_eth3
-A FORWARD -s 10.69.0.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.1.0/24 -j ACL_INBOUND_eth4
-A FORWARD -s 10.69.1.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.2.0/24 -j ACL_INBOUND_eth5
-A FORWARD -s 10.69.2.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.3.0/24 -j ACL_INBOUND_eth6
-A FORWARD -s 10.69.3.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -d 10.32.22.96/27 -o eth2 -j ACL_INBOUND_eth2
-A FORWARD -d 10.69.0.0/24 -o eth3 -j ACL_INBOUND_eth3
-A FORWARD -d 10.69.1.0/24 -o eth4 -j ACL_INBOUND_eth4
-A FORWARD -d 10.69.2.0/24 -o eth5 -j ACL_INBOUND_eth5
-A FORWARD -d 10.69.3.0/24 -o eth6 -j ACL_INBOUND_eth6
-A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth2 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth3 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth3 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth3 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 3000 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 4443 -j ACCEPT
-A ACL_INBOUND_eth3 -p udp -m udp --dport 10000 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 25 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 7999 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 7990 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8090 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8050 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8051 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.0.0.0/16 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth3 -j DROP
-A ACL_INBOUND_eth3 -j DROP
-A ACL_INBOUND_eth4 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth4 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth4 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 5000 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 3000 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8090 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 7990 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8050 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8051 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 7999 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 25 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.0.0.0/16 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.0.0.0/16 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth4 -j DROP
-A ACL_INBOUND_eth4 -j DROP
-A ACL_INBOUND_eth5 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth5 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth5 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth5 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.2.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth5 -j DROP
-A ACL_INBOUND_eth5 -j DROP
-A ACL_INBOUND_eth6 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth6 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth6 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 943 -j ACCEPT
-A ACL_INBOUND_eth6 -p udp -m udp --dport 1194 -j ACCEPT
-A ACL_INBOUND_eth6 -j DROP
-A ACL_INBOUND_eth6 -j DROP

outbound

# iptables-save | grep ACL_OUTBOUND
:ACL_OUTBOUND_eth3 - [0:0]
:ACL_OUTBOUND_eth4 - [0:0]
:ACL_OUTBOUND_eth5 - [0:0]
:ACL_OUTBOUND_eth6 - [0:0]
:ACL_OUTBOUND_eth2 - [0:0]
-A PREROUTING -s 10.69.0.0/24 ! -d 10.69.0.254/32 -i eth3 -m state --state NEW -j ACL_OUTBOUND_eth3
-A PREROUTING -s 10.69.1.0/24 ! -d 10.69.1.254/32 -i eth4 -m state --state NEW -j ACL_OUTBOUND_eth4
-A PREROUTING -s 10.69.2.0/24 ! -d 10.69.2.254/32 -i eth5 -m state --state NEW -j ACL_OUTBOUND_eth5
-A PREROUTING -s 10.69.3.0/24 ! -d 10.69.3.254/32 -i eth6 -m state --state NEW -j ACL_OUTBOUND_eth6
-A PREROUTING -s 10.32.22.96/27 ! -d 10.32.22.125/32 -i eth2 -m state --state NEW -j ACL_OUTBOUND_eth2
-A ACL_OUTBOUND_eth3 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth3 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth4 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth4 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth6 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth6 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth2 -d 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT

weizhouapache · 2021-09-16T09:46:52Z

@ravening can you add egress rules in ACL of vm tiers?

does vm have static nat IP ?

@weizhouapache

inbound

# iptables-save | grep ACL_INBOUND
:ACL_INBOUND_eth2 - [0:0]
:ACL_INBOUND_eth3 - [0:0]
:ACL_INBOUND_eth4 - [0:0]
:ACL_INBOUND_eth5 - [0:0]
:ACL_INBOUND_eth6 - [0:0]
-A FORWARD -s 10.32.22.96/27 -d 10.69.0.0/24 -j ACL_INBOUND_eth3
-A FORWARD -s 10.69.0.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.1.0/24 -j ACL_INBOUND_eth4
-A FORWARD -s 10.69.1.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.2.0/24 -j ACL_INBOUND_eth5
-A FORWARD -s 10.69.2.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.3.0/24 -j ACL_INBOUND_eth6
-A FORWARD -s 10.69.3.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -d 10.32.22.96/27 -o eth2 -j ACL_INBOUND_eth2
-A FORWARD -d 10.69.0.0/24 -o eth3 -j ACL_INBOUND_eth3
-A FORWARD -d 10.69.1.0/24 -o eth4 -j ACL_INBOUND_eth4
-A FORWARD -d 10.69.2.0/24 -o eth5 -j ACL_INBOUND_eth5
-A FORWARD -d 10.69.3.0/24 -o eth6 -j ACL_INBOUND_eth6
-A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth2 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth3 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth3 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth3 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 3000 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 4443 -j ACCEPT
-A ACL_INBOUND_eth3 -p udp -m udp --dport 10000 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 25 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 7999 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 7990 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8090 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8050 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8051 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.0.0.0/16 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth3 -j DROP
-A ACL_INBOUND_eth3 -j DROP
-A ACL_INBOUND_eth4 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth4 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth4 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 5000 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 3000 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8090 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 7990 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8050 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8051 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 7999 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 25 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.0.0.0/16 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.0.0.0/16 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth4 -j DROP
-A ACL_INBOUND_eth4 -j DROP
-A ACL_INBOUND_eth5 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth5 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth5 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth5 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.2.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth5 -j DROP
-A ACL_INBOUND_eth5 -j DROP
-A ACL_INBOUND_eth6 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth6 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth6 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 943 -j ACCEPT
-A ACL_INBOUND_eth6 -p udp -m udp --dport 1194 -j ACCEPT
-A ACL_INBOUND_eth6 -j DROP
-A ACL_INBOUND_eth6 -j DROP

outbound

# iptables-save | grep ACL_OUTBOUND
:ACL_OUTBOUND_eth3 - [0:0]
:ACL_OUTBOUND_eth4 - [0:0]
:ACL_OUTBOUND_eth5 - [0:0]
:ACL_OUTBOUND_eth6 - [0:0]
:ACL_OUTBOUND_eth2 - [0:0]
-A PREROUTING -s 10.69.0.0/24 ! -d 10.69.0.254/32 -i eth3 -m state --state NEW -j ACL_OUTBOUND_eth3
-A PREROUTING -s 10.69.1.0/24 ! -d 10.69.1.254/32 -i eth4 -m state --state NEW -j ACL_OUTBOUND_eth4
-A PREROUTING -s 10.69.2.0/24 ! -d 10.69.2.254/32 -i eth5 -m state --state NEW -j ACL_OUTBOUND_eth5
-A PREROUTING -s 10.69.3.0/24 ! -d 10.69.3.254/32 -i eth6 -m state --state NEW -j ACL_OUTBOUND_eth6
-A PREROUTING -s 10.32.22.96/27 ! -d 10.32.22.125/32 -i eth2 -m state --state NEW -j ACL_OUTBOUND_eth2
-A ACL_OUTBOUND_eth3 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth3 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth4 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth4 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth6 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth6 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth2 -d 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT

ravening · 2021-09-16T09:47:10Z

his is my current setup now
private gateway ends with .125 and public ip ends with .126
I reverted the code change
ping/ssh from vm to dedicated server works fine but ping from dedicated server to internet doesnt work
should I make both private gateway and public ip same and add the acl rules?
@ravening yes. if ping works (from vm to dedicated server and dedicated server to internet), VPC VR should be ok, then you need to check ACL rules.
@weizhouapache This is the test I did
made both private gateway and public ip end with .125
reverted the code changes
added the 3 cidr list you provided to the network acl
result:
ping from dedicated server to internet works
ssh from vm to dedicated server DOES NOT work. ping works though

@weizhouapache

after this changes, even the ssh from dedicated server to cloudstack vm also stopped

ravening · 2021-09-16T09:58:11Z

@ravening can you add egress rules in ACL of vm tiers?

does vm have static nat IP ?

@weizhouapache
inbound

# iptables-save | grep ACL_INBOUND
:ACL_INBOUND_eth2 - [0:0]
:ACL_INBOUND_eth3 - [0:0]
:ACL_INBOUND_eth4 - [0:0]
:ACL_INBOUND_eth5 - [0:0]
:ACL_INBOUND_eth6 - [0:0]
-A FORWARD -s 10.32.22.96/27 -d 10.69.0.0/24 -j ACL_INBOUND_eth3
-A FORWARD -s 10.69.0.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.1.0/24 -j ACL_INBOUND_eth4
-A FORWARD -s 10.69.1.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.2.0/24 -j ACL_INBOUND_eth5
-A FORWARD -s 10.69.2.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -s 10.32.22.96/27 -d 10.69.3.0/24 -j ACL_INBOUND_eth6
-A FORWARD -s 10.69.3.0/24 -d 10.32.22.96/27 -j ACL_INBOUND_eth2
-A FORWARD -d 10.32.22.96/27 -o eth2 -j ACL_INBOUND_eth2
-A FORWARD -d 10.69.0.0/24 -o eth3 -j ACL_INBOUND_eth3
-A FORWARD -d 10.69.1.0/24 -o eth4 -j ACL_INBOUND_eth4
-A FORWARD -d 10.69.2.0/24 -o eth5 -j ACL_INBOUND_eth5
-A FORWARD -d 10.69.3.0/24 -o eth6 -j ACL_INBOUND_eth6
-A ACL_INBOUND_eth2 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth2 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.1.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.69.2.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth2 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth2 -j DROP
-A ACL_INBOUND_eth3 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth3 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth3 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 3000 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 4443 -j ACCEPT
-A ACL_INBOUND_eth3 -p udp -m udp --dport 10000 -j ACCEPT
-A ACL_INBOUND_eth3 -p tcp -m tcp --dport 25 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 7999 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 7990 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8090 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8050 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.69.3.0/24 -p tcp -m tcp --dport 8051 -j ACCEPT
-A ACL_INBOUND_eth3 -s 10.0.0.0/16 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth3 -j DROP
-A ACL_INBOUND_eth3 -j DROP
-A ACL_INBOUND_eth4 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth4 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth4 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.3.0/24 -p tcp -m tcp --dport 5000 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 3000 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8090 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 7990 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8050 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 8051 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 7999 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.69.0.0/16 -p tcp -m tcp --dport 25 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.0.0.0/16 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth4 -s 10.0.0.0/16 -p tcp -m tcp --dport 8095 -j ACCEPT
-A ACL_INBOUND_eth4 -j DROP
-A ACL_INBOUND_eth4 -j DROP
-A ACL_INBOUND_eth5 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth5 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth5 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth5 -s 66.55.152.67/32 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 3389 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 9090 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 145 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 5900:5910 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.3.0/24 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 139 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 445 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 6379 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 10933 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 111 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 2049 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 3306 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 873 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 8086 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 1433 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22002 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 9000 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.0.0.0/16 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.32.22.96/27 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.2.0/24 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.0.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.1.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.69.2.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth5 -s 52.2.55.172/32 -p tcp -m tcp --dport 22001 -j ACCEPT
-A ACL_INBOUND_eth5 -j DROP
-A ACL_INBOUND_eth5 -j DROP
-A ACL_INBOUND_eth6 -d 225.0.0.50/32 -j ACCEPT
-A ACL_INBOUND_eth6 -d 224.0.0.18/32 -j ACCEPT
-A ACL_INBOUND_eth6 -p icmp -m icmp --icmp-type any -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 80 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 443 -j ACCEPT
-A ACL_INBOUND_eth6 -p tcp -m tcp --dport 943 -j ACCEPT
-A ACL_INBOUND_eth6 -p udp -m udp --dport 1194 -j ACCEPT
-A ACL_INBOUND_eth6 -j DROP
-A ACL_INBOUND_eth6 -j DROP

outbound

# iptables-save | grep ACL_OUTBOUND
:ACL_OUTBOUND_eth3 - [0:0]
:ACL_OUTBOUND_eth4 - [0:0]
:ACL_OUTBOUND_eth5 - [0:0]
:ACL_OUTBOUND_eth6 - [0:0]
:ACL_OUTBOUND_eth2 - [0:0]
-A PREROUTING -s 10.69.0.0/24 ! -d 10.69.0.254/32 -i eth3 -m state --state NEW -j ACL_OUTBOUND_eth3
-A PREROUTING -s 10.69.1.0/24 ! -d 10.69.1.254/32 -i eth4 -m state --state NEW -j ACL_OUTBOUND_eth4
-A PREROUTING -s 10.69.2.0/24 ! -d 10.69.2.254/32 -i eth5 -m state --state NEW -j ACL_OUTBOUND_eth5
-A PREROUTING -s 10.69.3.0/24 ! -d 10.69.3.254/32 -i eth6 -m state --state NEW -j ACL_OUTBOUND_eth6
-A PREROUTING -s 10.32.22.96/27 ! -d 10.32.22.125/32 -i eth2 -m state --state NEW -j ACL_OUTBOUND_eth2
-A ACL_OUTBOUND_eth3 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth3 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth4 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth4 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT
-A ACL_OUTBOUND_eth5 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth6 -d 224.0.0.18/32 -j ACCEPT
-A ACL_OUTBOUND_eth6 -d 225.0.0.50/32 -j ACCEPT
-A ACL_OUTBOUND_eth2 -d 10.32.22.0/24 -p tcp -m tcp --dport 22 -j ACCEPT

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1

weizhouapache · 2021-09-16T10:02:20Z

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1

@ravening can you ping/ssh from a vm without static nat ?

ravening · 2021-09-16T10:04:46Z

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1
@ravening can you ping/ssh from a vm without static nat ?

@weizhouapache unfortunately I dont have access to the vm without static nat

weizhouapache · 2021-09-16T10:22:15Z

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1
@ravening can you ping/ssh from a vm without static nat ?
@weizhouapache unfortunately I dont have access to the vm without static nat

@ravening it would be good to create a test vm in same vpc tier.

might be related to #4484

ravening · 2021-09-21T10:04:41Z

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1
@ravening can you ping/ssh from a vm without static nat ?

@weizhouapache yes i can ssh/ping from a vm without a static nat

weizhouapache · 2021-09-21T11:26:52Z

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1
@ravening can you ping/ssh from a vm without static nat ?
@weizhouapache yes i can ssh/ping from a vm without a static nat

@ravening
I cannot reproduce the issue in 4.16.
can you check if you have this commit ?
31bd821

ravening · 2021-09-22T07:47:18Z

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1
@ravening can you ping/ssh from a vm without static nat ?
@weizhouapache yes i can ssh/ping from a vm without a static nat
@ravening
I cannot reproduce the issue in 4.16.
can you check if you have this commit ?
31bd821

@weizhouapache yes we already have this code in our branch

ravening · 2021-09-22T08:18:08Z

@weizhouapache yes it has the static nat ending with 34 and its internal ip address is 10.69.3.1
@ravening can you ping/ssh from a vm without static nat ?
@weizhouapache yes i can ssh/ping from a vm without a static nat
@ravening
I cannot reproduce the issue in 4.16.
can you check if you have this commit ?
31bd821
@weizhouapache yes we already have this code in our branch

@weizhouapache we tested the same scenario in our test setup and the ping/ssh worked fine after applying this code change. without this change, it didnt work

weizhouapache · 2021-09-22T08:30:44Z

we tested the same scenario in our test setup and the ping/ssh worked fine after applying this code change. without this change, it didnt work

@ravening
is private gateway IP same as gateway ?
if yes, the iptables rules should be same (with this PR and without this PR). I suspect it might be caused by the order of iptables rules.

ravening · 2021-09-22T08:39:22Z

we tested the same scenario in our test setup and the ping/ssh worked fine after applying this code change. without this change, it didnt work
@ravening
is private gateway IP same as gateway ?
if yes, the iptables rules should be same (with this PR and without this PR). I suspect it might be caused by the order of iptables rules.

@weizhouapache no, the gateway and public IP are different.

weizhouapache · 2021-09-22T10:21:43Z

we tested the same scenario in our test setup and the ping/ssh worked fine after applying this code change. without this change, it didnt work
@ravening
is private gateway IP same as gateway ?
if yes, the iptables rules should be same (with this PR and without this PR). I suspect it might be caused by the order of iptables rules.
@weizhouapache no, the gateway and public IP are different.

@ravening
(1) I think the rules can only be added if Ip is same as gateway (which means VPC VR is gateway of dedicated servers).
(2) Please add an inbound rule in the Network ACL of private gateway:
source IP = NAT IP of the vm

ravening · 2021-09-22T10:27:49Z

we tested the same scenario in our test setup and the ping/ssh worked fine after applying this code change. without this change, it didnt work
@ravening
is private gateway IP same as gateway ?
if yes, the iptables rules should be same (with this PR and without this PR). I suspect it might be caused by the order of iptables rules.
@weizhouapache no, the gateway and public IP are different.
@ravening
(1) I think the rules can only be added if Ip is same as gateway (which means VPC VR is gateway of dedicated servers).

yes these rules are applied only when public ip and gateway are same.... but in our case they are different.
Should they be same or different?

(2) Please add an inbound rule in the Network ACL of private gateway:
source IP = NAT IP of the vm

we have added all ingress and egress but still no ping

@weizhouapache

weizhouapache · 2021-09-22T10:36:38Z

yes these rules are applied only when public ip and gateway are same.... but in our case they are different.

@ravening
I personally think you can the code changes in this PR in your fork.
but this is not the final solution in my opinion.
There might be a problem with firewall rules but I was not able to reproduce the issue.

ravening · 2021-09-22T10:42:59Z

yes these rules are applied only when public ip and gateway are same.... but in our case they are different.
@ravening
I personally think you can the code changes in this PR in your fork.
but this is not the final solution in my opinion.
There might be a problem with firewall rules but I was not able to reproduce the issue.

@weizhouapache ok. thanks for the help... will see what else we can find out

weizhouapache · 2021-09-22T10:45:05Z

yes these rules are applied only when public ip and gateway are same.... but in our case they are different.
@ravening
I personally think you can the code changes in this PR in your fork.
but this is not the final solution in my opinion.
There might be a problem with firewall rules but I was not able to reproduce the issue.

@ravening
for your information, here is what I have tested
(1) create a shared network (any test Ip range), for example 10.10.10.254/24. create a vm.
(2) create a vpc, a vpc tier, and a vm
(3) add private gateway, vlan is same as shared network in (1), IP is same as gateway (10.10.10.254).
(4) verify the connection between the vm in vpc and the vm in shared network.

ravening · 2021-09-22T12:13:56Z

yes these rules are applied only when public ip and gateway are same.... but in our case they are different.
@ravening
I personally think you can the code changes in this PR in your fork.
but this is not the final solution in my opinion.
There might be a problem with firewall rules but I was not able to reproduce the issue.
@ravening
for your information, here is what I have tested
(1) create a shared network (any test Ip range), for example 10.10.10.254/24. create a vm.
(2) create a vpc, a vpc tier, and a vm
(3) add private gateway, vlan is same as shared network in (1), IP is same as gateway (10.10.10.254).
(4) verify the connection between the vm in vpc and the vm in shared network.

@weizhouapache we found out the issue. the rule has to be added the beginning by passing "front" in the command.
In that case we dont need to put this rule outside of the if condition.

so this is what we tested

make private gateway and public ip as same,
pass "front" parameter to iptable rule which is present inside the if condition.
everything works fine.

i will make the code change and update the pr

When private gateway is configured, the external server which are connected through private gateway cannot send traffic to outside world. Add a source iptable rule so that the servers which are connected through private gateway can ping the public IP's

ravening · 2021-09-22T13:03:27Z

@weizhouapache @rhtyd updated code

weizhouapache · 2021-09-22T13:11:10Z

 (self.address["network"], address["network"], address["device"])])
 # Accept packet from private gateway if VPC VR is used as gateway
- self.fw.append(["filter", "", "-A FORWARD -s %s ! -d %s -j ACCEPT" %
+ self.fw.append(["filter", "front", "-A FORWARD -s %s ! -d %s -j ACCEPT" %


@ravening
can you move this to line 497 and test it ?

@weizhouapache without the front keyword?

@ravening
without front might not fix your problem. try with front

@weizhouapache still works... the rule is still being added at the beginning of the FORWARD chain

@ravening before 'ACL_INBOUND_ethX' ?

@weizhouapache yes
# iptables -L FORWARD --line-numbers Chain FORWARD (policy DROP) num target prot opt source destination 1 ACCEPT all -- 10.32.18.192/27 !10.32.18.192/27 2 ACL_INBOUND_eth3 all -- 10.32.18.192/27 10.0.0.0/24 3 ACL_INBOUND_eth2 all -- 10.0.0.0/24 10.32.18.192/27 4 ACL_INBOUND_eth4 all -- 10.32.18.192/27 10.0.1.0/24 5 ACL_INBOUND_eth2 all -- 10.0.1.0/24 10.32.18.192/27

@ravening
as you see in the rules, rule 2 and 4 will always be ignored.
can you send me the whole iptables rules by email (no matter if this PR is applied) ?

weizhouapache · 2021-09-28T14:53:40Z

@ravening could you please confirm if this can be closed ?

ravening · 2021-09-29T07:51:28Z

@ravening could you please confirm if this can be closed ?

@weizhouapache closing this as we need to configure extra firewall rules when compares to regular cases

ravening closed this Sep 1, 2021

ravening reopened this Sep 1, 2021

ravening force-pushed the private-gateway-ping branch from 25c5769 to cfbe1baCompare September 1, 2021 23:33

yadvr changed the base branch from 4.15 to mainSeptember 20, 2021 07:23

ravening force-pushed the private-gateway-ping branch from cfbe1ba to 685d299Compare September 22, 2021 13:03

weizhouapache reviewed Sep 22, 2021
View reviewed changes

ravening closed this Sep 29, 2021

Uh oh!

Conversation

ravening commented Sep 1, 2021

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

weizhouapache commented Sep 1, 2021

Uh oh!

ravening commented Sep 1, 2021

Uh oh!

weizhouapache commented Sep 1, 2021

Uh oh!

ravening commented Sep 1, 2021

Uh oh!

ravening commented Sep 1, 2021

Uh oh!

weizhouapache commented Sep 2, 2021

Uh oh!

ravening commented Sep 2, 2021

Uh oh!

weizhouapache commented Sep 2, 2021

Uh oh!

ravening commented Sep 2, 2021

Uh oh!

weizhouapache commented Sep 2, 2021

Uh oh!

ravening commented Sep 2, 2021

Uh oh!

ravening commented Sep 2, 2021

Uh oh!

weizhouapache commented Sep 2, 2021

Uh oh!

ravening commented Sep 2, 2021

Uh oh!

weizhouapache commented Sep 2, 2021

Uh oh!

ravening commented Sep 2, 2021

Uh oh!

yadvr commented Sep 7, 2021

Uh oh!

weizhouapache commented Sep 10, 2021

Uh oh!

weizhouapache commented Sep 13, 2021

Uh oh!

ravening commented Sep 14, 2021

Uh oh!

weizhouapache commented Sep 14, 2021

Uh oh!

ravening commented Sep 14, 2021

Uh oh!

weizhouapache commented Sep 14, 2021

Uh oh!

ravening commented Sep 14, 2021

Uh oh!

weizhouapache commented Sep 14, 2021

Uh oh!

ravening commented Sep 14, 2021

Uh oh!

weizhouapache commented Sep 14, 2021

Uh oh!

weizhouapache commented Sep 16, 2021

Uh oh!

ravening commented Sep 16, 2021

Uh oh!

weizhouapache commented Sep 16, 2021

Uh oh!

ravening commented Sep 16, 2021

Uh oh!

weizhouapache commented Sep 16, 2021

Uh oh!

ravening commented Sep 16, 2021

Uh oh!

ravening commented Sep 16, 2021• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ravening commented Sep 16, 2021•
edited
Loading

weizhouapache commented Sep 16, 2021•
edited
Loading