SNAT for multiple networks on different IPs

[ivoks]

Hello

I've read multiple posts on how to do this, but none of them seem to work or I'm missing something obvious. Therefore, pardon me if I'm repeating a topic that has been covered already.

I'm attaching a diagram of what the network looks like.

L3 switch is doing the routing between three networks and Internet. Three networks are getting to it, it routes and then what needs to go outside, goes over middle router (used to be Ubuntu, now is EdgeRouter). Middle router only has the bridge, so the traffic, from client perspective, only gets to end router (192.0.0.254). But while passing through middle router, different subnets get NATed to different public IPs.

While I had Linux machine (Ubuntu 16.04) running NAT I only needed these three iptables rules for NAT:

-A POSTROUTING -s 10.1.0.0/16 -j SNAT --to-source 192.0.0.1

-A POSTROUTING -s 10.2.0.0/21 -j SNAT --to-source 192.0.0.2
-A POSTROUTING -s 10.3.0.0/21 -j SNAT --to-source 192.0.0.3

With EdgeRouter, I can succesfully do most of what I did before. I've created the bridge, I've even managed to get traffic from 10.1.0.0/16 to be NATed to 192.0.0.1, 10.2.0.0/21 to 192.0.0.2 and 10.3.0.0/21 to 192.0.0.3. I can see traffic (icmp) generated from 10.3.0.0/16 leaving the EdgeRouter as 192.0.0.3, I can see target responding to 192.0.0.3 and that traffic coming back to EdgeRouter. But, it seems that EdgeRouter never passes it back to where it originated from. I'm not sure what I'm doing wrong.

Here's the relevant part of configuration:

interfaces {
bridge br0 {
address 192.0.0.1/24
address 192.0.0.2/24
address 192.0.0.3/24
aging 300
bridged-conntrack disable
hello-time 2
max-age 20
priority 0
promiscuous disable
stp false
}

service {

nat {

rule 5003 {
description 10.2.0.0/21
disable
log disable
outbound-interface br0
outside-address {
address 192.0.0.2
}
protocol all
source {
address 10.2.0.0/21
}
type source
}

rule 5004 {
description 10.3.0.0/21
disable
log disable
outbound-interface br0
outside-address {
address 192.0.0.3
}
protocol all
source {
address 10.3.0.0/21
}
type source
}

rule 5005 {
description NAT
log disable
outbound-interface br0
outside-address {
}
protocol all
source {
group {
address 10.0.0.0/8
}
}
type masquerade
}
}

Am I missing something? This was very easy on standard Linux machine, but I've lost two days on this one and I'm not seeing what I'm doing wrong. I know the network is not ideal, but I can't remove that last router on the right end, and I can't do NAT on the switch.

Thank you

[16again · Accepted solution]

1st of all, change the subnet between L3 switch and ER, so the ER can do its routing, not bridging.

Bridging disables offload, and its speed advantage.

Assign the 192.0.0.1  , 2 and.3 address to WAN side, and add 3 sNAT rules, filtering on source network.

[16again · Accepted solution]

1st of all, change the subnet between L3 switch and ER, so the ER can do its routing, not bridging.

Bridging disables offload, and its speed advantage.

Assign the 192.0.0.1  , 2 and.3 address to WAN side, and add 3 sNAT rules, filtering on source network.

[ivoks]

You are right, I know this is a mess. Problem is that I cannot influence the right most router, but I do control rest of the public subnet.

Thank you for the advice, it made me rethink all this and I plan to reorganize whole network in next couple of days. That should simplify this diagram and resolve the problem.

[eJc2_]

PBR

https://help.ubnt.com/hc/en-us/articles/204952274-EdgeRouter-Policy-based-routing-source-address-bas...

[16again]

You don't need PBR, as all LAN traffic needs to be routed to a single  WAN link.   Use PBR when using multiple WANs.

[ivoks]

I had exactly the same problem when I moved to routing-only approach. Turns out that the top right router (the one I can't do anything about) has a long arp cache. Very long. I didn't notice that in the begining, but it was sending packets to the wrong mac address, and of course, ER was just dropping that.

In any case, I have now moved to setup where L3 switch does no routing, all internal traffic is on private subnets, and ER does SNAT/DNAT/DHCP. All servers (a dozen of them) and some client required IP changes, but at least the setup is clean now. I was playing with an idea of keeping some routing on the L3 switch, but then it's just a problem to maintain routing and firewalling in two different places.

Time will tell how performant new setup is; 400+ clients behind the firewall and peek network traffic on Saturday was 700mbit/s. Thanks for all the help.