Reply
New Member
Posts: 4
Registered: ‎01-13-2016

USG-PRO-4 - Dual Wan Port Question

The USG PRO 4 Has Dual WAN Ports 

 

Is this for Fail Safe / Roll Over? Or Can i setup Link / Load Balancing Between the two ISP's used simultaneously?

 

Thanks

 

Aaron

Established Member
Posts: 1,998
Registered: ‎04-26-2014
Kudos: 838
Solutions: 23

Re: USG-PRO-4 - Dual Wan Port Question

[ Edited ]

If you want to use the GUI the last time I checked it's set for fail over only. If you want to setup the unit via CLI and create a json file then you can do load balance. 

 

Stand corrected.. Time to turn up my second connection and test this.

When you receive a solution to your question/issue, don't forget to mark your thread as solved and to give kudos to the people who have helped you out!
Established Member
Posts: 1,480
Registered: ‎04-22-2016
Kudos: 323
Solutions: 119

Re: USG-PRO-4 - Dual Wan Port Question

@HDMSIT

WAN1 to WAN 2 failover:

With the latest unstable 5.6.7 release, failover in my case is working OK.

 

Load Balancing with two ISP's

With 5.6.7 it can be done also in the GUI. If you search for it, users have reported there it's still an issue about the load balancing.

 

To get the beta releases, you need to sign up for it.

Please reward people who have helped you with kudo's and mark your thread as solved when you receive a solution to your issue.
Ubiquiti Employee
Posts: 1,226
Registered: ‎02-28-2017
Kudos: 366
Solutions: 121

Re: USG-PRO-4 - Dual Wan Port Question

In stable versions, you can configure a weighted load balance in the GUI and configure the weight % for each WAN, or you can configure WAN2 for fail-over only.
Brandon Jaffe | UniFi Routing & Switching | Austin, TX
New Member
Posts: 4
Registered: ‎01-13-2016

Re: USG-PRO-4 - Dual Wan Port Question

With this Load Balancing feature will my customer see the true benefit?

 

Our end goal is this - essentially having two High Speed Charter Connections (200mb each) tieing both those to the USG PRO 4 - then out to several UniFi Managed switches and Wifi 

 

With your load balancing feature - will we see the the 400mb of throughput im looking for - or is this more for directing traffic (example - server #1 and #2 can be tied to ISP 1) and then all the employees clients then can have ISP 2)

 

This customer does a ton of Video and Graphics productions are we are just looking into combining a couple ISP provides to get them the largest amount of download speed, (fiber isnt possible in this area, so we are limited to this route)

 

 

Thanks again,

 

Aaron

Ubiquiti Employee
Posts: 1,226
Registered: ‎02-28-2017
Kudos: 366
Solutions: 121

Re: USG-PRO-4 - Dual Wan Port Question

Yeah you can put them each at 50% weight since they're the same speed and you'll see 400 Mb/s total throughput if you're pushing that much traffic. However, you might not see 400 Mb/s on a single download because multiWAN doesn't split single connections across WANs.

It's usually better to get different ISPs for multiWANs, because if one goes down, the other is still up. Also, you'll want to make sure both of your connections are in different subnets (different gateway IPs).

There's some best practices for multi-wan listed here: https://community.ubnt.com/t5/EdgeMAX/multi-WAN-for-beginner/m-p/1169500#M55558
Brandon Jaffe | UniFi Routing & Switching | Austin, TX
Regular Member
Posts: 394
Registered: ‎05-24-2017
Kudos: 145
Solutions: 11

Re: USG-PRO-4 - Dual Wan Port Question

@UBNT-jaffe @UBNT-David

 

I'm thinking of using 2 Spectrum internet connections that are at a camping resort to get load balancing and more internet bandwidth.  Being the two modems are on the same portion of the cable plant, it's possible they will have the same default gateway.

 

Do you know if with the USG-PRO-4 with dual wan load balanced setup (50% for each since they match speeds) if the two WAN connections point to the same default gateway at the ISP is this still a problem?  Is there any hope to making it work if they are on the same cable subnet.

 

There is a decent chance though that they will be on different subnets.  The cable company has so many subnets and these two connections were brought up at totally different times -- one has been online for a while an the other just recently added.  So there is hope.  I don't have the ability to check at the moment an wanted to research before we dive in!

 

Thanks

 

Ubiquiti Employee
Posts: 1,226
Registered: ‎02-28-2017
Kudos: 366
Solutions: 121

Re: USG-PRO-4 - Dual Wan Port Question

Yes it's still a restriction, each WAN interface must have a unique gateway IP.

Brandon Jaffe | UniFi Routing & Switching | Austin, TX
Regular Member
Posts: 394
Registered: ‎05-24-2017
Kudos: 145
Solutions: 11

Re: USG-PRO-4 - Dual Wan Port Question

I put the USG-PRO-4 in place last week (w/ upgraded quieter fans since it's in the owner's office Man Happy and it's working well.  Smart Queues taking care of the 100x12 link.

 

I confirmed each modem is on a completely different public IP subnet w/ different gateways.

So this is an option for us and we may give it a shot if we need the capacity.

 

Plan to use some nanostations or nanobeams to connect the two buildings and I'll do a VLAN only network on the UniFi switch where the gateway is and then put another Unifi switch at the other end of the nano pair and bring out that VLAN on one of the ports (untagged native network) and plug into the 2nd modem.  Same thing on the switch near the gateway and plug into WAN2.  Then the gateway has 2 x cable modems on different public IP subnets (thus different gateways) and we can do a 50/50 load balance.

 

Very interested in trying this out now.

 

Thanks!

 

Member
Posts: 288
Registered: ‎06-16-2017
Kudos: 189
Solutions: 6

Re: USG-PRO-4 - Dual Wan Port Question

It is interesting to read a member of the UBNT staff encourage you trying multi wan with load balance given that it is well known that multi wan implementation on the USG is quite buggy (my experience says it is completely broken but there are some people saying it works for them).

 

So, if you find yourself banging your head against the wall after enabling multi wan, remember that it does not work for many of us and therefore may not be your fault.

Regular Member
Posts: 394
Registered: ‎05-24-2017
Kudos: 145
Solutions: 11

Re: USG-PRO-4 - Dual Wan Port Question

@mbello can you share whatever woes you went through?

 

Have you tried it with 4.4.X gateway firmware?  Last year they were in the 4.3's for a while and then 4.4 series got stable and has been working really well for me on several USG and USG-PRO-4

 

I certainly don't want to use it if it's going to create all kinds of problems.

 

I'm looking for 2 x internet connections with totally different public subnets and load balance by percentage likely 50/50 or 60/40 or similar.  Would be nice to know if the dashboard shows what's going on and health of both wan connections.

 

I would be fine w/ CLI or GUI to get more details such as NAT'd connections that are directed to one WAN or the other to get an idea which is using which and importantly to know if either connection is getting saturated

 

Also the smart queues would be important on each through at least 100 Mbps.

 

If there are threads you could point to that would help too.  Thanks for chiming in.

 

Member
Posts: 288
Registered: ‎06-16-2017
Kudos: 189
Solutions: 6

Re: USG-PRO-4 - Dual Wan Port Question

@firefi I have tested with all versions you can think of and currently on 4.4.22 and 5.9.4. I have tried 4.5.1, 4.5.2 and 4.5.3 also on the USG.

 

Copied from another thread my summary of the multi wan woes I have experienced:

 

 

"multiwan has a ridiculous number of issues, including:

1. WAN1 can failover to WAN2 but WAN2 won't always failover to WAN1;

2. Traffic originating at the USG under multiwan will often see packet losses of > 50% because somehow the USG at some point starts sending packets to WAN2 with source ip of WAN1 and vice-versa;

3. S2S VPN will eventually break if you have multiwan on either side (probbly as a consequence of (1 and 2 above);

4. USG does not know that the DNS servers it gets from the ISP dhcp server on WAN1/2 should only be reached through WAN1/2. It actually consider all DNS servers to be reachable from both WAN interfaces which is often - but not always - true.

5. Load balance - besides the many routing problems it gets you - does not seem to actually balance the load;

6. Port forward under multi wan scenario will work EITHER on WAN1 OR on WAN2 but not on both.

 

All of these issues exist since August last year which is when I first purchased Unify equipment. As far as I know, none of these issues have been fixed so I have WAN2 on ALL of my sites disabled for multiple months now."

 

A few threads showing people struggling with multi wan:

https://community.ubnt.com/t5/UniFi-Routing-Switching/Load-balancing-seems-not-to-work/m-p/2244740/h...

https://community.ubnt.com/t5/UniFi-Routing-Switching/WAN-Failover-Not-going-back-to-WAN1/td-p/22451...

https://community.ubnt.com/t5/UniFi-Routing-Switching/USG-WAN-failover-route-wonkiness-after-failbac...

https://community.ubnt.com/t5/UniFi-Routing-Switching-Beta/DNS-forwarding-not-responding-to-requests...

https://community.ubnt.com/t5/UniFi-Routing-Switching-Beta/USG-PRO-4-keeps-disconnecting/td-p/213038...

https://community.ubnt.com/t5/UniFi-Routing-Switching/USG-3-Load-Balance-using-a-L3-Controller-showi...

https://community.ubnt.com/t5/UniFi-Routing-Switching/VPN-Site-to-Site-Issue/m-p/2264914#M79770

https://community.ubnt.com/t5/UniFi-Routing-Switching/VPN-vs-dual-WAN/m-p/2264892#M79767

https://community.ubnt.com/t5/UniFi-Routing-Switching/Dual-WAN-eth2-and-pppoe1-and-routes-blackholed...

https://community.ubnt.com/t5/UniFi-Routing-Switching-Beta/L2TP-VPN-not-working-with-WAN2-enabled-fo...

https://community.ubnt.com/t5/UniFi-Routing-Switching-Beta/site-to-site-VPN-and-weighted-WAN2-proble...

 

Regular Member
Posts: 394
Registered: ‎05-24-2017
Kudos: 145
Solutions: 11

Re: USG-PRO-4 - Dual Wan Port Question

[ Edited ]

OUCH!

 

Thanks for letting me know.  Wow.

 

@UBNT-jaffe any idea where the issues mentioned currently stand?

 

Ubiquiti Employee
Posts: 1,226
Registered: ‎02-28-2017
Kudos: 366
Solutions: 121

Re: USG-PRO-4 - Dual Wan Port Question

1. WAN1 can failover to WAN2 but WAN2 won't always failover to WAN1;
This did regress somewhere between 4.4.12-4.4.21, but I've never had this issue on 4.4.22+

2. Traffic originating at the USG under multiwan will often see packet losses of > 50% because somehow the USG at some point starts sending packets to WAN2 with source ip of WAN1 and vice-versa;

This was fixed a while ago in version 5.7.8:

-"Add NAT of other WAN IPs out of WANs, fixing several issues related to multi-WAN."

It provisions 2 NAT rules with a similar description to "'MASQ eth0 out other WAN'"

 

3. S2S VPN will eventually break if you have multiwan on either side (probbly as a consequence of (1 and 2 above);

More likely a consequence of not having DPD and traffic isn't being sent across 24/7. Auto VPNs now provision DPD in the latest controller versions, and DPD will be configurable in manual IPsec VPNs in a future controller version.

4. USG does not know that the DNS servers it gets from the ISP dhcp server on WAN1/2 should only be reached through WAN1/2. It actually consider all DNS servers to be reachable from both WAN interfaces which is often - but not always - true.

dnsmasq will forward queries out to all configured DNS servers, if WAN2 can't reach a DNS server only reachable by WAN1 (and WAN1 is down), it'll hop to the next one in the list in /etc/resolv.conf. 

 

5. Load balance - besides the many routing problems it gets you - does not seem to actually balance the load;

Sure it does, I've taken plenty of stats/pcaps over the past year to prove this. The WAN_IN / WAN_OUT / LOCAL_OUT counters on "show load-balance status" are accurate and can be proved so with tcpdumps / pcaps. 
"Sticky" connections will prefer one WAN over another for single connections to prevent tcp retransmissions etc... so you might not see a perfect 50/50 load, but that's because one single connection stream can't be split across two separate links (physical NICs).

6. Port forward under multi wan scenario will work EITHER on WAN1 OR on WAN2 but not on both.

This is true in the GUI, however you can set custom DNAT rules via config.gateway.json to have both WAN1/WAN2 port forwards working simultaneously. We also have more granular SNAT/DNAT controls coming to the GUI which will tackle this.

 

One of the threads you linked is actually marked solved, and on most of the others, @UBNT-cmb has addressed.

Brandon Jaffe | UniFi Routing & Switching | Austin, TX
Member
Posts: 288
Registered: ‎06-16-2017
Kudos: 189
Solutions: 6

Re: USG-PRO-4 - Dual Wan Port Question

[ Edited ]

@UBNT-jaffe So are you saying plain and simple that multi wan on the USG is stable and fully supported on both failover and loadbalance? Even with auto S2S VPN on?

Does @UBNT-cmb share the same opinion?

 

 

Regular Member
Posts: 394
Registered: ‎05-24-2017
Kudos: 145
Solutions: 11

Re: USG-PRO-4 - Dual Wan Port Question

I like to use the PPTP or L2TP VPN in the USG as an access method so I can login to my airmax radios remotely since that can't be done via cloud access.

 

Part of making that work is having Dynamic DNS update an entry on my DNS provider (EasyDNS) so I can easily access the remote USG via public DNS name even though the USG has an ISP supplied dynamic public IP.

 

On Dual Wan -- how is this handled?  Will Dynamic DNS only work for WAN1 or is there a way to have Dynamic DNS update on a per WAN basis two different DNS entries?

 

Without some kind of support it will be difficult to VPN to a remote USG site.  

 

As far as devices behind the USG doing VPN that isn't as critical.

 

Site to site VPN I've used to allow a customer who has two sites access their controller at their remote site as if it's local.  The OpenVPN has worked really well for this.  Wondering what implications dual WAN has on site to site VPN also.  

 

I don't use any type of IPsec VPN at this time -- so no problem there.

 

Thanks.

 

Emerging Member
Posts: 51
Registered: ‎06-27-2017
Kudos: 9

Re: USG-PRO-4 - Dual Wan Port Question


@mbello wrote:

@UBNT-jaffe So are you saying plain and simple that multi wan on the USG is stable and fully supported on both failover and loadbalance? Even with auto S2S VPN on?

Does @UBNT-cmb share the same opinion?

 

 


Did anyone ever respond to you?

New Member
Posts: 1
Registered: ‎12-07-2016

Re: USG-PRO-4 - Dual Wan Port Question

[ Edited ]

Hi @UBNT-jaffe , 

 

Is this still an issue? i have 2  connections from the same carrier so the gateway IP is the same on both WAN's.

 

Is this a enhancement thing or a permenant limitation- for the time being to get it to work (which it works well) i have setup a Pfsense VM with a WAN2 vlan just to get a different Gateway IP in front of the second Wan port on the USG Pro

 

Without the PfSense in front of WAN2 mine just selects one of the WAN's the runs all the traffic through it:

show load-balance watchdog

Group wan_failover
  eth2
  status: Running
  pings: 142
  fails: 0
  run fails: 0/3
  route drops: 0
  ping gateway: ping.ubnt.com - REACHABLE

  eth3
  status: Running
  pings: 142
  fails: 0
  run fails: 0/3
  route drops: 0
  ping gateway: ping.ubnt.com - REACHABLE

show load-balance status

Group wan_failover
  interface   : eth2
  carrier     : up
  status      : active
  gateway     : (ISP gateway)
  route table : 201
  weight      : 50%
  flows
      WAN Out : 2622
      WAN In  : 0
    Local Out : 4

  interface   : eth3
  carrier     : up
  status      : active
  gateway     : (Same ISP gateway)
  route table : 202
  weight      : 50%
  flows
      WAN Out : 2346
      WAN In  : 0
    Local Out : 0

UF.PNG

 

I'm not fussed running it with a PfSense front on one of the WAN's but just curious if its possible to create an option for 2 seperate WAN connections which share the same ISP gateway's.

 

Thanks

Reply