Upcoming Maintenance Alert:

The UBNT Community will be upgraded at 5pm MDT on April 25th. During this time the community forums will be set to read-only status.

Learn more

×
Reply
Highlighted
New Member
Posts: 11
Registered: ‎03-13-2017

Dual WAN Failover Only Misbehaving

I've read through countless threads on failover configurations, but none seem to have the same issue I have.

I previously had this working perfectly fine, but I had to factory reset the device and didn't have a config backup.

 

Running 1.9.7 hotfix 4 on an EdgeRouter X.

Used load balance wizard to set up (with eth1 being failover only)

 

eth0 is primary, eth1 is failover only. eth2-5 are switched with a couple of vlans.

When I plug in eth1 while eth0 is working, about 10 seconds later and all hell breaks loose.

It starts to presumably load balance between the two, despite the explicit instruction to not do so.

All web pages struggle to load, pingplotter shows traces being mangled beyond all recognition, etc.

 

Both interfaces are in the same default "G" group, with eth1 set to failover-only.

I've disabled lb-local, and set eth0 to a higher weight, no luck.

 

Here are various configs and outputs.

I think there might be some firewall rule that I'm missing, but I'm honestly not sure.

It worked without any additional configuration last time, so I'm unsure what to look for.

Any help would be appreciated, sorry to add to the pile of failover posts!

 

$ show load-balance watchdog
Group G
  eth0
  status: Running
  pings: 39
  fails: 0
  run fails: 0/3
  route drops: 2
  ping gateway: ping.ubnt.com - REACHABLE
  last route drop   : Thu Dec  7 04:10:25 2017
  last route recover: Thu Dec  7 04:11:27 2017

  eth1
  status: Waiting on recovery (0/3)
  failover-only mode
  pings: 3
  fails: 3
  run fails: 3/3
  route drops: 3
  ping gateway: ping.ubnt.com - DOWN
  last route drop   : Thu Dec  7 04:14:33 2017
  last route recover: Thu Dec  7 04:13:06 2017
$ show load-balance status
Group G
  interface   : eth0
  carrier     : up
  status      : active
  gateway     : x.x.x.x
  route table : 201
  weight      : 100%
  flows
      WAN Out : 23
      WAN In  : 497
    Local Out : 0

  interface   : eth1
  carrier     : down
  status      : failover
  gateway     : unknown
  route table : 203
  weight      : 0%
  flows
      WAN Out : 2
      WAN In  : 40
    Local Out : 0

^ eth1 is unplugged so I can use the internet and type this thread up.

 

# show load-balance
 group G {
     interface eth0 {
         route {
             default
         }
         weight 100
     }
     interface eth1 {
         failover-only
     }
     lb-local disable
     lb-local-metric-change enable
 }
show interfaces ethernet
 ethernet eth0 {
     address dhcp
     description "WAN: XXXX"
     duplex auto
     firewall {
         in {
             name WAN_IN
         }
         local {
             name WAN_LOCAL
         }
     }
     speed auto
 }
 ethernet eth1 {
     address dhcp
     description "WAN: YYYY"
     duplex auto
     firewall {
         in {
             name WAN_IN
         }
         local {
             name WAN_LOCAL
         }
     }
     speed auto
 }
 ethernet eth2 {
     description "LAN: Loft Switch"
     duplex auto
     speed auto
 }
 ethernet eth3 {
     description "LAN: AP-AC-PRO"
     duplex auto
     speed auto
 }
 ethernet eth4 {
     description "LAN: AP-AC MESH"
     duplex auto
     poe {
         output pthru
     }
     speed auto
 }
show firewall
 all-ping enable
 broadcast-ping disable
 group {
     network-group PRIVATE_NETS {
         network 192.168.0.0/16
         network 172.16.0.0/12
         network 10.0.0.0/8
     }
 }
 ipv6-receive-redirects disable
 ipv6-src-route disable
 ip-src-route disable
 log-martians disable
 modify balance {
     rule 10 {
         action modify
         description "do NOT load balance lan to lan"
         destination {
             group {
                 network-group PRIVATE_NETS
             }
         }
         modify {
             table main
         }
     }
     rule 20 {
         action modify
         description "do NOT load balance destination public address"
         destination {
             group {
                 address-group ADDRv4_eth0
             }
         }
         modify {
             table main
         }
     }
     rule 30 {
         action modify
         description "do NOT load balance destination public address"
         destination {
             group {
                 address-group ADDRv4_eth1
             }
         }
         modify {
             table main
         }
     }
     rule 70 {
         action modify
         modify {
             lb-group G
         }
     }
 }
 name WAN_IN {
     default-action drop
     description "WAN to internal"
     rule 10 {
         action accept
         description "Allow established/related"
         state {
             established enable
             related enable
         }
     }
     rule 30 {
         action drop
         description "Drop invalid state"
         state {
             invalid enable
         }
     }
 }
 name WAN_LOCAL {
     default-action drop
     description "WAN to router"
     rule 10 {
         action accept
         description "Allow established/related"
         state {
             established enable
             related enable
         }
     }
     rule 20 {
         action accept
         description "Allow ICMP"
         destination {
             group {
                 address-group ADDRv4_eth0
             }
         }
         log disable
         protocol icmp
     }
     rule 30 {
         action drop
         description "Drop invalid state"
         state {
             invalid enable
         }
     }
 }
 receive-redirects disable
 send-redirects enable
 source-validation disable
 syn-cookies enable

47576

New Member
Posts: 11
Registered: ‎03-13-2017

Re: Dual WAN Failover Only Misbehaving

It breifly worked after I posted this.

Unplugged primary, it immediately switched to secondary, but when I plugged secondary back in, same mish-mash load balancing occured.

Established Member
Posts: 3,675
Registered: ‎01-04-2017
Kudos: 451
Solutions: 167

Re: Dual WAN Failover Only Misbehaving

It's working as designed. When you plug WAN1 back in only new connections start on WAN1. You'll need to add a script during failover that clears the state table.
"What we've got here is failure to communicate"
Feature Request: RFC 6296 (IPv6 to IPv6 Npt)
Unbound for DNS
New Member
Posts: 11
Registered: ‎03-13-2017

Re: Dual WAN Failover Only Misbehaving

Odd, how did it work without scripts and whatnot before?

I thought failover-only meant, failover only.

Before I factory reset, I did the same wizard and it just worked out of the box with no issues. Failover switched over, and switched back to main once main came back up. No scripts, etc.

Established Member
Posts: 3,675
Registered: ‎01-04-2017
Kudos: 451
Solutions: 167

Re: Dual WAN Failover Only Misbehaving

It is switching back, but only for new connections not estab/related
"What we've got here is failure to communicate"
Feature Request: RFC 6296 (IPv6 to IPv6 Npt)
Unbound for DNS
New Member
Posts: 11
Registered: ‎03-13-2017

Re: Dual WAN Failover Only Misbehaving

I don't think so...

The moment I plug the failover in, it starts to send a whole lot of traffic over eth1.

ICMP packets are split between the two, so I get a traceroute that is a mix between both routes.

This lasts an indefinite amount of time until I unplug the failover.

The internet is completely unusable if I have the failover plugged in at any time.

Established Member
Posts: 3,675
Registered: ‎01-04-2017
Kudos: 451
Solutions: 167

Re: Dual WAN Failover Only Misbehaving

Well duh, that's because you haven't applied the modify rules to any interfaces yet ;-p. Sorry should if looked at your config before
"What we've got here is failure to communicate"
Feature Request: RFC 6296 (IPv6 to IPv6 Npt)
Unbound for DNS
New Member
Posts: 11
Registered: ‎03-13-2017

Re: Dual WAN Failover Only Misbehaving

Ahh, that would probably explain a thing or two.

Though I'm not sure I'm applying the rule correctly still.

 

 ethernet eth0 {
     address dhcp
     description "WAN: SkyFiber"
     duplex auto
     firewall {
         in {
             modify balance
             name WAN_IN
         }
         local {
             name WAN_LOCAL
         }
     }
     speed auto
 }
 ethernet eth1 {
     address dhcp
     description "WAN: Frontier"
     duplex auto
     firewall {
         in {
             modify balance
             name WAN_IN
         }
         local {
             name WAN_LOCAL
         }
     }
     speed auto
 }

commited change, and still seem to have the same issue?

Established Member
Posts: 3,675
Registered: ‎01-04-2017
Kudos: 451
Solutions: 167

Re: Dual WAN Failover Only Misbehaving

The modify get out on your lan in interfaces
"What we've got here is failure to communicate"
Feature Request: RFC 6296 (IPv6 to IPv6 Npt)
Unbound for DNS
Established Member
Posts: 3,675
Registered: ‎01-04-2017
Kudos: 451
Solutions: 167

Re: Dual WAN Failover Only Misbehaving

Put*
"What we've got here is failure to communicate"
Feature Request: RFC 6296 (IPv6 to IPv6 Npt)
Unbound for DNS
New Member
Posts: 11
Registered: ‎03-13-2017

Re: Dual WAN Failover Only Misbehaving

Noticed I didn't put the full interface config in, only part of it.

I unset the WAN rules, and put it on LAN. Also worth noting that it's on the switch interface.

Still not working Man Sad

 

Re-did the entire load balance config from scatch, following this post:

https://community.ubnt.com/t5/EdgeMAX/802-3ad-failover-QOS-help/m-p/768682#M26442

 

Though it still doesn't work, starting to lose my mind lol.

 

show load-balance
 group WAN_FAILOVER {
     interface eth0 {
     }
     interface eth1 {
         failover-only
     }
 }
show firewall modify WAN_POLICY
 rule 10 {
     action modify
     modify {
         lb-group WAN_FAILOVER
     }
 }
show interfaces
 ethernet eth0 {
     address dhcp
     description "WAN: SkyFiber"
     duplex auto
     firewall {
         in {
             name WAN_IN
         }
         local {
             name WAN_LOCAL
         }
     }
     speed auto
 }
 ethernet eth1 {
     address dhcp
     description "WAN: Frontier"
     duplex auto
     firewall {
         in {
             name WAN_IN
         }
         local {
             name WAN_LOCAL
         }
     }
     speed auto
 }
 ethernet eth2 {
     description "LAN: Loft Switch"
     duplex auto
     firewall {
         in {
             modify WAN_POLICY
         }
     }
     speed auto
 }
 ethernet eth3 {
     description "LAN: AP-AC-PRO"
     duplex auto
     firewall {
         in {
             modify WAN_POLICY
         }
     }
     speed auto
 }
 ethernet eth4 {
     description "LAN: AP-AC MESH"
     duplex auto
     firewall {
         in {
             modify WAN_POLICY
         }
     }
     poe {
         output pthru
     }
     speed auto
 }
 loopback lo {
 }
 switch switch0 {
     description Local
     firewall {
         in {
             modify WAN_POLICY
         }
     }
     mtu 1500
     switch-port {
         interface eth2 {
             vlan {
                 pvid 1
                 vid 100
             }
         }
         interface eth3 {
             vlan {
                 pvid 1
                 vid 100
             }
         }
         interface eth4 {
             vlan {
                 pvid 1
                 vid 100
             }
         }
         vlan-aware enable
     }
     vif 1 {
         address 172.20.0.1/20
         description Private
         mtu 1500
     }
     vif 100 {
         address 172.20.10.1/24
         description Public
         mtu 1500
     }
 }

 

Established Member
Posts: 3,675
Registered: ‎01-04-2017
Kudos: 451
Solutions: 167

Re: Dual WAN Failover Only Misbehaving

You have now changed everything, therefore now we need to start from scratch. Post your full config
"What we've got here is failure to communicate"
Feature Request: RFC 6296 (IPv6 to IPv6 Npt)
Unbound for DNS
Reply