Upcoming Maintenance Alert:

The UBNT Community will be upgraded at 5pm MDT on April 25th. During this time the community forums will be set to read-only status.

Learn more

×
Reply
Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3055
Solutions: 945
Contributions: 16

WAN load-balancing except for some traffic

Some have reported having trouble with WAN load-balance when going to secure sites (e.g. banking, insurance, etc.).  So they would prefer to have https traffic always use the same WAN interface except in the case of fail-over.  To accomplish this we start with defining 2 load-balance groups - one will use both WAN interfaces while the other will always use eth1 unless eth1 is down:

bnt@wlb# show load-balance 
 group HTTPS {
     interface eth1 {
     }
     interface eth2 {
         failover-only
     }
 }
 group LB-LAN {
     interface eth1 {
     }
     interface eth2 {
     }
 }
[edit]

 Then define the firewall modify rule to select which load-balance group to use:

ubnt@wlb# show firewall modify  
 modify WLB {
     rule 10 {
         action modify
         destination {
             port 443
         }
         modify {
             lb-group HTTPS
         }
         protocol tcp
     }
     rule 20 {
         action modify
         modify {
             lb-group LB-LAN
         }
     }
 }

Note: it's important to make the most specific match 1st.  If rules 10 and 20 were swapped then all the traffic would use LB-LAN since there is no match criteria in that rule.

Then apply the modify rule to the LAN interface:

ubnt@wlb# show interfaces    
 ethernet eth0 {
     address 192.168.1.1/24
     description LAN
     duplex auto
     firewall {
         in {
             modify WLB
         }
     }
     speed auto
 }

 Now how do we test that all https will go out eth1.  One way is to do a packet capture on both interfaces while a LAN client logs into his bank.

ubnt@wlb:~$ sudo tcpdump -n -v -i eth1 -w test port https
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
^C689 packets captured
786 packets received by filter
97 packets dropped by kernel

 Repeat test while capture port 443 on eth2:

ubnt@wlb:~$ sudo tcpdump -n -v -i eth2 -w test port https
tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
^C0 packets captured
0 packets received by filter
0 packets dropped by kernel

If we look at the load-balance status we can see for group HTTPS all the traffic has gone out eth0 while group LB-LAN is load-balanced:

ubnt@wlb:~$ show load-balance status 
Group HTTPS
  interface   : eth1
  carrier     : up
  status      : active
  gateway     : 172.16.3.242
  weight      : 100
  flows
      WAN Out : 60
      WAN In  : 0
    Local Out : 1319

  interface   : eth2
  carrier     : up
  status      : failover
  gateway     : 2.2.2.2
  weight      : 0
  flows
      WAN Out : 0
      WAN In  : 0
    Local Out : 348

Group LB-LAN
  interface   : eth1
  carrier     : up
  status      : active
  gateway     : 172.16.3.242
  weight      : 50
  flows
      WAN Out : 320
      WAN In  : 0
    Local Out : 348

  interface   : eth2
  carrier     : up
  status      : active
  gateway     : 2.2.2.2
  weight      : 50
  flows
      WAN Out : 306
      WAN In  : 0
    Local Out : 348

 Attached is the complete config.boot file.

EdgeMAX Router Software Development
Attachment
Member
Posts: 134
Registered: ‎09-13-2013
Kudos: 9

Re: WAN load-balancing except for some traffic

Awesome! Any way you could tell me how I would also send ftp through only one?

Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3055
Solutions: 945
Contributions: 16

Re: WAN load-balancing except for some traffic

Well if you want ftp to go out the same interface as http, you could just add port 21 to in the modify rule.  Otherwise create an new load-balance group.

EdgeMAX Router Software Development
Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3055
Solutions: 945
Contributions: 16

Re: WAN load-balancing except for some traffic

Well if you want ftp to go out the same interface as http, you could just add port 21 to in the modify rule.  Otherwise create an new load-balance group.

EdgeMAX Router Software Development
Member
Posts: 134
Registered: ‎09-13-2013
Kudos: 9

Re: WAN load-balancing except for some traffic

Hey Stig, I have another question. We already configured our router for load balancing based on this.

How would we go about adding this to our current config? Or do we have to rebuild completely?

 

We are having issues with HTTPS, FTP, and mail services so we want all of those to go out the fiber connection only (unless there is a failure).

Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3055
Solutions: 945
Contributions: 16

Re: WAN load-balancing except for some traffic


Joshamo wrote:

Hey Stig, I have another question. We already configured our router for load balancing based on this.

How would we go about adding this to our current config? Or do we have to rebuild completely?

 

We are having issues with HTTPS, FTP, and mail services so we want all of those to go out the fiber connection only (unless there is a failure).


If you post your config I can look if there's anything that can be salvaged.

EdgeMAX Router Software Development
Member
Posts: 134
Registered: ‎09-13-2013
Kudos: 9

Re: WAN load-balancing except for some traffic

[ Edited ]

Here ya go!

Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3055
Solutions: 945
Contributions: 16

Re: WAN load-balancing except for some traffic


Joshamo wrote:

Here ya go!


Well some can be re-used.  I think the changes needed would be something like:

configure
delete interfaces ethernet eth0 firewall
delete interfaces ethernet eth1 firewall
commit

delete firewall modify ISP1_IN
delete firewall modify ISP2_IN
delete firewall modify balance rule
commit

delete protocols static table
commit

set firewall group port-group WAN1 description "port to force to WAN1"
set firewall group port-group WAN1 port 443
set firewall group port-group WAN1 port 21
commit

set load-balance group WAN1 interface eth0
set load-balance group WAN1 interface eth1 failover-only
set load-balance group BOTH interface eth0
set load-balance group BOTH interface eth1
commit

set firewall modify balance rule 10 destination group address-group ADDRv4_eth0
set firewall modify balance rule 10 modify table main
set firewall modify balance rule 20 destination group address-group ADDRv4_eth1
set firewall modify balance rule 20 modify table main
set firewall modify balance rule 30 destination group address-group ADDRv4_eth2
set firewall modify balance rule 30 modify table main
set firewall modify balance rule 40 destination group port-group WAN1
set firewall modify balance rule 40 modify lb-group WAN1
set firewall modify balance rule 50 modify lb-group BOTH
commit

 And then if you want to use eth1 also:

delete interfaces ethernet eth1 disable
commit

 

I notice you have no firewall. Is there another firewall in front of these wan interfaces?

I also noticed your LAN address is a public address.  If this is a routed public address that you own then you don't need NAT.

You also mentioned having trouble with ftp in load balancing.  Are you are ftp is an unsecure protocol and that you should be using scp instead?  I have a firewall rule to block telnet and ftp from going out the WAN since both use clear text passwords.

EdgeMAX Router Software Development
Member
Posts: 134
Registered: ‎09-13-2013
Kudos: 9

Re: WAN load-balancing except for some traffic

Alright it seems to be working, is there any way I can test to make sure it is forcing https out the correct port?

As for the firewall and such, we do have a firewall but thank you for pointing that out Man Happy

Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3055
Solutions: 945
Contributions: 16

Re: WAN load-balancing except for some traffic

You could use tcpdump to verify "sudo tcpdump -n -v -i <interface> port 443"

EdgeMAX Router Software Development
Member
Posts: 360
Registered: ‎08-03-2012
Kudos: 28
Solutions: 12

Re: WAN load-balancing except for some traffic

[ Edited ]

I have it setup by default contained in wizard.

 

eth0 = internet 1 (interface 1Mb/1Mb)
eth1 = internet 2 (interface 4Mb/1Mb)
eth2 = lan (interface DHCP)

 

for my case I have to change some interface?

when we used to bank, corporate domain, the ip changes and disconnections have that problem.

"La tecnología no se detiene tampoco nosotros" - "فناوری را متوقف کند، نه می کنیم" - "Технология не останавливается, мы тоже" - "A tecnologia não pára, nem nós"
New Member
Posts: 2
Registered: ‎01-09-2015
Kudos: 1

Re: WAN load-balancing except for some traffic

I just bought an Edge router and I cannot access banking and have tried to figure out the config tree changes or command line changes to replicate what you show here, but I'm clearly too new to your system to figure it out without pulling all my hair out.

 

Is it possible to post the command line instructions to configure what you show here?

 

Thanks!

Member
Posts: 266
Registered: ‎01-09-2014
Kudos: 165
Solutions: 21

Re: WAN load-balancing except for some traffic

If you used the Wizard to setup load balancing the commands below will set it up for you.

 

delete firewall modify balance rule 1

set firewall modify balance rule 10 action modify
set firewall modify balance rule 10 destination port 443
set firewall modify balance rule 10 modify lb-group HTTPS
set firewall modify balance rule 10 protocol tcp
set firewall modify balance rule 20 action modify
set firewall modify balance rule 20 modify lb-group G

set load-balance group G interface eth0
set load-balance group G interface eth1
set load-balance group HTTPS interface eth0
set load-balance group HTTPS interface eth1 failover-only

 There is one thing I have noticed since doing this per the config Stig gave Netflix stops working

Ubiquiti Certified - UEWA / UBRSS / UBWS / USRS
Senior Member
Posts: 3,797
Registered: ‎09-12-2010
Kudos: 1317
Solutions: 31

Re: WAN load-balancing except for some traffic

Before those commands, type 'configure' to be able to edit anything first.

Then when done the commands, type:
commit
save
exit

In order to have it saved to the boot config and exit the config mode.
TicoBytes.com | www.ticobytes.com
Ubiquiti Solutions and English / Spanish IT Support in Costa Rica
UBRSS, UBWA, UEWA - Ubiquiti Certified Trainer
New Member
Posts: 2
Registered: ‎01-09-2015
Kudos: 1

Re: WAN load-balancing except for some traffic

Thanks ryancris and codyloco. Yes, it was set up via the wizard and I used the commands you sent to change the config and save. I have tested the bank that was giving us problems -- works fine. What is curious is that the speeds I'm getting are lower now at http://www.speedtest.net/.

 

For example, prior to the changes you suggested these are the results pretty consistently:

Orig

 

After the change, it has dropped down to this, consistently:

New

The banking is clearly more important than the speed and since one WAN is 5Mbps and the other 4Mbps (dsl), it looks like the benefit I was getting merging both of these connections is gone as this result is the same as when I use just the Cable WAN connection. The dashboard seems to reflect a little bit of load balancing between the two during a speed test, so maybe you have some more suggestions on tweaking it -- or maybe speedtest uses port 443 in some way ??

 

I did change the commands slightly just to accommodate the dsl connection as follows, in case it helps anyone...

 

cli@ubnt:~$ configure                                                        
cli@ubnt# delete firewall modify balance rule 1                              
cli@ubnt# set firewall modify balance rule 10 action modify                  
cli@ubnt# set firewall modify balance rule 10 destination port 443           
cli@ubnt# set firewall modify balance rule 10 modify lb-group HTTPS          
cli@ubnt# set firewall modify balance rule 10 protocol tcp                   
cli@ubnt# set firewall modify balance rule 20 action modify                  
cli@ubnt# set firewall modify balance rule 20 modify lb-group G              
cli@ubnt# set load-balance group G interface eth0                            
The specified configuration node already exists                                
cli@ubnt# set load-balance group G interface pppoe1                          
The specified configuration node already exists                                
cli@ubnt# set load-balance group HTTPS interface eth0                        
cli@ubnt# set load-balance group HTTPS interface pppoe1 failover-only        
cli@ubnt# commit                                                             
cli@ubnt# save                                                               
Saving configuration to '/config/config.boot'...                               
Done 
cli@ubnt# exit                                                               
exit 

 

Member
Posts: 266
Registered: ‎01-09-2014
Kudos: 165
Solutions: 21

Re: WAN load-balancing except for some traffic

[ Edited ]

I have not noticed a problem with my speeds dropping.  Now port 443 is used on alot of website as a result Netflix website will not work anymore nor will Playstion or XBOX.....you can get XBOX and Playstion to work again by setting up a Source NAT from the gaming consoles static address's and masqurade them to one or the other internet connections.  But you will not have load balancing when gaming.

 

I am still trying to resolve this issuse were Netflix and Banks will work not one or the other and same with gaming consoles.

 

If i disable the commands above all will work again but the banks.

Ubiquiti Certified - UEWA / UBRSS / UBWS / USRS
New Member
Posts: 1
Registered: ‎04-21-2015

Re: WAN load-balancing except for some traffic

Hello I have a EdgeRouter Lite v1.6.0 i have set up the load balancing using the wizard. I wont to route all the traffic for port 5060 via eth0 and the rest via eth1. i tried using the code by UBNT-stig with a few changes and also the code by ryancris but no luck. I know this is quite an old post but maybe someone can help.

 

firewall {
    all-ping enable
    broadcast-ping disable
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians disable
    modify balance {
        rule 1 {
            action modify
            modify {
                lb-group G
            }
        }
    }
    name WAN_IN {
        default-action drop
        description "WAN to internal"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    name WAN_LOCAL {
        default-action drop
        description "WAN to router"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    receive-redirects disable
    send-redirects enable
    source-validation disable
    syn-cookies enable
}
interfaces {
    ethernet eth0 {
        address 119.82.116.54/30
        description Internet
        duplex auto
        firewall {
            in {
                name WAN_IN
            }
            local {
                name WAN_LOCAL
            }
        }
        speed auto
        traffic-policy {
            out shaper1
        }
    }
    ethernet eth1 {
        address 192.168.0.232/24
        description "Internet 2"
        duplex auto
        firewall {
            in {
                name WAN_IN
            }
            local {
                name WAN_LOCAL
            }
        }
        speed auto
    }
    ethernet eth2 {
        address 192.168.1.1/24
        description Local
        duplex auto
        firewall {
            in {
                modify balance
            }
        }
        speed auto
        traffic-policy {
            out shaper1
        }
    }
    loopback lo {
    }
}
load-balance {
    group G {
        interface eth0 {
        }
        interface eth1 {
            failover-only
        }
    }
}
protocols {
    static {
        route 0.0.0.0/0 {
            next-hop 119.82.116.53 {
            }
            next-hop 192.168.0.1 {
            }
        }
    }
}
service {
    dhcp-server {
        disabled false
        hostfile-update disable
        shared-network-name LAN {
            authoritative enable
            subnet 192.168.1.0/24 {
                default-router 192.168.1.1
                dns-server 192.168.1.1
                lease 86400
                start 192.168.1.150 {
                    stop 192.168.1.243
                }
                static-mapping SIP-T21P {
                    ip-address 192.168.1.155
                    mac-address 00:15:65:63:1c:5a
                }
                static-mapping switch3f8a51 {
                    ip-address 192.168.1.176
                    mac-address 34:bd:c8:3f:8a:51
                }
                static-mapping switch3f6903 {
                    ip-address 192.168.1.178
                    mac-address 34:bd:c8:3f:69:03
                }
                static-mapping switch3f6920 {
                    ip-address 192.168.1.177
                    mac-address 34:bd:c8:3f:69:20
                }
            }
        }
    }
    dns {
        forwarding {
            cache-size 5000
            listen-on eth2
        }
    }
    gui {
        https-port 443
    }
    nat {
        rule 5000 {
            outbound-interface eth0
            type masquerade
        }
        rule 5002 {
            outbound-interface eth1
            type masquerade
        }
    }
    ssh {
        port 22
        protocol-version v2
    }
}
system {
    conntrack {
        expect-table-size 4096
        hash-size 4096
        table-size 32768
        tcp {
            half-open-connections 512
            loose enable
            max-retrans 3
        }
    }
    host-name ubnt
    login {
        user admin {
            authentication {
                encrypted-password $6$4IWzMkn33G$iAp2sBhOhOzubrAv4.TXwwSbWjj.rPuEHN0jp1kn1hl8dE7D4AXO3M3p9bRiVw9rAZXD3XXg.bmtbf1gcMFqT/
                plaintext-password ""
            }
            level admin
        }
        user ubnt {
            authentication {
                encrypted-password $1$zKNoUbAo$gomzUbYvgyUMcD436Wo66.
            }
            level admin
        }
    }
    name-server 8.8.8.8
    name-server 8.8.4.4
    ntp {
        server 0.ubnt.pool.ntp.org {
        }
        server 1.ubnt.pool.ntp.org {
        }
        server 2.ubnt.pool.ntp.org {
        }
        server 3.ubnt.pool.ntp.org {
        }
    }
    package {
        repository squeeze {
            components "main contrib non-free"
            distribution squeeze
            password ""
            url http://ftp.us.debian.org/debian/
            username ""
        }
        repository squeeze-updates {
            components "main contrib"
            distribution squeeze/updates
            password ""
            url http://security.debian.org/
            username ""
        }
    }
    syslog {
        global {
            facility all {
                level notice
            }
            facility protocols {
                level debug
            }
        }
    }
    time-zone UTC
}
traffic-policy {
    shaper shaper1 {
        bandwidth 5mbit
        class 2 {
            bandwidth 75%
            burst 15k
            ceiling 80%
            match PORT80 {
                ip {
                    source {
                        port 80
                    }
                }
            }
            match PORT443 {
                ip {
                    source {
                        port 443
                    }
                }
            }
            priority 3
            queue-type fair-queue
        }
        class 3 {
            bandwidth 1%
            burst 15k
            ceiling 25%
            match PORT20 {
                ip {
                    source {
                        port 20
                    }
                }
            }
            match PORT21 {
                ip {
                    source {
                        port 21
                    }
                }
            }
            priority 4
            queue-type fair-queue
        }
        class 4 {
            bandwidth 100%
            burst 15k
            ceiling 100%
            match PORT22 {
                ip {
                    source {
                        port 22
                    }
                }
            }
            priority 7
            queue-type fair-queue
        }
        class 5 {
            bandwidth 5%
            burst 15k
            ceiling 15%
            match PORT53 {
                ip {
                    source {
                        port 53
                    }
                }
            }
            match PORT5060 {
                ip {
                    source {
                        port 5060
                    }
                }
            }
            match PORT5061 {
                ip {
                    source {
                        port 5061
                    }
                }
            }
            match PORT5062 {
                ip {
                    source {
                        port 5062
                    }
                }
            }
            queue-type fair-queue
        }
        class 6 {
            bandwidth 90%
            burst 15k
            ceiling 100%
            priority 1
            queue-type fair-queue
        }
        default {
            bandwidth 15%
            burst 15k
            ceiling 35%
            priority 7
            queue-type fair-queue
        }
    }
}


/* Warning: Do not remove the following line. */
/* === vyatta-config-version: "config-management@1:conntrack@1:cron@1:dhcp-relay@1:dhcp-server@4:firewall@5:ipsec@4:nat@3:qos@1:quagga@2:system@4:ubnt-pptp@1:ubnt-util@1:vrrp@1:webgui@1:webproxy@1:zone-policy@1" === */
/* Release version: v1.6.0.4716006.141031.1731 */

 

New Member
Posts: 10
Registered: ‎02-19-2015

Re: WAN load-balancing except for some traffic

I'm unable to set these changes, is it something broken in 1.7 software?  I set up load-balancing via the wizard and the CLI fails at step;

 

ubnt@ubnt# set load-balance group G interface eth0

 

cannot set node "eth0": number of values exceeds limit (2 allowed)

set failed

 

Please can anyone advise?

 

Ta Andy

Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3055
Solutions: 945
Contributions: 16

Re: WAN load-balancing except for some traffic

@TelecomAndy currently the load-balance feature is limited to 2 WAN interfaces per load-balance group.  That error means your trying to add a 3rd.  If that's not the case, then start your own thread include your full confiig file and the problem you're having.  It's also a good idea to read the load-balance KB when using the feature - LINK.

EdgeMAX Router Software Development
New Member
Posts: 9
Registered: ‎06-08-2016

Re: WAN load-balancing except for some traffic

[ Edited ]

hi there, 

I currently having issues with this topic, I tried the sticky feature from the CLI:

configure
set load-balance group G sticky dest-addr enable
commit
save

but this didnt help.

 

Here is my setup: http://pastebin.com/7RpqPFj5

 

Can anyone help me with the right commands for my setup? 

Really appreciated! Man Happy)

Reply