WAN load-balancing except for some traffic


  • Beta - EdgeRouter

    Some have reported having trouble with WAN load-balance when going to secure sites (e.g. banking, insurance, etc.).  So they would prefer to have https traffic always use the same WAN interface except in the case of fail-over.  To accomplish this we start with defining 2 load-balance groups - one will use both WAN interfaces while the other will always use eth1 unless eth1 is down:

    bnt@wlb# show load-balance 
     group HTTPS {
         interface eth1 {
         }
         interface eth2 {
             failover-only
         }
     }
     group LB-LAN {
         interface eth1 {
         }
         interface eth2 {
         }
     }
    [edit]
    
    

    Then define the firewall modify rule to select which load-balance group to use:

    ubnt@wlb# show firewall modify  
     modify WLB {
         rule 10 {
             action modify
             **destination {
                 port 443
             }**
             modify {
                 lb-group **HTTPS**
             }
             protocol tcp
         }
         rule 20 {
             action modify
             modify {
                 lb-group **LB-LAN**
             }
         }
     }
    
    

    Note: it's important to make the most specific match 1st.  If rules 10 and 20 were swapped then all the traffic would use LB-LAN since there is no match criteria in that rule.

    Then apply the modify rule to the LAN interface:

    ubnt@wlb# show interfaces    
     ethernet eth0 {
         address 192.168.1.1/24
         description LAN
         duplex auto
         firewall {
             in {
                 modify **WLB**
             }
         }
         speed auto
     }
    
    

    Now how do we test that all **https **will go out eth1.  One way is to do a packet capture on both interfaces while a LAN client logs into his bank.

    ubnt@wlb:~$ sudo tcpdump -n -v -i eth1 -w test port https
    tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
    ^C689 packets captured
    786 packets received by filter
    97 packets dropped by kernel
    
    

    Repeat test while capture port 443 on eth2:

    ubnt@wlb:~$ sudo tcpdump -n -v -i eth2 -w test port https
    tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
    ^C0 packets captured
    0 packets received by filter
    0 packets dropped by kernel
    
    

    If we look at the load-balance status we can see for group HTTPS all the traffic has gone out eth0 while group LB-LAN is load-balanced:

    ubnt@wlb:~$ show load-balance status 
    Group **HTTPS**
      interface   : eth1
      carrier     : up
      status      : active
      gateway     : 172.16.3.242
      weight      : **100**
      flows
          WAN Out : **60**
          WAN In  : 0
        Local Out : 1319
    
      interface   : eth2
      carrier     : up
      status      : failover
      gateway     : 2.2.2.2
      weight      : **0**
      flows
          WAN Out : **0**
          WAN In  : 0
        Local Out : 348
    
    Group **LB-LAN**
      interface   : eth1
      carrier     : up
      status      : active
      gateway     : 172.16.3.242
      weight      : **50**
      flows
          WAN Out : **320**
          WAN In  : 0
        Local Out : 348
    
      interface   : eth2
      carrier     : up
      status      : active
      gateway     : 2.2.2.2
      weight      : **50**
      flows
          WAN Out : **306**
          WAN In  : 0
        Local Out : 348
    
    

    Attached is the complete config.boot file.

    config.boot


  • Beta Testers

    Awesome! Any way you could tell me how I would also send ftp through only one?


  • Beta - EdgeRouter

    Well if you want ftp to go out the same interface as http, you could just add port 21 to in the modify rule.  Otherwise create an new load-balance group.


  • Beta - EdgeRouter

    Well if you want ftp to go out the same interface as http, you could just add port 21 to in the modify rule.  Otherwise create an new load-balance group.


  • Beta Testers

    Hey Stig, I have another question. We already configured our router for load balancing based on this.

    How would we go about adding this to our current config? Or do we have to rebuild completely?

    We are having issues with HTTPS, FTP, and mail services so we want all of those to go out the fiber connection only (unless there is a failure).


  • Beta - EdgeRouter

    Joshamo wrote:

    Hey Stig, I have another question. We already configured our router for load balancing based on this.

    How would we go about adding this to our current config? Or do we have to rebuild completely?

    We are having issues with HTTPS, FTP, and mail services so we want all of those to go out the fiber connection only (unless there is a failure).


    If you post your config I can look if there's anything that can be salvaged.


  • Beta Testers

    Here ya go!

    config.boot


  • Beta - EdgeRouter

    Joshamo wrote:

    Here ya go!


    Well some can be re-used.  I think the changes needed would be something like:

    configure
    delete interfaces ethernet eth0 firewall
    delete interfaces ethernet eth1 firewall
    commit
    
    delete firewall modify ISP1_IN
    delete firewall modify ISP2_IN
    delete firewall modify balance rule
    commit
    
    delete protocols static table
    commit
    
    set firewall group port-group WAN1 description "port to force to WAN1"
    set firewall group port-group WAN1 port 443
    set firewall group port-group WAN1 port 21
    commit
    
    set load-balance group WAN1 interface eth0
    set load-balance group WAN1 interface eth1 failover-only
    set load-balance group BOTH interface eth0
    set load-balance group BOTH interface eth1
    commit
    
    set firewall modify balance rule 10 destination group address-group ADDRv4_eth0
    set firewall modify balance rule 10 modify table main
    set firewall modify balance rule 20 destination group address-group ADDRv4_eth1
    set firewall modify balance rule 20 modify table main
    set firewall modify balance rule 30 destination group address-group ADDRv4_eth2
    set firewall modify balance rule 30 modify table main
    set firewall modify balance rule 40 destination group port-group WAN1
    set firewall modify balance rule 40 modify lb-group WAN1
    set firewall modify balance rule 50 modify lb-group BOTH
    commit
    
    

    And then if you want to use eth1 also:

    delete interfaces ethernet eth1 disable
    commit
    
    

    I notice you have no firewall. Is there another firewall in front of these wan interfaces?

    I also noticed your LAN address is a public address.  If this is a routed public address that you own then you don't need NAT.

    You also mentioned having trouble with ftp in load balancing.  Are you are ftp is an unsecure protocol and that you should be using scp instead?  I have a firewall rule to block telnet and ftp from going out the WAN since both use clear text passwords.


  • Beta Testers
    This post is deleted!

  • Beta Testers

    Alright it seems to be working, is there any way I can test to make sure it is forcing https out the correct port?

    As for the firewall and such, we do have a firewall but thank you for pointing that out :)


  • Beta - EdgeRouter

    You could use tcpdump to verify "sudo tcpdump -n -v -i <interface>port 443"</interface>


  • Beta Testers

    I have it setup by default contained in wizard.

    eth0 = internet 1 (interface 1Mb/1Mb)
    eth1 = internet 2 (interface 4Mb/1Mb)
    eth2 = lan (interface DHCP)

    for my case I have to change some interface?

    when we used to bank, corporate domain, the ip changes and disconnections have that problem.


  • Beta - EdgeRouter
    This post is deleted!

  • D
    Custom Avatar

    I just bought an Edge router and I cannot access banking and have tried to figure out the config tree changes or command line changes to replicate what you show here, but I'm clearly too new to your system to figure it out without pulling all my hair out.

    Is it possible to post the command line instructions to configure what you show here?

    Thanks!


  • R
    Beta Testers

    If you used the Wizard to setup load balancing the commands below will set it up for you.

    delete firewall modify balance rule 1
    
    set firewall modify balance rule 10 action modify
    set firewall modify balance rule 10 destination port 443
    set firewall modify balance rule 10 modify lb-group HTTPS
    set firewall modify balance rule 10 protocol tcp
    set firewall modify balance rule 20 action modify
    set firewall modify balance rule 20 modify lb-group G
    
    set load-balance group G interface eth0
    set load-balance group G interface eth1
    set load-balance group HTTPS interface eth0
    set load-balance group HTTPS interface eth1 failover-only
    

    There is one thing I have noticed since doing this per the config Stig gave Netflix stops working


  • C
    Beta - EdgeRouter

    Before those commands, type 'configure' to be able to edit anything first.

    Then when done the commands, type:
    commit
    save
    exit

    In order to have it saved to the boot config and exit the config mode.


  • D
    Custom Avatar

    Thanks ryancris and codyloco. Yes, it was set up via the wizard and I used the commands you sent to change the config and save. I have tested the bank that was giving us problems – works fine. What is curious is that the speeds I'm getting are lower now at


  • R
    Beta Testers

    I have not noticed a problem with my speeds dropping.  Now port 443 is used on alot of website as a result Netflix website will not work anymore nor will Playstion or XBOX…..you can get XBOX and Playstion to work again by setting up a Source NAT from the gaming consoles static address's and masqurade them to one or the other internet connections.  But you will not have load balancing when gaming.

    I am still trying to resolve this issuse were Netflix and Banks will work not one or the other and same with gaming consoles.

    If i disable the commands above all will work again but the banks.


  • Z
    Custom Avatar

    Hello I have a EdgeRouter Lite v1.6.0 i have set up the load balancing using the wizard. I wont to route all the traffic for port 5060 via eth0 and the rest via eth1. i tried using the code by UBNT-stig with a few changes and also the code by ryancris but no luck. I know this is quite an old post but maybe someone can help.

    firewall {
        all-ping enable
        broadcast-ping disable
        ipv6-receive-redirects disable
        ipv6-src-route disable
        ip-src-route disable
        log-martians disable
        modify balance {
            rule 1 {
                action modify
                modify {
                    lb-group G
                }
            }
        }
        name WAN_IN {
            default-action drop
            description "WAN to internal"
            rule 10 {
                action accept
                description "Allow established/related"
                state {
                    established enable
                    related enable
                }
            }
            rule 20 {
                action drop
                description "Drop invalid state"
                state {
                    invalid enable
                }
            }
        }
        name WAN_LOCAL {
            default-action drop
            description "WAN to router"
            rule 10 {
                action accept
                description "Allow established/related"
                state {
                    established enable
                    related enable
                }
            }
            rule 20 {
                action drop
                description "Drop invalid state"
                state {
                    invalid enable
                }
            }
        }
        receive-redirects disable
        send-redirects enable
        source-validation disable
        syn-cookies enable
    }
    interfaces {
        ethernet eth0 {
            address 119.82.116.54/30
            description Internet
            duplex auto
            firewall {
                in {
                    name WAN_IN
                }
                local {
                    name WAN_LOCAL
                }
            }
            speed auto
            traffic-policy {
                out shaper1
            }
        }
        ethernet eth1 {
            address 192.168.0.232/24
            description "Internet 2"
            duplex auto
            firewall {
                in {
                    name WAN_IN
                }
                local {
                    name WAN_LOCAL
                }
            }
            speed auto
        }
        ethernet eth2 {
            address 192.168.1.1/24
            description Local
            duplex auto
            firewall {
                in {
                    modify balance
                }
            }
            speed auto
            traffic-policy {
                out shaper1
            }
        }
        loopback lo {
        }
    }
    load-balance {
        group G {
            interface eth0 {
            }
            interface eth1 {
                failover-only
            }
        }
    }
    protocols {
        static {
            route 0.0.0.0/0 {
                next-hop 119.82.116.53 {
                }
                next-hop 192.168.0.1 {
                }
            }
        }
    }
    service {
        dhcp-server {
            disabled false
            hostfile-update disable
            shared-network-name LAN {
                authoritative enable
                subnet 192.168.1.0/24 {
                    default-router 192.168.1.1
                    dns-server 192.168.1.1
                    lease 86400
                    start 192.168.1.150 {
                        stop 192.168.1.243
                    }
                    static-mapping SIP-T21P {
                        ip-address 192.168.1.155
                        mac-address 00:15:65:63:1c:5a
                    }
                    static-mapping switch3f8a51 {
                        ip-address 192.168.1.176
                        mac-address 34:bd:c8:3f:8a:51
                    }
                    static-mapping switch3f6903 {
                        ip-address 192.168.1.178
                        mac-address 34:bd:c8:3f:69:03
                    }
                    static-mapping switch3f6920 {
                        ip-address 192.168.1.177
                        mac-address 34:bd:c8:3f:69:20
                    }
                }
            }
        }
        dns {
            forwarding {
                cache-size 5000
                listen-on eth2
            }
        }
        gui {
            https-port 443
        }
        nat {
            rule 5000 {
                outbound-interface eth0
                type masquerade
            }
            rule 5002 {
                outbound-interface eth1
                type masquerade
            }
        }
        ssh {
            port 22
            protocol-version v2
        }
    }
    system {
        conntrack {
            expect-table-size 4096
            hash-size 4096
            table-size 32768
            tcp {
                half-open-connections 512
                loose enable
                max-retrans 3
            }
        }
        host-name ubnt
        login {
            user admin {
                authentication {
                    encrypted-password $6$4IWzMkn33G$iAp2sBhOhOzubrAv4.TXwwSbWjj.rPuEHN0jp1kn1hl8dE7D4AXO3M3p9bRiVw9rAZXD3XXg.bmtbf1gcMFqT/
                    plaintext-password ""
                }
                level admin
            }
            user ubnt {
                authentication {
                    encrypted-password $1$zKNoUbAo$gomzUbYvgyUMcD436Wo66.
                }
                level admin
            }
        }
        name-server 8.8.8.8
        name-server 8.8.4.4
        ntp {
            server 0.ubnt.pool.ntp.org {
            }
            server 1.ubnt.pool.ntp.org {
            }
            server 2.ubnt.pool.ntp.org {
            }
            server 3.ubnt.pool.ntp.org {
            }
        }
        package {
            repository squeeze {
                components "main contrib non-free"
                distribution squeeze
                password ""
                url http://ftp.us.debian.org/debian/
                username ""
            }
            repository squeeze-updates {
                components "main contrib"
                distribution squeeze/updates
                password ""
                url http://security.debian.org/
                username ""
            }
        }
        syslog {
            global {
                facility all {
                    level notice
                }
                facility protocols {
                    level debug
                }
            }
        }
        time-zone UTC
    }
    traffic-policy {
        shaper shaper1 {
            bandwidth 5mbit
            class 2 {
                bandwidth 75%
                burst 15k
                ceiling 80%
                match PORT80 {
                    ip {
                        source {
                            port 80
                        }
                    }
                }
                match PORT443 {
                    ip {
                        source {
                            port 443
                        }
                    }
                }
                priority 3
                queue-type fair-queue
            }
            class 3 {
                bandwidth 1%
                burst 15k
                ceiling 25%
                match PORT20 {
                    ip {
                        source {
                            port 20
                        }
                    }
                }
                match PORT21 {
                    ip {
                        source {
                            port 21
                        }
                    }
                }
                priority 4
                queue-type fair-queue
            }
            class 4 {
                bandwidth 100%
                burst 15k
                ceiling 100%
                match PORT22 {
                    ip {
                        source {
                            port 22
                        }
                    }
                }
                priority 7
                queue-type fair-queue
            }
            class 5 {
                bandwidth 5%
                burst 15k
                ceiling 15%
                match PORT53 {
                    ip {
                        source {
                            port 53
                        }
                    }
                }
                match PORT5060 {
                    ip {
                        source {
                            port 5060
                        }
                    }
                }
                match PORT5061 {
                    ip {
                        source {
                            port 5061
                        }
                    }
                }
                match PORT5062 {
                    ip {
                        source {
                            port 5062
                        }
                    }
                }
                queue-type fair-queue
            }
            class 6 {
                bandwidth 90%
                burst 15k
                ceiling 100%
                priority 1
                queue-type fair-queue
            }
            default {
                bandwidth 15%
                burst 15k
                ceiling 35%
                priority 7
                queue-type fair-queue
            }
        }
    }
    
    /* Warning: Do not remove the following line. */
    /* === vyatta-config-version: "config-management@1:conntrack@1:cron@1:dhcp-relay@1:dhcp-server@4:firewall@5:ipsec@4:nat@3:qos@1:quagga@2:system@4:ubnt-pptp@1:ubnt-util@1:vrrp@1:webgui@1:webproxy@1:zone-policy@1" === */
    /* Release version: v1.6.0.4716006.141031.1731 */
    

  • T
    Custom Avatar

    I'm unable to set these changes, is it something broken in 1.7 software?  I set up load-balancing via the wizard and the CLI fails at step;

    ubnt@ubnt# set load-balance group G interface eth0

    cannot set node "eth0": number of values exceeds limit (2 allowed)

    set failed

    Please can anyone advise?

    Ta Andy


Posts 25Views 34
Log in to reply

Looks like your connection to Ubiquiti Networks Community was lost, please wait while we try to reconnect.