Highlighted
Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2
Accepted Solution

VLAN routing

I have always had my ERL and other devices managed on the LAN interface. I have just changed this set up so that the ERL and everything else is managed on VLAN10. There are several other VLANs 10-15. I use them to manage bandwidth requirements differently for different WIFI users. The router has the following IPs

10.1.1.1 - Untagged (VLAN 1) 

10.10.1.1 - VLAN 10 (Management)

.....

.....

10.15.1.1 - VLAN 15

 

If I am a switch port that is an access port in VLAN 1. Then I cannot access devices in VLAN 10 apart from the ERL itself. If I am in any of the other VLANs I can access VLAN 10 no problems. If I do a trace route, the ERL sends the packets straight out the WAN to the internet where it gets dropped. It seems to ignore the connected network of 10.10.0.0/16 which appears in the routing table. There are only the connected networks and a default route in the table. Is there something strange going on here??


Accepted Solutions
Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

Hi. So I found the culprit. The load balancing config below causes the LAN to send all its traffic out the WAN regardless of its routing table. As a side note, this also implies that the balance would need to be applied to each of the VLANs individually to work correctly. 

set firewall modify BALRULE rule 1 action modify
set firewall modify BALRULE rule 1 modify lb-group BALANCE

set interfaces ethernet eth0 firewall in modify BALRULE

set load-balance group BALANCE interface eth1

set load-balance group BALANCE interface eth2

 

View solution in original post


All Replies
Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

Sanitize and post your config for us to look at.  Tough to help if we don't know how you have your router configured.

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

firewall {
    all-ping enable
    broadcast-ping disable
    group {
        address-group ASTERISK-SERVER {
            address 10.1.50.2
            description ""
        }
        address-group UNIFI-SERVER {
            address 10.1.50.1
            description ""
        }
        network-group VLAN10 {
            network 10.10.0.0/16
        }
        network-group VLAN11 {
            description ""
            network 10.11.0.0/16
        }
        network-group VLAN12 {
            description ""
            network 10.12.0.0/16
        }
        network-group VLAN13 {
            description ""
            network 10.13.0.0/16
        }
        network-group VLAN14 {
            description ""
            network 10.14.0.0/16
        }
        network-group VLAN15 {
            description ""
            network 10.15.0.0/16
        }
    }
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians enable
    modify BALRULE {
        rule 1 {
            action modify
            modify {
                lb-group BALANCE
            }
        }
    }
    name INTERNET-TO-LAN {
        default-action drop
        description ""
        rule 1 {
            action accept
            description ESTABLISHED
            log disable
            protocol all
            state {
                established enable
                invalid disable
                new disable
                related enable
            }
        }
        rule 2 {
            action accept
            description UNIFI
            destination {
                address 10.1.50.1
                group {
                }
                port 8080
            }
            log disable
            protocol tcp
        }
        rule 3 {
            action accept
            description STUN
            destination {
                address 10.1.50.1
                group {
                }
                port 3478
            }
            log disable
            protocol tcp_udp
        }
        rule 4 {
            action accept
            description 8443
            destination {
                address 10.1.50.1
                port 8443
            }
            log disable
            protocol tcp
        }
    }
    name INTERNET-TO-LOCAL {
        default-action drop
        description ""
        rule 1 {
            action accept
            description GRE
            log disable
            protocol gre
        }
        rule 2 {
            action accept
            description PPTP
            destination {
                port 1723
            }
            log disable
            protocol tcp
        }
        rule 8 {
            action accept
            description ESTABLISHED
            log disable
            protocol all
            state {
                established enable
                invalid disable
                new disable
                related enable
            }
        }
        rule 11 {
            action accept
            description PING
            log disable
            protocol icmp
        }
        rule 12 {
            action accept
            description "MANAGEMENT HTTPS"
            destination {
                port https
            }
            log disable
            protocol tcp
        }
    }
    name INTERNET-TO-LAN-2 {
        default-action drop
        description ""
        rule 1 {
            action accept
            log disable
            protocol all
            state {
                established enable
                invalid disable
                new disable
                related enable
            }
        }
    }
    options {
        mss-clamp {
            interface-type pppoe
            interface-type pptp
            mss 1412
        }
    }
    receive-redirects disable
    send-redirects enable
    source-validation disable
    syn-cookies enable
}
interfaces {
    ethernet eth0 {
        address 10.1.1.1/16
        description LAN
        duplex auto
        firewall {
            in {
                modify BALRULE
            }
        }
        redirect ifb1
        speed auto
        traffic-policy {
        }
        vif 9 {
            address 10.9.1.1/16
            description VLAN9
            mtu 1500
        }
        vif 10 {
            address 10.10.1.1/16
            description MGMT
        }
        vif 11 {
            address 10.11.1.1/16
            description VLAN11
            firewall {
                in {
                }
                out {
                }
            }
            traffic-policy {
            }
        }
        vif 12 {
            address 10.12.1.1/16
            description VLAN12
            firewall {
                out {
                }
            }
        }
        vif 13 {
            address 10.13.1.1/16
            description VLAN13
            mtu 1500
        }
        vif 14 {
            address 10.14.1.1/16
            description VLAN14
            mtu 1500
        }
        vif 15 {
            address 10.15.1.1/16
            description VLAN15
            firewall {
                out {
                }
            }
        }
    }
    ethernet eth1 {
        address 192.168.1.50/24
        description INTERNET-MAIN
        duplex auto
        firewall {
            in {
                name INTERNET-TO-LAN
            }
            local {
                name INTERNET-TO-LOCAL
            }
        }
        pppoe 0 {
            default-route auto
            mtu 1492
            name-server auto
        }
        speed auto
        traffic-policy {
        }
    }
    ethernet eth2 {
        address dhcp
        description INTERNET-BACKUP
        duplex auto
        firewall {
            in {
                name INTERNET-TO-LAN-2
            }
        }
        speed auto
    }
    input ifb1 {
        traffic-policy {
        }
    }
    loopback lo {
    }
}
load-balance {
    group BALANCE {
        interface eth1 {
        }
        interface eth2 {
        }
    }
}
port-forward {
    auto-firewall disable
    hairpin-nat disable
    wan-interface eth1
}
protocols {
    static {
        route 0.0.0.0/0 {
            next-hop 192.168.1.1 {
            }
        }
    }
}
service {
    dhcp-server {
        disabled false
        hostfile-update disable
        shared-network-name LAN {
            authoritative disable
            subnet 10.1.0.0/16 {
                default-router 10.1.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 86400
                start 10.1.10.1 {
                    stop 10.1.20.254
                }
            }
        }
        shared-network-name VLAN9 {
            authoritative disable
            subnet 10.9.0.0/16 {
                default-router 10.9.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 86400
                start 10.9.10.1 {
                    stop 10.9.20.254
                }
            }
        }
        shared-network-name VLAN10 {
            authoritative disable
            subnet 10.10.0.0/16 {
                default-router 10.10.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 86400
                start 10.10.10.1 {
                    stop 10.10.20.254
                }
            }
        }
        shared-network-name VLAN11 {
            authoritative disable
            subnet 10.11.0.0/16 {
                default-router 10.11.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 14400
                start 10.11.10.1 {
                    stop 10.11.20.254
                }
            }
        }
        shared-network-name VLAN12 {
            authoritative disable
            subnet 10.12.0.0/16 {
                default-router 10.12.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 14400
                start 10.12.10.1 {
                    stop 10.12.20.254
                }
            }
        }
        shared-network-name VLAN13 {
            authoritative disable
            subnet 10.13.0.0/16 {
                default-router 10.13.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 14400
                start 10.13.10.1 {
                    stop 10.13.20.254
                }
            }
        }
        shared-network-name VLAN14 {
            authoritative disable
            subnet 10.14.0.0/16 {
                default-router 10.14.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 14400
                start 10.14.10.1 {
                    stop 10.14.20.254
                }
            }
        }
        shared-network-name VLAN15 {
            authoritative disable
            subnet 10.15.0.0/16 {
                default-router 10.15.1.1
                dns-server 8.8.8.8
                dns-server 4.2.2.4
                lease 86400
                start 10.15.10.1 {
                    stop 10.15.20.254
                }
            }
        }
    }
    dns {
        forwarding {
            cache-size 5000
            listen-on eth0.9
            listen-on eth0.10
            listen-on eth0.11
            listen-on eth0.12
            listen-on eth0.13
            listen-on eth0.14
            listen-on eth0.15
        }
    }
    gui {
        https-port 443
    }
    nat {
        rule 1 {
            description RTP
            destination {
                group {
                }
                port 10000-20000
            }
            disable
            inbound-interface eth1
            inside-address {
                address 10.1.50.2
                port 10000-20000
            }
            log disable
            protocol tcp_udp
            type destination
        }
        rule 2 {
            description SIP
            destination {
                group {
                }
                port 5060
            }
            disable
            inbound-interface eth1
            inside-address {
                address 10.1.50.2
                port 5060
            }
            log disable
            protocol tcp_udp
            type destination
        }
        rule 3 {
            description IAX
            destination {
                port 4569
            }
            disable
            inbound-interface eth1
            inside-address {
                address 10.1.50.2
                port 4569
            }
            log disable
            protocol tcp_udp
            type destination
        }
        rule 4 {
            description "HAIRPIN 8080"
            destination {
                port 8080
            }
            inbound-interface eth0+
            inside-address {
                address 10.1.50.1
                port 8080
            }
            log disable
            protocol tcp
            type destination
        }
        rule 5 {
            description "HAIRPIN 8443"
            destination {
                port 8443
            }
            inbound-interface eth0+
            inside-address {
                address 10.1.50.1
                port 8443
            }
            log disable
            protocol tcp
            type destination
        }
        rule 6 {
            description UNIFI
            destination {
                group {
                }
                port 8080
            }
            inbound-interface eth1
            inside-address {
                address 10.1.50.1
                port 8080
            }
            log disable
            protocol tcp_udp
            source {
            }
            type destination
        }
        rule 7 {
            description STUN
            destination {
                group {
                }
                port 3478
            }
            inbound-interface eth1
            inside-address {
                address 10.1.50.1
                port 3478
            }
            log disable
            protocol tcp_udp
            type destination
        }
        rule 8 {
            description 8443
            destination {
                port 8443
            }
            inbound-interface eth1
            inside-address {
                address 10.1.50.1
                port 8443
            }
            log disable
            protocol tcp
            type destination
        }
        rule 5000 {
            description "ALLOW OUT ETH1"
            log disable
            outbound-interface eth1
            protocol all
            source {
                address 10.0.0.0/8
            }
            type masquerade
        }
        rule 5001 {
            description "ALLOW OUT ETH2"
            log disable
            outbound-interface eth2
            protocol all
            source {
                address 10.0.0.0/8
            }
            type masquerade
        }
        rule 5002 {
            description HAIRPIN
            destination {
                address 10.1.0.0/16
            }
            log disable
            outbound-interface eth0
            outside-address {
            }
            protocol all
            source {
                address 10.0.0.0/8
            }
            type masquerade
        }
    }
 
   }
    ssh {
        port 22
        protocol-version v2
    }
    telnet {
        port 23
    }
}
system {
    conntrack {
        expect-table-size 4096
        hash-size 4096
        table-size 32768
        tcp {
            half-open-connections 512
            loose enable
            max-retrans 3
        }
    }
    flow-accounting {
        ingress-capture pre-dnat
        interface eth1
        netflow {
            enable-egress {
            }
            server 10.10.50.3 {
                port 2055
            }
            version 9
        }
        syslog-facility daemon
    }
  
            }
            level admin
        }

            }

        }
    }
    name-server 8.8.8.8
    name-server 4.2.2.4
    ntp {
        server 0.ubnt.pool.ntp.org {
        }
        server 1.ubnt.pool.ntp.org {
        }
        server 2.ubnt.pool.ntp.org {
        }
        server 3.ubnt.pool.ntp.org {
        }
    }
    offload {
        ipsec enable
        ipv4 {
            forwarding enable
        }
        ipv6 {
            forwarding disable
        }
    }
    syslog {
        global {
            facility all {
                level notice
            }
            facility protocols {
                level debug
            }
        }
    }
    time-zone Europe/London
}
traffic-policy {
}

            }
            dns-servers {
                server-1 8.8.8.8
                server-2 4.2.2.4
            }
            mtu 1492
            outside-address 192.168.1.50
        }
    }
}


/* Warning: Do not remove the following line. */
/* === vyatta-config-version: "config-management@1:conntrack@1:cron@1:dhcp-relay@1:dhcp-server@4:firewall@5:ipsec@3:nat@3:qos@1:quagga@2:system@4:ubnt-pptp@1:ubnt-util@1:vrrp@1:webgui@1:webproxy@1:zone-policy@1" === */
/* Release version: v1.5.0rc1.4675120.140611.1821 */

 One of the bits I took out was a PPTP VPN which allocates 10.10.30.x for VPN users. May or may not be relevent.

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

[ Edited ]

The first issue I see without looking hard relates to your address scheme.

    group {
        address-group ASTERISK-SERVER {
            address 10.1.50.2
            description ""
        }
        address-group UNIFI-SERVER {
            address 10.1.50.1
            description ""
        }
        network-group VLAN10 {
            network 10.10.0.0/16
        }
        network-group VLAN11 {
            description ""
            network 10.11.0.0/16
        }
        network-group VLAN12 {
            description ""
            network 10.12.0.0/16
        }
        network-group VLAN13 {
            description ""
            network 10.13.0.0/16
        }
        network-group VLAN14 {
            description ""
            network 10.14.0.0/16
        }
        network-group VLAN15 {
            description ""
            network 10.15.0.0/16
        }
    }
---------------------------------------------
interfaces {
    ethernet eth0 {
        address 10.1.1.1/16
        description LAN
        duplex auto
        firewall {
            in {
                modify BALRULE
            }
        }
        redirect ifb1
        speed auto
        traffic-policy {
        }
        vif 9 {
            address 10.9.1.1/16
            description VLAN9
            mtu 1500
        }
        vif 10 {
            address 10.10.1.1/16
            description MGMT
        }
        vif 11 {
            address 10.11.1.1/16
            description VLAN11
            firewall {
                in {
                }
                out {
                }
            }
            traffic-policy {
            }
        }
        vif 12 {
            address 10.12.1.1/16
            description VLAN12
            firewall {
                out {
                }
            }
        }
        vif 13 {
            address 10.13.1.1/16
            description VLAN13
            mtu 1500
        }
        vif 14 {
            address 10.14.1.1/16
            description VLAN14
            mtu 1500
        }
        vif 15 {
            address 10.15.1.1/16
            description VLAN15
            firewall {
                out {
                }
            }
        }
    }

 Your addresses in your groups don't match the addresses on your interfaces.

I'll look further, but that was the first thing that jumped out at me.

 

EDIT:  I guess if I looked further, I would have noticed that they are /16 networks.

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

Thanks. None of the network objects you mention are actually used in the firewall. It seems to me that this is related to routing not firewall but I may be wrong. Also, those addresses do match the VLANs and interfaces.

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

[ Edited ]

@martyh wrote:

Thanks. None of the network objects you mention are actually used in the firewall. It seems to me that this is related to routing not firewall but I may be wrong. Also, those addresses do match the VLANs and interfaces.


Yea, I edited my post.  I missed the /16 on the first glance.

What switch are you using?  Could it be related to the native vlan setting on the switche's trunk port to the router?  I have a similar setup for trunking and have no issues.

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

I have used a Cisco 3750 and a Netgear 8 port. It is the same right now while I am plugged straight into the router. It just routes the VLAN interfaces straight out to the WAN when I am on LAN interface. I think it must be NAT related as I had some hairpin config in there but I tried removing that. Good to know that your set up works as expected.

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

I haven't done rules for hairpin NAT, but I thought I saw a thread recently that showed using outbound-interface rather than inbound-interface.  That caught my eye, but as I said, I haven't had a chance to play with it at all yet.  I jusdt remember that it was backwards from standard NAT.  I'm going to have to figure out how it relates to my zone-based setup when I get the chance.

Just curious, even though it should be default, have you tried switchport trunk native vlan 1 on the switchport for your trunk on the Cisco switch?

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

Hairpin nat should only affect using your public address from the inside.  Doesn't mean it can't mess something up in routing though.

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

Yeah I agree on the hairpin thing. I tried removing it. The web gui left some hairpin config so I removed that in the CLI but no change. I also tried setting the native to 1 manually as you mentioned but no change. It is definitely something going wrong within the router somewhere. One other thing I was playing with is load balancing so I am going to try and remove that to see if that has any effect...

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

How do you like your 3750?  I currently have several 2960s in my network and a 24-port 2960 lan-lite for my core switch.  I've been considering moving to a 3750 for my core so that I can move inter-lan routing down to the core and off of my edge router (ERL).

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing


@martyh wrote:

Yeah I agree on the hairpin thing. I tried removing it. The web gui left some hairpin config so I removed that in the CLI but no change. I also tried setting the native to 1 manually as you mentioned but no change. It is definitely something going wrong within the router somewhere. One other thing I was playing with is load balancing so I am going to try and remove that to see if that has any effect...


There was a lot of loose unused code in the config.  Maybe cleaning some of that up might fix something.  I know that empty firewalls on interfaces shouldn't do anything, but too many times I have seen undefined code do strange things.  I try hard to keep my code as clean as possible.  Another side benifit is that it becomes much easier to troubleshoot as well since it is easier to spot something out of place.

You might try rebooting the router occasionaly as well when modifying code.  I was reading this morning from one of the UBNT staff that there can be bugs related to changing code and not doing a reboot.  Things lingering on old interfaces ect.

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

They are good. I have some 2960s as well. To be honest I don't really demand much of my switches. They do make a very good core switch. We use them for some of the larger networks. I tend to always use routers for the inter VLAN routing though. The networks I work on a temporary and varying in scale so it helps to be able to just have a router and an 8 port one day and a router with a whole load of switches the next.

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

Yeah. I think I will clean up some of that code. A lot of it gets put in by the gui but it doesn't remove it again. I will try the reboots too. Can never underestimate the power of a good ol reboot!!

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

[ Edited ]

You can always save a backup then get rid of all the excess stuff back down to a basic config.  Get that working, then start putting pieces back in and finding where it breaks.

Post your new config once you clean it up.  We'll start with trying to get a simpler config working before playing with all the more complex stuff.  For now, if you're not using it, pull it out.

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

Hi. So I found the culprit. The load balancing config below causes the LAN to send all its traffic out the WAN regardless of its routing table. As a side note, this also implies that the balance would need to be applied to each of the VLANs individually to work correctly. 

set firewall modify BALRULE rule 1 action modify
set firewall modify BALRULE rule 1 modify lb-group BALANCE

set interfaces ethernet eth0 firewall in modify BALRULE

set load-balance group BALANCE interface eth1

set load-balance group BALANCE interface eth2

 

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

I just read back over the info in the load balance instructions and it does actually explain how to work around this. I obviously didn't read it thoroughly enough!!

Thanks for your help @CowboyJed 

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

[ Edited ]

For anyone that has the same problem. This is the extra confg to fix it

set firewall group network-group LAN-NETS network 10.0.0.0/8

set firewall modify BALANCE rule 10 destination group network-group LAN-NETS
set firewall modify BALANCE rule 10 action modify
set firewall modify BALANCE rule 10 modify table main



set firewall modify BALANCE rule 20 action modify
set firewall modify BALANCE rule 20 modify lb-group BALGROUP

set interfaces ethernet eth0 firewall in modify BALANCE
set interfaces ethernet eth0 vif 10 firewall in modify BALANCE
set interfaces ethernet eth0 vif 11 firewall in modify BALANCE
set interfaces ethernet eth0 vif 12 firewall in modify BALANCE
set interfaces ethernet eth0 vif 13 firewall in modify BALANCE
set interfaces ethernet eth0 vif 14 firewall in modify BALANCE
set interfaces ethernet eth0 vif 15 firewall in modify BALANCE

set load-balance group BALGROUP interface eth1

set load-balance group BALGROUP interface eth2

 

 It tells the router to use the main route table for LAN networks so they don't route out the WAN

Established Member
Posts: 1,043
Registered: ‎02-17-2014
Kudos: 388
Solutions: 40

Re: VLAN routing

Glad to help.  Not sure that I really helped much though.  Other than for some encouragement.  LOL

Member
Posts: 209
Registered: ‎04-22-2013
Kudos: 233
Solutions: 2

Re: VLAN routing

haha yeah but its good to bounce ideas off people. It also helped to know that your set up worked as expected.