VLAN tagging on Unifi switch not working

[cubbz]

Hey guys, I've searched and read different posts on similar topics, most of which have a config far more complex than mine. It's actually very simple.

I've read your Support articles on networking and vlan tagging.

I have a pfsense firewall, dual nics, lan & wan.

I have 3 unifi switches, all 8 ports 150w poe, for completeness sakes called "dinner", "living", "upstairs". The firewall, a server and my pc are connected to the "dinner" switch. There are 2 AP's, connected to the other 2 switches. Out of scope for this as far as I'm concerned.

the server is running the controller. Switches firmware is 3.9.54.9373, and Unifi software is version 5.9.29.

On pfsense, I created a new VLAN 50, with ID 50 and configured the interface with IP 192.168.50.1. Created a firewall rule to allow all.

Basically, I created a network, vlan only, called vlan50 and id 50. Next I created a profile, vlan50 with native network vlan50.

I don't think its necessary, but I created a trunk profile with LAN and VLAN50 selected. The reason I don't think its necessary is because by default (I checked), the port config on the switches have all profiles selected so by default, so all ports are trunks?

On CentOS, on my pc in virtual box, I created bridged networking and made this config:

VLAN=yes
TYPE=Vlan
PHYSDEV=enp0s3
VLAN_ID=50
REORDER_HDR=yes
GVRP=no
MVRP=no
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
IPADDR=192.168.50.10
PREFIX=24
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=vlan50
UUID=3d4576d5-da57-4745-b909-3b3340a42d7c
DEVICE=enp0s3.50
ONBOOT=yes

On pfsense I see this with tcpdump:

tcpdump: listening on re1.50, link-type EN10MB (Ethernet), capture size 262144 bytes
13:12:08.750422 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.10, length 46
13:12:08.750447 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.50.1 is-at 00:01:2e:78:04:f5, length 28
13:12:09.752472 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.10, length 46
13:12:09.752495 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.50.1 is-at 00:01:2e:78:04:f5, length 28
13:12:10.754608 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.10, length 46
13:12:10.754633 ARP, Ethernet (len 6), IPv4 (len 4), Reply 192.168.50.1 is-at 00:01:2e:78:04:f5, length 28

But I do not get a reply and I cannot reach anything else from the client.

Reply 192.168.50.1 is-at 00:01:2e:78:04:f5

Is actually correct, that mac address is the re1 interface on my pfsense box.

I have already posted on the Netgate forums to ask to confirm my VLAN config is correct, and it looks like it is. I'm pretty sure my centOS config is correct too.

Is my switch/VLAN config in Unifi correct? Could someone please help me troubleshoot and fix this?

Thanks!

[SprockTech · Accepted solution]

@cubbz,

If I were you I would just create a new site in your UniFi controller, factory reset your switch and re-adopt it into that new site so you can start from scratch. It sounds like things could be in a weird state, so any further suggestions may or may not work depending on what changes you've made so far. Worst case is you can just move the switch back to your original site to get that config back if you want it.

With that said, @depasseg is right, besides the default network (VLAN1 untagged, marked as "Corporate") all of the other networks you make in the UniFi controller should be "VLAN Only". Assuming that VLAN50 is tagged on the pfSense interface, then I would just add VLAN50 and "VLAN Only" to the UniFi controller. Then from that point you can do two things:

• Leave all ports as trunk ports (the default) and configure your PC/server to use the proper VLAN ID
• Change the port profile that the PC/server is plugged into to the VLAN50 network and turn off VLAN tagging on the PC/server
  • Note that this native network port profile is automatically created for you when you create a network in UniFi

Make sense? Hope it does. If not, reply back. 

If all goes well, then move your other switches/APs to this new site, create your SSIDs, etc, and abandon/delete the other site.

--

Klint

[cubbz]

Here's a dump from the client while the ping was running in another terminal:

tcpdump: listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
14:39:41.930296 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.113, length 46
14:39:42.100801 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:42.205517 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.103, length 46
14:39:42.821884 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.1 (ff:ff:ff:ff:ff:ff) tell 172.16.10.156, length 46
14:39:42.928984 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.113, length 46
14:39:43.050400 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.114, length 46
14:39:43.103602 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:44.045721 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.114, length 46
14:39:44.087673 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.25 tell 172.16.10.162, length 46
14:39:44.087683 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.26 tell 172.16.10.162, length 46
14:39:44.105009 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:44.865275 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.25 tell 172.16.10.162, length 46
14:39:44.865284 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.26 tell 172.16.10.162, length 46
14:39:45.045539 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.114, length 46
14:39:45.865749 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.25 tell 172.16.10.162, length 46
14:39:45.865761 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.26 tell 172.16.10.162, length 46
14:39:46.103501 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:46.982038 IP (tos 0x0, ttl 64, id 6838, offset 0, flags [DF], proto UDP (17), length 323)
    172.16.10.186.36682 > 172.16.10.255.21027: [udp sum ok] UDP, length 295
14:39:47.106551 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:48.084188 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.156, length 46
14:39:48.109377 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:49.084293 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.156, length 46
14:39:50.084301 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.156, length 46
14:39:50.104693 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:50.575609 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.104, length 46
14:39:50.812860 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 152) fe80::201:2eff:fe78:4f5 > ff02::1: [icmp6 sum ok] ICMP6, router advertisement, length 152
	hop limit 64, Flags [managed, other stateful], pref medium, router lifetime 60s, reachable time 0ms, retrans time 0ms
	  prefix info option (3), length 32 (4): fd12:19f1:239f:3a6f::/64, Flags [onlink, auto, router], valid time 86400s, pref. time 14400s
	    0x0000:  40e0 0001 5180 0000 3840 0000 0000 fd12
	    0x0010:  19f1 239f 3a6f 0000 0000 0000 0000
	  route info option (24), length 24 (3):  ::/0, pref=medium, lifetime=60s
	    0x0000:  0000 0000 003c 0000 0000 0000 0000 0000
	    0x0010:  0000 0000 0000
	  rdnss option (25), length 24 (3):  lifetime 20s, addr: fd12:19f1:239f:3a6f::1
	    0x0000:  0000 0000 0014 fd12 19f1 239f 3a6f 0000
	    0x0010:  0000 0000 0001
	  dnssl option (31), length 40 (5):  lifetime 20s, domain(s): internal.mydomain.com.
	    0x0000:  0000 0000 0014 0869 6e74 6572 6e61 6c0b
	    0x0010:  7072 6976 6174 6562 6974 7303 6e65 7400
	    0x0020:  0000 0000 0000
	  mtu option (5), length 8 (1):  1500
	    0x0000:  0000 0000 05dc
	  source link-address option (1), length 8 (1): 00:01:2e:78:04:f5
	    0x0000:  0001 2e78 04f5
14:39:50.845458 IP6 (hlim 1, next-header UDP (17) payload length: 108) fe80::84f7:949b:9751:1baa.546 > ff02::1:2.547: [bad udp cksum 0xca92 -> 0x47f5!] dhcp6 solicit (xid=dd5f80 (client-ID type 4) (option-request DNS-server DNS-search-list DNS-server DNS-search-list client-ID) (elapsed-time 3415) (Client-FQDN) (IA_NA IAID:666206221 T1:3600 T2:5400))
14:39:51.093836 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 330)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 302
14:39:51.106354 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:51.116226 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 377)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 349
14:39:51.144523 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 321)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 293
14:39:51.158795 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 385)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 357
14:39:51.180479 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 387)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 359
14:39:51.202432 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 375)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 347
14:39:51.225470 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 330)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 302
14:39:51.236811 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 08:00:27:b5:80:0d, length 300, xid 0x86373e17, secs 40, Flags [none] (0x0000)
	  Client-Ethernet-Address 08:00:27:b5:80:0d
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message Option 53, length 1: Discover
	    Requested-IP Option 50, length 4: 172.16.10.196
	    Hostname Option 12, length 8: "centos01"
	    Parameter-Request Option 55, length 18: 
	      Subnet-Mask, BR, Time-Zone, Classless-Static-Route
	      Domain-Name, Domain-Name-Server, Hostname, YD
	      YS, NTP, MTU, Option 119
	      Default-Gateway, Classless-Static-Route, Classless-Static-Route-Microsoft, Static-Route
	      Option 252, NTP
	    END Option 255, length 0
	    PAD Option 0, length 0, occurs 20
14:39:51.245543 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 377)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 349
14:39:51.267740 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 321)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 293
14:39:51.289074 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 385)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 357
14:39:51.310752 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 387)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 359
14:39:51.335803 IP (tos 0x0, ttl 2, id 0, offset 0, flags [DF], proto UDP (17), length 375)
    172.16.10.187.1900 > 239.255.255.250.1900: [udp sum ok] UDP, length 347
14:39:51.575450 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.104, length 46
14:39:52.005551 IP (tos 0x0, ttl 128, id 32446, offset 0, flags [none], proto UDP (17), length 72)
    172.16.10.162.57621 > 172.16.10.255.57621: [udp sum ok] UDP, length 44
14:39:52.108636 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:52.575684 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.104, length 46
14:39:53.488896 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.1 (ff:ff:ff:ff:ff:ff) tell 172.16.10.113, length 46
14:39:53.629994 IP (tos 0x0, ttl 128, id 32448, offset 0, flags [none], proto UDP (17), length 305)
    172.16.10.162.63983 > 172.16.10.255.21027: [udp sum ok] UDP, length 277
14:39:53.630017 IP6 (flowlabel 0xdeb65, hlim 1, next-header UDP (17) payload length: 285) fe80::74ab:e9b5:eca0:7349.63982 > ff12::8384.21027: [udp sum ok] UDP, length 277
14:39:53.630043 IP6 (flowlabel 0xdeb65, hlim 1, next-header UDP (17) payload length: 285) fe80::74ab:e9b5:eca0:7349.63982 > ff12::8384.21027: [udp sum ok] UDP, length 277
14:39:53.630047 IP6 (flowlabel 0xdeb65, hlim 1, next-header UDP (17) payload length: 285) fe80::74ab:e9b5:eca0:7349.63982 > ff12::8384.21027: [udp sum ok] UDP, length 277
14:39:53.630051 IP6 (flowlabel 0xdeb65, hlim 1, next-header UDP (17) payload length: 285) fe80::74ab:e9b5:eca0:7349.63982 > ff12::8384.21027: [udp sum ok] UDP, length 277
14:39:53.630054 IP6 (flowlabel 0xdeb65, hlim 1, next-header UDP (17) payload length: 285) fe80::74ab:e9b5:eca0:7349.63982 > ff12::8384.21027: [udp sum ok] UDP, length 277
14:39:53.630132 IP6 (flowlabel 0xdeb65, hlim 1, next-header UDP (17) payload length: 285) fe80::74ab:e9b5:eca0:7349.63982 > ff12::8384.21027: [udp sum ok] UDP, length 277
14:39:53.700715 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.1 (ff:ff:ff:ff:ff:ff) tell 172.16.10.114, length 46
14:39:53.739636 LLDP, length 90
	Chassis ID TLV (1), length 7
	  Subtype MAC address (4): f0:9f:c2:10:10:e1
	  0x0000:  04f0 9fc2 1010 e1
	Port ID TLV (2), length 4
	  Subtype Local (7): 0/7
	  0x0000:  0730 2f37
	Time to Live TLV (3), length 2: TTL 120s
	  0x0000:  0078
	Port Description TLV (4), length 6: Port 7
	  0x0000:  506f 7274 2037
	System Name TLV (5), length 15: switch-dinner
	  0x0000:  7377 6974 6368 2d65 6574 6b61 6d65 72
	System Description TLV (6), length 36
	  USW-8P-150, 3.9.54.9373, Linux 3.6.5
	  0x0000:  5553 572d 3850 2d31 3530 2c20 332e 392e
	  0x0010:  3534 2e39 3337 332c 204c 696e 7578 2033
	  0x0020:  2e36 2e35
	System Capabilities TLV (7), length 4
	  System  Capabilities [Bridge] (0x0004)
	  Enabled Capabilities [Bridge] (0x0004)
	  0x0000:  0004 0004
	End TLV (0), length 0
14:39:54.110608 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:55.029515 IP (tos 0x0, ttl 4, id 13994, offset 0, flags [DF], proto UDP (17), length 542)
    172.16.10.148.42009 > 239.255.255.250.1900: [udp sum ok] UDP, length 514
14:39:55.033736 IP (tos 0x0, ttl 4, id 13995, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.42009 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.036366 IP (tos 0x0, ttl 4, id 13996, offset 0, flags [DF], proto UDP (17), length 594)
    172.16.10.148.42009 > 239.255.255.250.1900: [udp sum ok] UDP, length 566
14:39:55.038724 IP (tos 0x0, ttl 4, id 13997, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.44699 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.040659 IP (tos 0x0, ttl 4, id 13998, offset 0, flags [DF], proto UDP (17), length 598)
    172.16.10.148.44699 > 239.255.255.250.1900: [udp sum ok] UDP, length 570
14:39:55.042674 IP (tos 0x0, ttl 4, id 13999, offset 0, flags [DF], proto UDP (17), length 596)
    172.16.10.148.35323 > 239.255.255.250.1900: [udp sum ok] UDP, length 568
14:39:55.044116 IP (tos 0x0, ttl 4, id 14000, offset 0, flags [DF], proto UDP (17), length 608)
    172.16.10.148.38867 > 239.255.255.250.1900: [udp sum ok] UDP, length 580
14:39:55.048468 IP (tos 0x0, ttl 4, id 14001, offset 0, flags [DF], proto UDP (17), length 606)
    172.16.10.148.47675 > 239.255.255.250.1900: [udp sum ok] UDP, length 578
14:39:55.048487 IP (tos 0x0, ttl 4, id 14002, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.44073 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.050476 IP (tos 0x0, ttl 4, id 14003, offset 0, flags [DF], proto UDP (17), length 598)
    172.16.10.148.44073 > 239.255.255.250.1900: [udp sum ok] UDP, length 570
14:39:55.052087 IP (tos 0x0, ttl 4, id 14004, offset 0, flags [DF], proto UDP (17), length 600)
    172.16.10.148.54757 > 239.255.255.250.1900: [udp sum ok] UDP, length 572
14:39:55.054000 IP (tos 0x0, ttl 4, id 14005, offset 0, flags [DF], proto UDP (17), length 598)
    172.16.10.148.54944 > 239.255.255.250.1900: [udp sum ok] UDP, length 570
14:39:55.055928 IP (tos 0x0, ttl 4, id 14006, offset 0, flags [DF], proto UDP (17), length 600)
    172.16.10.148.44532 > 239.255.255.250.1900: [udp sum ok] UDP, length 572
14:39:55.060302 IP (tos 0x0, ttl 4, id 14007, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.46734 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.061447 IP (tos 0x0, ttl 4, id 14008, offset 0, flags [DF], proto UDP (17), length 594)
    172.16.10.148.46734 > 239.255.255.250.1900: [udp sum ok] UDP, length 566
14:39:55.062719 IP (tos 0x0, ttl 4, id 14009, offset 0, flags [DF], proto UDP (17), length 606)
    172.16.10.148.33746 > 239.255.255.250.1900: [udp sum ok] UDP, length 578
14:39:55.063827 IP (tos 0x0, ttl 4, id 14010, offset 0, flags [DF], proto UDP (17), length 608)
    172.16.10.148.39098 > 239.255.255.250.1900: [udp sum ok] UDP, length 580
14:39:55.065059 IP (tos 0x0, ttl 4, id 14011, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.55489 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.066251 IP (tos 0x0, ttl 4, id 14012, offset 0, flags [DF], proto UDP (17), length 592)
    172.16.10.148.55489 > 239.255.255.250.1900: [udp sum ok] UDP, length 564
14:39:55.067512 IP (tos 0x0, ttl 4, id 14013, offset 0, flags [DF], proto UDP (17), length 582)
    172.16.10.148.40727 > 239.255.255.250.1900: [udp sum ok] UDP, length 554
14:39:55.112258 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:55.137680 IP (tos 0x0, ttl 4, id 14022, offset 0, flags [DF], proto UDP (17), length 542)
    172.16.10.148.48085 > 239.255.255.250.1900: [udp sum ok] UDP, length 514
14:39:55.137702 IP (tos 0x0, ttl 4, id 14023, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.48085 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.140768 IP (tos 0x0, ttl 4, id 14024, offset 0, flags [DF], proto UDP (17), length 594)
    172.16.10.148.48085 > 239.255.255.250.1900: [udp sum ok] UDP, length 566
14:39:55.141572 IP (tos 0x0, ttl 4, id 14025, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.44011 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.142718 IP (tos 0x0, ttl 4, id 14026, offset 0, flags [DF], proto UDP (17), length 598)
    172.16.10.148.44011 > 239.255.255.250.1900: [udp sum ok] UDP, length 570
14:39:55.143835 IP (tos 0x0, ttl 4, id 14027, offset 0, flags [DF], proto UDP (17), length 596)
    172.16.10.148.53804 > 239.255.255.250.1900: [udp sum ok] UDP, length 568
14:39:55.149332 IP (tos 0x0, ttl 4, id 14028, offset 0, flags [DF], proto UDP (17), length 608)
    172.16.10.148.48237 > 239.255.255.250.1900: [udp sum ok] UDP, length 580
14:39:55.151837 IP (tos 0x0, ttl 4, id 14029, offset 0, flags [DF], proto UDP (17), length 606)
    172.16.10.148.59330 > 239.255.255.250.1900: [udp sum ok] UDP, length 578
14:39:55.154954 IP (tos 0x0, ttl 4, id 14030, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.37272 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.159330 IP (tos 0x0, ttl 4, id 14031, offset 0, flags [DF], proto UDP (17), length 598)
    172.16.10.148.37272 > 239.255.255.250.1900: [udp sum ok] UDP, length 570
14:39:55.161384 IP (tos 0x0, ttl 4, id 14032, offset 0, flags [DF], proto UDP (17), length 600)
    172.16.10.148.48142 > 239.255.255.250.1900: [udp sum ok] UDP, length 572
14:39:55.163873 IP (tos 0x0, ttl 4, id 14033, offset 0, flags [DF], proto UDP (17), length 598)
    172.16.10.148.34308 > 239.255.255.250.1900: [udp sum ok] UDP, length 570
14:39:55.165182 IP (tos 0x0, ttl 4, id 14034, offset 0, flags [DF], proto UDP (17), length 600)
    172.16.10.148.33365 > 239.255.255.250.1900: [udp sum ok] UDP, length 572
14:39:55.166383 IP (tos 0x0, ttl 4, id 14035, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.48212 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.169097 IP (tos 0x0, ttl 4, id 14036, offset 0, flags [DF], proto UDP (17), length 594)
    172.16.10.148.48212 > 239.255.255.250.1900: [udp sum ok] UDP, length 566
14:39:55.169106 IP (tos 0x0, ttl 4, id 14037, offset 0, flags [DF], proto UDP (17), length 606)
    172.16.10.148.54962 > 239.255.255.250.1900: [udp sum ok] UDP, length 578
14:39:55.170284 IP (tos 0x0, ttl 4, id 14038, offset 0, flags [DF], proto UDP (17), length 608)
    172.16.10.148.45418 > 239.255.255.250.1900: [udp sum ok] UDP, length 580
14:39:55.172449 IP (tos 0x0, ttl 4, id 14039, offset 0, flags [DF], proto UDP (17), length 551)
    172.16.10.148.50282 > 239.255.255.250.1900: [udp sum ok] UDP, length 523
14:39:55.173034 IP (tos 0x0, ttl 4, id 14040, offset 0, flags [DF], proto UDP (17), length 592)
    172.16.10.148.50282 > 239.255.255.250.1900: [udp sum ok] UDP, length 564
14:39:55.173960 IP (tos 0x0, ttl 4, id 14041, offset 0, flags [DF], proto UDP (17), length 582)
    172.16.10.148.52523 > 239.255.255.250.1900: [udp sum ok] UDP, length 554
14:39:55.216924 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.103, length 46
14:39:55.952001 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.113, length 46
14:39:56.117870 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:56.217172 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.103, length 46
14:39:56.951549 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.113, length 46
14:39:57.217881 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.103, length 46
14:39:57.428221 IP (tos 0x0, ttl 64, id 28319, offset 0, flags [DF], proto UDP (17), length 72)
    172.16.10.10.57621 > 172.16.10.255.57621: [udp sum ok] UDP, length 44
14:39:57.684851 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 152) fe80::201:2eff:fe78:4f5 > ff02::1: [icmp6 sum ok] ICMP6, router advertisement, length 152
	hop limit 64, Flags [managed, other stateful], pref medium, router lifetime 60s, reachable time 0ms, retrans time 0ms
	  prefix info option (3), length 32 (4): fd12:19f1:239f:3a6f::/64, Flags [onlink, auto, router], valid time 86400s, pref. time 14400s
	    0x0000:  40e0 0001 5180 0000 3840 0000 0000 fd12
	    0x0010:  19f1 239f 3a6f 0000 0000 0000 0000
	  route info option (24), length 24 (3):  ::/0, pref=medium, lifetime=60s
	    0x0000:  0000 0000 003c 0000 0000 0000 0000 0000
	    0x0010:  0000 0000 0000
	  rdnss option (25), length 24 (3):  lifetime 20s, addr: fd12:19f1:239f:3a6f::1
	    0x0000:  0000 0000 0014 fd12 19f1 239f 3a6f 0000
	    0x0010:  0000 0000 0001
	  dnssl option (31), length 40 (5):  lifetime 20s, domain(s): internal.mydomain.com.
	    0x0000:  0000 0000 0014 0869 6e74 6572 6e61 6c0b
	    0x0010:  7072 6976 6174 6562 6974 7303 6e65 7400
	    0x0020:  0000 0000 0000
	  mtu option (5), length 8 (1):  1500
	    0x0000:  0000 0000 05dc
	  source link-address option (1), length 8 (1): 00:01:2e:78:04:f5
	    0x0000:  0001 2e78 04f5
14:39:57.951680 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.113, length 46
14:39:58.068151 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.114, length 46
14:39:58.125198 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:58.659466 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.1 (ff:ff:ff:ff:ff:ff) tell 172.16.10.103, length 46
14:39:59.068468 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.114, length 46
14:39:59.126840 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:39:59.309134 IP (tos 0x0, ttl 64, id 28448, offset 0, flags [DF], proto UDP (17), length 298)
    172.16.10.10.53340 > 172.16.10.255.21027: [udp sum ok] UDP, length 270
14:40:00.068095 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.16.10.254 tell 172.16.10.114, length 46
14:40:00.129107 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:40:02.122090 IP6 (hlim 1, next-header Options (0) payload length: 56) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 2 group record(s) [gaddr ff02::1:ff87:8782 to_in { }] [gaddr ff02::1:ff51:1baa to_ex { }]
14:40:02.129679 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.68 > 255.255.255.255.67: [udp sum ok] BOOTP/DHCP, Request from 08:00:27:b5:80:0d, length 300, xid 0xb6f9e717, Flags [none] (0x0000)
	  Client-Ethernet-Address 08:00:27:b5:80:0d
	  Vendor-rfc1048 Extensions
	    Magic Cookie 0x63825363
	    DHCP-Message Option 53, length 1: Discover
	    Requested-IP Option 50, length 4: 172.16.10.196
	    Hostname Option 12, length 8: "centos01"
	    Parameter-Request Option 55, length 18: 
	      Subnet-Mask, BR, Time-Zone, Classless-Static-Route
	      Domain-Name, Domain-Name-Server, Hostname, YD
	      YS, NTP, MTU, Option 119
	      Default-Gateway, Classless-Static-Route, Classless-Static-Route-Microsoft, Static-Route
	      Option 252, NTP
	    END Option 255, length 0
	    PAD Option 0, length 0, occurs 20
14:40:02.130587 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 192.168.50.1 tell 192.168.50.99, length 28
14:40:02.855660 IP6 (hlim 1, next-header Options (0) payload length: 56) :: > ff02::16: HBH (rtalert: 0x0000) (padn) [icmp6 sum ok] ICMP6, multicast listener report v2, 2 group record(s) [gaddr ff02::1:ff87:8782 to_in { }] [gaddr ff02::1:ff51:1baa to_ex { }]
14:40:02.986257 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 24) :: > ff02::1:ff51:1baa: [icmp6 sum ok] ICMP6, neighbor solicitation, length 24, who has fe80::84f7:949b:9751:1baa
^C
119 packets captured
119 packets received by filter
0 packets dropped by kernel

[svons]

Yes all ports are trunk ports with the default "ALL" selected.

So far it does seem like a config issue if i had to gues

I will assume that yourt LAN subnet is set to 172.16.10.0/24?

so far if i had to guess - you have the native vlan set to 50 on the switch port - maybe set it back to lan and let the vlan tagging on the pfsense nic move it to vlan 50 instead of tagging it on the nic and then having it native on the switch port. In your current config you would not need to tag 50 on the pfsense nic for it to work.

Alternatively

Can you try a quick test.

Instead of creating a vlan of 50 on the unifi, can you delete that.

Create a new Corporate network and tag it on vlan 50.

Set the unifi to have an available vlan 50 ip and disable dhcp - that can still be hosted on the pfsense or whatever device you want it on or if it is hosted on the unifi point the default gateway to your pfsense at .1

If you want to leave pfsense tagging its nic to 50 - then set its switchport profile back to ALL and lets the blan tagging work. Or remove the tagging and make 50 the native vlan on the port and lets the switch force all traffic onto that subnet.

Do you have the same results?

[cubbz]

 Thanks for your reply. My answers are inline.

---

@svonswrote:

Yes all ports are trunk ports with the default "ALL" selected.

So far it does seem like a config issue if i had to gues

I will assume that yourt LAN subnet is set to 172.16.10.0/24?

---

Thanks for confirming, and yes, that's my LAN subnet.

---

@svons wrote:

so far if i had to guess - you have the native vlan set to 50 on the switch port - maybe set it back to lan and let the vlan tagging on the pfsense nic move it to vlan 50 instead of tagging it on the nic and then having it native on the switch port. In your current config you would not need to tag 50 on the pfsense nic for it to work.

---

 Since all switch ports are set to default ALL, I haven't changed any port config. All I did was create the network and profile in the Unifi manager/controller.

---

@svons wrote:

Alternatively

Can you try a quick test.

Instead of creating a vlan of 50 on the unifi, can you delete that.

Create a new Corporate network and tag it on vlan 50.

Set the unifi to have an available vlan 50 ip and disable dhcp - that can still be hosted on the pfsense or whatever device you want it on or if it is hosted on the unifi point the default gateway to your pfsense at .1

If you want to leave pfsense tagging its nic to 50 - then set its switchport profile back to ALL and lets the blan tagging work. Or remove the tagging and make 50 the native vlan on the port and lets the switch force all traffic onto that subnet.

Do you have the same results?

---

 Oke I created the corporate network, tag it 50 but that doesn't change anything at all. Same result on the tcpdump on both the firewall and the client.

What confuses me though about your reply, is you say that if want to leave pfsense tagging its nic to 50, what do you mean by that? I NEED to create a vlan and interface on pfsense to have a VLAN on my firewall to do policy based routing. This is not optional, but maybe I misunderstand what you're trying to say?

Thanks so far!

Ps. I think/suspect you think I have a USG, I don't.

[svons]

@cubbz What confuses me though about your reply, is you say that if want to leave pfsense tagging its nic to 50, what do you mean by that? I NEED to create a vlan and interface on pfsense to have a VLAN on my firewall to do policy based routing. This is not optional, but maybe I misunderstand what you're trying to say?

Thanks so far!

Yes i see waht you mean with your setup.


So you are using both ports ion the pfsense - 1 WAN  and he other LAN

i forgot you are NOT using a usg also.

If you set your unifi settings back so  that vlan 50 is a VLAN only - do you have two clients that are on vlan 50 that you could test with? just verifying vlan 50 is being acknowledged on the switches?

For your clients how are you getting them onto vlan 50? connecting to an SSID that is forcing them to use 50 or on a untagged vlan 50 switchport?

[cubbz]

Everything is wired, the AP's are not in play here.

The client is a virtual machine on my pc, connected to a switchport. The firewall pfsense is connected to the same switch.

All ports are configured with ALL profiles. I've changed nothing since all means all ports handle all vlans.

The virtual machine is configured in bridge mode (contrary to NAT) and is configured as specified in my first post. Its running CentOS 7.5 with a vlan tagging 50. It has a static IP in the same subnet as pfsense vlan 50 interface (192.168.50.0/24). If I don't configure the vlan and use dhcp, I'll get a 172.16. address from dhcp in my LAN.

As you can see in the tcpdumps, I think level 2 is functional on a network level, the ARP is getting through to the firewall and I get some weird replies back as you can see in the tcpdump on the client.

[svons]

yea it sounds all correct  the tagging looks good on your client pc example - I am just saying prove it is the Unifi setup by connecting 2 other devices on vlan 50 and verify they have the same issue. Then you will know for sure. If they cant communicate then you can check other pfsense network settings.

testing with forcing a client to 50 by setting the siwtchport to untagged 50 and connecting to see if that makes any difference would potentially help too! 

sorry i am not more help! I use 15+ vlans on 10 or so different Unifi networks but generally dont use vlan tagging on the NIC of the devices with a few exceptions - but they were able to route properly.

[cubbz]

Thanks.

I tried configuring my 2nd nic in my pc (usually disabled) on dhcp (no vlan id tagging on the pc) and just set the switchport #7 to which its connected to vlan50. It doesn't get a dhcp address and when I configure a  fixed ip it can't ping the firewall.

I don't think vlan tagging on the switches is working.

[svons]

that test still relies on the pfsense to be functioning properly.

When setting a fixed IP on a device - i think you may need the USG in the setup (but i am not sure i have not tested this function where a USG is not already hosting dhcp - the setting creates a dhcp reservation which you cannot due because dhcp is hosted on a different device) - a better test would potentially be setting a static ip on the 2nd nic that is plugged into port 7.

So you set switch port 7 to untagged vlan 50  have your 2nd nic with a static address - i would be interested if you had a 3rd device to test with instead of the pfsense - or if you have a way to test vlan tagging on the pfsense specifically and verify it is working.

Good luck!

[svons]

Or just skip the switch all together.

Plug your test pc that has vlan 50 tagged on it directly into your pfsense on the port it has vlan 50 tagged.

That would be a good test to prove the switch is not VLAN tagging properly like you think

[cubbz]

I need a moment to test that as it would disconnect the internet.

I see if I can test that tomorrow, also will try with 2 physical devices on the switch on 2 ports with vlan 50 configured.

Thanks for the help so far.

If anyone reading this thread has ideas please share them as well.

[depasseg]

A couple of things, if you don't have a USG, then the "corporate settings" doen't apply and you have effectively the same thing as a vlan only network.
You don't need to create a profile, nor apply it to the switch port, because in your example, you want the vlan 50 traffi c to pass as tagged. In your OP, you mentioned that you had a native network set to vlan 50. This means that any nic plugged into that port would be connected t o vlan 50. Just leave the ports as default on the switches.
The second you create a new vlan or corporate network, the switches and APs should all get provisioned with this new information. If you look at the device screen right after hitting save, confirm that they are being provisioned.
Try connecting a DHCP client to the trunk port and see if that pulls the correct native network. Then try changing the switch port to be vlan 50 (it's created automatically) and then reconnect the network cable.

You unfortunately have many potential areas for configuration issue, but it's safe to assume that every unifi switch (and AP), has the native and tagged vlans that you have configured. I have many vlans spread out across many sites and I haven't had an issue. But I also use a USG which makes that side of the path simple. I've also created trunks and other native ports for other switches and server infrastructure. I have had issues with Windows and guest VMs though (pain with Windows). ESXi, and linux boxes have been perfectly fine though.

[eshwayri]

First, disclaimer, I have used this product for only about 12 hours now; however, strangely enough we have a similar setup.  The first thing I would suggest you check is your firewall rule in pfsense.  The allow any rule defaults to TCP only; make sure you changed the protocol to Any.  It took me 10 minutes before I caught that stupid mistake.  On the controller side, I am not exactly sure what you did.  When you create a VLAN only network, it automatically creates a profile with the same name, set as a native / access port (non-tagged) in that VLAN.  That auto-created profile cannot be changed.  If you want to create a profile with a tagged vlan, which is what it sounds like you want to feed pfsense, you will need to create a profile with another name where you use a different network as native (or set to none), and tick the box next to your vlan50 to trunk it tagged.  From google, it seems All is supposed to trunk everything (probably everything except the native vlan assigned to the port), but I don't trust defaults like that; anyway, my setup is more complex with a bunch of vlans I need trunked through -- and a bunch  I don't.  Still testing to be honest.

[SprockTech · Accepted solution]

@cubbz,

If I were you I would just create a new site in your UniFi controller, factory reset your switch and re-adopt it into that new site so you can start from scratch. It sounds like things could be in a weird state, so any further suggestions may or may not work depending on what changes you've made so far. Worst case is you can just move the switch back to your original site to get that config back if you want it.

With that said, @depasseg is right, besides the default network (VLAN1 untagged, marked as "Corporate") all of the other networks you make in the UniFi controller should be "VLAN Only". Assuming that VLAN50 is tagged on the pfSense interface, then I would just add VLAN50 and "VLAN Only" to the UniFi controller. Then from that point you can do two things:

• Leave all ports as trunk ports (the default) and configure your PC/server to use the proper VLAN ID
• Change the port profile that the PC/server is plugged into to the VLAN50 network and turn off VLAN tagging on the PC/server
  • Note that this native network port profile is automatically created for you when you create a network in UniFi

Make sense? Hope it does. If not, reply back. 

If all goes well, then move your other switches/APs to this new site, create your SSIDs, etc, and abandon/delete the other site.

--

Klint

[tording99z28]

On your CentOS box I see you have defroute set to yes but it doesn't have an actual gateway set for that default route anywhere.  On all my *nix boxes I have ever setup I always set a default gateway of the .1 address of my router/firewall.  I am doing something similar to what you have configured here and I have had no issues.  Only difference is I am running an ER-X instead of pfsense. 

Do a netstat -rn on your CentOS box and see what it has for a gateway for your defaule route. 

[cubbz]

---

@tording99z28 wrote:

On your CentOS box I see you have defroute set to yes but it doesn't have an actual gateway set for that default route anywhere.  On all my *nix boxes I have ever setup I always set a default gateway of the .1 address of my router/firewall.  I am doing something similar to what you have configured here and I have had no issues.  Only difference is I am running an ER-X instead of pfsense. 

Do a netstat -rn on your CentOS box and see what it has for a gateway for your defaule route. 

---

Thanks, but default route has no effect on the same local network.

My pings reach pfsense and sends back replies, but the replies never reach my guest vm.

I have removed all networks and profiles, reprovisioned LAN as the only and default network. I recreated vlan50 and configured all switch ports to LAN instead of all. Only the pfense port has ALL and port 2 with a single client is on vlan50. Still doesnt work, im troubleshooting as we speak.

[SprockTech]

@cubbz,

Just to clarify, without a route your CentOS box will only be aware of it's own network, which is 192.168.50.0/24.

I only see ARP traffic in your tcpdumps, not ICMP pings.

Is your host machine, or some other machine, able to use VLAN50 properly? Sorry if you answered that already. Are you able to set the VLAN ID at the host level for that VM?

--

Klint

[tording99z28]

I assumed you were trying to reach something on your LAN which is on the 172.16.10.0/24 network you said which is a different subnet than the 192.168.50.0/24 network so it would matter for that.  

[cubbz]

---

@tording99z28 wrote:

I assumed you were trying to reach something on your LAN which is on the 172.16.10.0/24 network you said which is a different subnet than the 192.168.50.0/24 network so it would matter for that.  

---

Sorry if I was unclear. My LAN is that subnet, but for VLAN 50 I just want the client to be able to reach the firewall (pfsense), nothing else for the moment.

---

@SprockTech wrote:

@cubbz,

Just to clarify, without a route your CentOS box will only be aware of it's own network, which is 192.168.50.0/24.

---

 That's fine. I just want to ping and communicate with pfsense box at this moment. Once I get that working, I'll work in the rest.

---

@SprockTech wrote:

I only see ARP traffic in your tcpdumps, not ICMP pings.

---

 That is one of the weird things and the reason I posted them because I was hoping it would help understand what is going wrong.

---

@SprockTech wrote:

Is your host machine, or some other machine, able to use VLAN50 properly? Sorry if you answered that already. Are you able to set the VLAN ID at the host level for that VM?

--

Klint

---

---

 No. That's why I removed all networks and profiles, reprovisioned everything and testing with a physical machine connected to the switch. If I configure this port as vlan50, it should tag that traffic and I don't need a vm with bridged config and vlan tagging from the guest. The config doesn't get simpler than this, see screenshots.

I will set all switch ports to LAN, set port 1 to VLAN50 for my laptop from which I am testing. I set the pfsense switchport to VLAN50 + LAN (ALL).

I'll let you know what happens. Got wireshark ready to analyze the dumps.

[SprockTech]

---

@cubbz wrote:

If I configure this port as vlan50, it should tag that traffic and I don't need a vm with bridged config and vlan tagging from the guest. The config doesn't get simpler than this, see screenshots.

---

If you configure a switch port using the VLAN50 profile then VLAN50 will be the native untagged VLAN on that port.

--

Klint