Reply
New Member
Posts: 6
Registered: ‎02-14-2017
Kudos: 2

Re: 3.9.15.8011 blocks DHCP

DHCP guarding is configurable but even when it is disabled DHCP snooping stays active.

New Member
Posts: 27
Registered: ‎12-21-2017
Kudos: 4

Re: 3.9.15.8011 blocks DHCP

I'd love to get some move visibility around the suggestion regarding multicast traffic causing the issue from @shpoellet

 

Is everybody (or a large number of us) that is experiencing the issue using IGMP or something similar with multicast? Or is it a red herring?

New Member
Posts: 15
Registered: ‎05-14-2016
Kudos: 3

Re: 3.9.15.8011 blocks DHCP

There is "a lot" of multicast traffic on my network, but only ~10% PPS compared to Layer3.

New Member
Posts: 16
Registered: ‎09-06-2016

Re: 3.9.15.8011 blocks DHCP

From my experience we are facing more than the DHCP snooping issue.

I disabled DHCP snooping through config properties. But during my test with the latest firmware I started facing issues with Home Kit and even with my Printer. I think both use mDNS to announce themselfes. I get intermittent connection issues. So it could be a broadcast and or multicast issue beneath it.

 

If there would be a solution to disable this behaviour somehow I would gladly update to latest firmwares.

Emerging Member
Posts: 45
Registered: ‎04-09-2015
Kudos: 8
Solutions: 1

Re: 3.9.15.8011 blocks DHCP

Just putting my two cents in.

 

I recently upgraded from 3.9.6 to the latest firmware on my 2 switches (16 and a 48) as well as upgrading the AP AC Pros to the lastest version. I use multicast for internal IPTV on a VLAN. 

 

Randomly; devices connected to the core switch (48) would or wouldnt get an address from the DHCP server (an edgerouter poe 5). But 99% of the time; the devices wouldnt get an address assigned to them. I thought I'd done something wrong during the upgrade, (also went from 5.6 to 5.7 and then to 5.8) and was absolutely certain that in the "Corporate" network in Unifi I had originally set No DHCP as it was handled by the edgerouter etc. I ended up rebuilding my network config (stupidly didnt have easy access to a unifi backup) and hit the issue again. It didnt matter whether I was using Multicast etc.

 

If I connected the core switch the the 16 to a dumb gigabit switch then they'd both get IP addresses etc but would sporadically allow DHCP leases to be given out but most of the time, if a client didnt already have a lease (ie from me putting it on the dumb gigabit switch first) then it would fail. 100% sporadic. I thought I was going mad.

 

Decided to dowgrade the switches to the last known OK firmware. Downgraded and instantly everything got given a lease the way it should have all along - it was like a christmas tree being plugged in.

 

There is 100% a fault with the latest firmware - I havent taken the AP's back down - they seem OK from what I can tell - allowing clients to get DHCP leases etc. But I had to take the switch firmware back to 3.9.6.7613

 

To a degree, I'm happy to help diagnose the issue but when I upgrade, my network dies - I don't have hardware lying around to test with. If Ubiquiti want to give me a hand with an extra switch to be able to help pinpoint this down I'm more than willing to do everything I can to help. 

 

I managed to get back onto the original controller and get the config off that before I reverted down, so I know the config I was using is the same as the config that had the issues - no changes etc. So it 100% is a firmware related issue.

 

 

New Member
Posts: 27
Registered: ‎12-21-2017
Kudos: 4

Re: 3.9.15.8011 blocks DHCP

As I've said in the past and has been tacitly confirmed,  (check witg @UBNT-MikeD & @UBNT-cid) Ubnt do not care about firmware bugs or issues reported either here or via support.

There will be no fixes for issues like this.

The "fix" is to replace the Ubnt hardware with that of a different manafacturer if you want bugs fixed.

In my opinion Ubnt is home hardware and not production ready, and should not be used in production environments.

Emerging Member
Posts: 45
Registered: ‎04-09-2015
Kudos: 8
Solutions: 1

Re: 3.9.15.8011 blocks DHCP

I'll be making a support ticket but doubt it'll go far without me destroying my network and getting some logs.

 

Funnily enough - this is my home network Man Wink

New Member
Posts: 17
Registered: ‎05-06-2014
Kudos: 15

Re: 3.9.15.8011 blocks DHCP

I agree. This is unacceptable. Every UBNT firmware upgrade is a chance to totally break your network. How come we the users are the experts on this issue and have identified exactly the last switch firmware without the problem (3.9.6) after which all future firmwares are broken? Why if we've drilled it down like this can't UBNT quickly and easily identify the code that broke the switches? Only a few changes go into each firmware release, and it shouldn't be hard to track down.

 

Why are we the users the experts on this issue? Why hasn't there been a post by a UBNT employee in ages? This should be an easy fix, but it appears that rolling out new features and new products is more important than actually making the platform bug free.

New Member
Posts: 2
Registered: ‎09-21-2014

Re: 3.9.15.8011 blocks DHCP

Hi all,

 

I've banged my head to the wall for months now with this very problem. It took me a long time to dig up enough information from my customers end users and gather log data to see what is the source of this problem. After a long time, I was able to verify that customer's DHCP relateted problems are caused by Dante's multicast audio traffic.

 

The DHCP problem was tricky. Maybe a few times a week I got reports at completely random times that the network wouldn't connect (wifi not leasing any IPs) and I saw ~100 pcs of 169.254.x.x IP-adresses on the controller. It took me some time to figure out that the problems started exactly at the time when switch interface connected to Dante-multicast streaming device went up. The problem appeared on all networks (VLANs, WLANs), although Dante networks are separate, non-routed VLANs.

 

I tried literally everything during this fight, but the hints in this thread about downgrading all switches to 3.9.6 were finally the point where the network began to work as it should. Until today I always had the newest firmware in UniFi switches, hoping that the new one would fix these problems since I had a feeling that they were switch related.

 

The site's primary network's topology is large but pretty simple: ISP switch - USG Pro as firewall/router - USW 16XG as core switch - many USW 24P250W and USW 48P500W and Cisco/HP switches as access switches - UniFi access points (anything from UAP-Pro to UAP-AC-HD). Most of the actual traffic is generated by Wi-Fi clients, but there are also many wired clients. USG does all routing and firewalling between site's 16 subnets. The Dante nets (primary and secondary) are "VLAN only" nets and most other networks are routed "Corporate" networks.

 

I wrote this message in two moods. First, I'm thankful that I managed to find the problem and was able to create a workaround (downgrade) so service level won't be so poor. Secondly, I'm angry to UBNT for not fixing this bug yet. Hope that it happens soon, since this one customer of mine alone has 70+ UBNT devices in their network and I will have to start recommending Cisco or some other more "reliable" vendor in the future. To be mentioned: I've been very happy with UBNT products and services, until I found this problem.

 

Please UBNT fix this ASAP.

New Member
Posts: 1
Registered: ‎02-19-2014

Re: 3.9.15.8011 blocks DHCP

That is a really interesting observation about Dante triggering the issues.

 

I’ve been following this thread for a while, and gave up hope that Ubiquiti would fix this bug directly. So I’ve implemented the DHCP snooping config workaround mentioned here: https://community.ubnt.com/t5/UniFi-Routing-Switching/Disable-DHCP-Snooping-on-USW/td-p/1510229/page...

 

But the mention of Dante multicast caught my attention. We don’t use Dante specifically, but we do have a similar setup, with separate network specifically for multicast video.

 

While I don’t particularly want to test this on our production network, as I’m thinking back, the times when DHCP started failing seem like they coincided with the times our multicast video server was turned on.

 

Is it possible that certain types of multicast traffic are somehow triggering the improper snooping behavior? And since it seems to span all networks (not just the ones carrying the multicast traffic), could the snooping problem be affecting trunk ports that carry multiple tagged VLANs?

 

If I can find a good time on our production network, I’ll test whether starting our multicast video server actually triggers the DHCP issues.

Emerging Member
Posts: 45
Registered: ‎04-09-2015
Kudos: 8
Solutions: 1

Re: 3.9.15.8011 blocks DHCP

My network is set up in exactly that way too. Multicast traffic on its on vlan (vlan only), with other vlans set to corporate.

 

Trouble is, now i've taken the netowkr back down and its working, i really don't want to break it again just to get some log files which won't be enough and therefore I'll have to break it again, and again and again.

 

Ubnt please take notice and fix this!

 

 

New Member
Posts: 2
Registered: ‎09-21-2014

Re: 3.9.15.8011 blocks DHCP

I will test the snooping workaround some night in the future and tell you how well it works for me.

 

I'm not yet ready to implement it in "production" since one of the WLANs is fully open guest network and rogue DHCP-server is an actual threat when thinking of availability and security. So I think that DHCP snooping must be enabled. On the other hand, this bug affects availability so badly that the security sacrifice might be possible.. Man Happy

Emerging Member
Posts: 60
Registered: ‎01-14-2014
Kudos: 6

Re: 3.9.15.8011 blocks DHCP

Woah! Have I finally found people with the same issue as me?

 

I started this thread the other day:

 

https://community.ubnt.com/t5/UniFi-Routing-Switching/DHCP-Stops-working-with-Video-Encoder/m-p/2456...

 

What I am finding is with these Just add power video encoders, when they are encoding video (eg. the HDMI is connected), DHCP stops working on the switch... but only that switch.. it appears that DHCP does work on other switches upstream and begins working as soon as I unplug the video decoder.

 

I can reproduce the problem at 2 different sites... both are on US-16-150W and are both running firmware 3.9.42.9152

 

Emerging Member
Posts: 60
Registered: ‎01-14-2014
Kudos: 6

Re: 3.9.15.8011 blocks DHCP

Hey Guys, I just reverted my switch back to version 3.9.6.7613 and that appears to have fixed my issue.. I will do more testing with other clients today..... can someone help me understand what the difference is between the 2 firmware versions? Is it a bug in DHCP snooping or is DHCP snooping disabled in the 3.9.6.7613 but enabled in 3.9.42.9152 ?
Emerging Member
Posts: 45
Registered: ‎04-09-2015
Kudos: 8
Solutions: 1

Re: 3.9.15.8011 blocks DHCP

[ Edited ]

Not entirely sure myself yet. Trouble is, upgrading completely kills my network and I don't have any other switches (not even a small 8 or 16) to be able to test it without it wiping out all functionality in my home network. I really must buy a 8-60W to conduct some tests on. Maybe others can chime in? Definitely create a support ticket with ubnt with anything you find out. - have a look at the first page to see what debugging they want to see - I haven't been able to create a support ticket yet as I have no actionable info. But we need to get Ubnt support working on this - even though they've gone quiet - none of us can live on old firmware forever 

Regular Member
Posts: 714
Registered: ‎02-17-2015
Kudos: 89
Solutions: 9

Re: 3.9.15.8011 blocks DHCP

@Smarthomes @UBNT-cmb 

 

having a similar issue 

 

https://community.ubnt.com/t5/UniFi-Routing-Switching/Dell-RS630-and-unifi-switch-USW24/td-p/2465427

 

My issue is with a server 

 

did you try the downgrade with other clients ?

New Member
Posts: 1
Registered: ‎08-17-2016

Re: 3.9.15.8011 blocks DHCP

[ Edited ]

got the same issues starting with 3.9.15 firmware on UAP AC PRO with MS Windows DHCP Server.

DHCP is not working all the time, had to reboot AP in order to get DHCP working back for wifi clients.

 

So i cant upgrade my UAP AC PRO APs to more than v3.9.3.7537 (have not tried 3.9.6).

 

Set up :

 * WIFI1 : Untagged VLAN, ipv4, 10.0.0.0/8 network, WPA 2 Enterprise, Radius, DHCP from Windows Server (most issues with this network)

 * WIFI2 : Tagged VLAN, ipv4, 192.168.x.0/24 network, WPA 2 Perwonal, DHCP from Ubuntu ich-dhcpd (less errors)

New Member
Posts: 4
Registered: ‎12-05-2017
Kudos: 12

Re: 3.9.15.8011 blocks DHCP

I also got DHCP dropouts with 3.9.42.9152 until a couple of weeks ago when I downgraded the switch to 3.9.6.7613 and all has been well since then. Just tried to upgrade again and had dropouts again.

 

I added the flag to disable DHCP snooping (https://community.ubnt.com/t5/UniFi-Routing-Switching/Disable-DHCP-Snooping-on-USW/td-p/1510229) just in case (and checked that snooping was disabled) but that didn't make any difference (I didn't expect it to as I'm not doing DHCP relay - the DHCP server is on the LAN).

 

I don't have a USG (2 x AC-PROs and a US-8-150W). DHCP is from a Cisco 5505 ASA. The AC-PROs are both on firmware 3.9.42.9152 (and havn't been changed at all during this time) -  I've now downgraded the switch again and everything started working again...

 

I can't try 3.9.50.9295 because there's no downgrade path to 3.9.6.7613 (which seems like the latest one that actually works). What do I do now Ubiquity?

New Member
Posts: 4
Registered: ‎06-25-2015

Re: 3.9.15.8011 blocks DHCP

Hello and sorry I could not understand I have latest firmware on my APS and switches if I disable the snooping feature does it get solved I'm really desperate because I have more than 110 a APS rolled out thank you
New Member
Posts: 4
Registered: ‎01-29-2015
Kudos: 1

Re: 3.9.15.8011 blocks DHCP

Mad5If you have ended up here because of DHCP blocking issues - PLEASE READMad5

Ubiquiti do not care about these sorts of issues. This issue has existed for well over a year, across many firmwares.

DO NOT WASTE YOUR TIME.

Return your product, get a refund. Use this as an example if you need to RMA.

If you manage more than one ubiquiti device - let this serve as your warning. Any money you save on their hardware will be wasted supporting issues like this that they will flat out refuse to resolve.

 

Reply