Reply
Highlighted
Emerging Member
Posts: 96
Registered: ‎05-24-2018
Kudos: 27

5.9.29 Update - lots of devices disconnected

2 days ago we updated our 67 UniFi servers from 5.8.30 to 5.9.29

 

Since then we've had several complaints that devices disconnected since the upgrade.

 

Several of my own sites have disconnected devices as well.

 

Wondering if this happened to others too, and what can be done to prevent this in the future?

UniFi, UNMS, UCRM, and UniFi Video cloud hosting https://hostifi.net
Veteran Member
Posts: 4,650
Registered: ‎06-13-2015
Kudos: 1261
Solutions: 218

Re: 5.9.29 Update - lots of devices disconnected

@HostiFi especially as an MSP I recommend to test, test, test. The best way to prevent issues at that scale.

Art of WiFi
Check out our UniFi API browser tool on GitHub. The PHP API client which it uses, can be found here on GitHub.
Details on our UniFi Device Search tool can be found here.
Emerging Member
Posts: 76
Registered: ‎09-29-2015
Kudos: 8

Re: 5.9.29 Update - lots of devices disconnected

I have a site where the USG is now stuck in Adopting. And my Switch it totally missing. It does not show offline, it is just gone.  it is up because it powers all APs and the Cloudkey. 

 

What is very strange. Everything is up. 

Emerging Member
Posts: 96
Registered: ‎05-24-2018
Kudos: 27

Re: 5.9.29 Update - lots of devices disconnected

I have a system in place for testing, I roll out the update to my own 3 servers first and if it looks good I push it out to everyone.

It seems the device disconnects happened a few days afterwards tho

I would expect better from a “stable” release
UniFi, UNMS, UCRM, and UniFi Video cloud hosting https://hostifi.net
Emerging Member
Posts: 76
Registered: ‎09-29-2015
Kudos: 8

Re: 5.9.29 Update - lots of devices disconnected

The USG that was in a repeat Adopting came back fine. I am still missing a switch. As stated above, it is online and working. It is just not in the Dashboards. What is even stranger, the CLIENT does not show Swtich / Port. It shows the LAN connection on USG. 

Established Member
Posts: 1,429
Registered: ‎02-01-2014
Kudos: 374
Solutions: 54

Re: 5.9.29 Update - lots of devices disconnected

@egarrison- Its not strange at all that the devices are still up. The controller has nothing to do with them working. If you don't have guest access enabled and you don't want the stats that it can give you from the USG, then you don't even need to run a controller. Thats the beauty of it. If the controller dies, devices keep working.
Established Member
Posts: 1,429
Registered: ‎02-01-2014
Kudos: 374
Solutions: 54

Re: 5.9.29 Update - lots of devices disconnected

@HostiFi- Jumping that much between upgrades, did you go through any of the sites having issues and check the SSH authentication section in the controller-> settings area? It is possible that it either turned itself off or got hosed and it doesn't have the correct creds in there anymore.

I would do a sweep of the settings in the controller for those sites and see if anything has changed. I know sometimes if I upgrade, my inWall AP decides to put its wifi back to "enabled" when I don't want them to be.

The controller has a mind of its own sometimes so it working for a while and then not, isn't a weird thing.
New Member
Posts: 39
Registered: ‎11-23-2017
Kudos: 4
Solutions: 1

Re: 5.9.29 Update - lots of devices disconnected

I upgraded a controller to 5.9.29, but also updated java v8 from 151 to 191, all devices were disconnected.

Uninstalling 191 en reinstalling 151 fixed it. This is obviously only a workaround, but something is not right

New Member
Posts: 2
Registered: ‎12-04-2017
Kudos: 1

Re: 5.9.29 Update - lots of devices disconnected

I've found that the sites I'm having this issue with are all running Sophos XG firewalls.

 

It may be a security setting and I'm able to currently able to bypass it with a firewall rule (at the top, LAN/any to WAN/controller, no restrictions, not user based, NAT and logging on). Some of our firewalls had this rule already in place due to communication issues with ScreenConnect (which we also host) clients disconnecting every 5 minutes.

 

It's worth pointing out that we were working just fine on 5.6.22 before upgrading to 5.9.29 and had to add the exception ONLY after upgrading.  I don't know what has changed in the communications but something is being detected as malicious traffic (still investigating).

 

Some devices would show online for a while.  One site had no APs connected but the switch online until the firmware was updated. After adding the firewall rule into the Sophos XG we now have the entire site connected.

 

 

New Member
Posts: 2
Registered: ‎12-04-2017
Kudos: 1

Re: 5.9.29 Update - lots of devices disconnected

Found another way to fix it on the Sophos XG.  Have to create a custom IPS exception for Signature 40146 (browser exploit).  However, since this would expose all the Windows computers I'm trying to protect I've decided a special controller access rule still works best for my customers.

 

From the log viewer:

 

 
 
IPS
2018-11-05 14:58:18
Signatures
Drop
 
controller WAN address
device LAN address
40146
BROWSER-IE Microsoft Edge malformed response information disclosure attempt
Reply