Upcoming Maintenance Alert:

The UBNT Community will be upgraded at 5pm MDT on April 25th. During this time the community forums will be set to read-only status.

Learn more

×
Reply
Emerging Member
Posts: 66
Registered: ‎12-18-2016
Kudos: 9
Solutions: 1

[Solved] CloudKey/Controller will not startup after power outage

[ Edited ]

Hi! Maybe someone can please help me out?

 

After a power outage, the CloudKey won't startup again (it's keeping flashing white). Because of the boot-fail of the CK I don't have access to the controller webmin.

 

FYI, my latest (full) backup was done on the 3rd of december version 5.6.24.

 

I can access/login the CK by SSH. So what to do next?

 

Gateway: UniFi Security Gateway 3P
Switches: UniFi Switch 24 POE-250W / UniFi Switch 8 POE-150W
Access Points: UniFi AP-AC-Lite / UniFi AP-AC-Mesh / UniFi AP-AC-Mesh-Pro
New Member
Posts: 15
Registered: Tuesday
Kudos: 1

Re: [Help Needed] CloudKey/Controller will not startup after power outage

[ Edited ]

What's in /srv/unifi/logs/server.log ?

 

Easiest way if you have a backup is ensure you have a local copy then factory reset then restore. It's likely the db has become corrupted, which would be evident in the above log file when starting the Controller or directly after a reboot.

 

If you don't want to sift through the entire file then grep it for mongo and look for any error messages:

 

grep mongo /srv/unifi/logs/server.log

 

If that doesn't show you anything then the contents of the log file may.

Highlighted
Emerging Member
Posts: 66
Registered: ‎12-18-2016
Kudos: 9
Solutions: 1

Re: [Help Needed] CloudKey/Controller will not startup after power outage

Hi @chrisfd2, thanks for responding.

 

When doing a grep mongo /srv/unifi/logs/server.log I donnot have any messages later then 2017-11-25. In the server.log only 2017-12-07 latest messages appear from 03:44. My shortage was round 09:45, so it looks like there are no problems are reported to the log after that time.

 

Kind of strange.

 

Because it's not booting, and the problem is there, it would be nice to figure out what is/went wrong. Any suggestions?

 

Are there ways to find out where it is stranding?

Gateway: UniFi Security Gateway 3P
Switches: UniFi Switch 24 POE-250W / UniFi Switch 8 POE-150W
Access Points: UniFi AP-AC-Lite / UniFi AP-AC-Mesh / UniFi AP-AC-Mesh-Pro
New Member
Posts: 15
Registered: Tuesday
Kudos: 1

Re: [Help Needed] CloudKey/Controller will not startup after power outage

If there is nothing showing in that log file after the power cut then it's probably best to do a factory reset and restore from backup.

Emerging Member
Posts: 66
Registered: ‎12-18-2016
Kudos: 9
Solutions: 1

Re: [Help Needed] CloudKey/Controller will not startup after power outage

Hmm, okay.

 

I never had to do  a factory on my cloud key. Is there a (safe) procedure not to mess up my current LAN setup?

 

Gateway: UniFi Security Gateway 3P
Switches: UniFi Switch 24 POE-250W / UniFi Switch 8 POE-150W
Access Points: UniFi AP-AC-Lite / UniFi AP-AC-Mesh / UniFi AP-AC-Mesh-Pro
New Member
Posts: 15
Registered: Tuesday
Kudos: 1

Re: [Help Needed] CloudKey/Controller will not startup after power outage

The configuration is all held in the backup as far as I'm aware, so it is safe to do so.
New Member
Posts: 7
Registered: ‎11-22-2016

Re: [Help Needed] CloudKey/Controller will not startup after power outage

I've run into similar issues when restarting the CloudKey or after a power failure. Seems the DB gets corrupted. I have it set to auto-backup weekly, so it's easy enough to go back to a backup but you have to do that manually, which is frustrating.

 

I would recommend you set the auto-backup to happen more frequently, and keep 7-10 days worth.

 

Regardless, there needs to be a better way for the CloudKey DB to monitor itself and be more resiliant. Power outages happen. You power cycle the switch it's plugged into and you can lose your config.

Emerging Member
Posts: 58
Registered: ‎07-30-2016
Kudos: 12
Solutions: 5

Re: [Help Needed] CloudKey/Controller will not startup after power outage

Putting your networking gear on a decent UPS would keep this from happening during typical power outages. Fairly inexpensive way to mitigate a pain in the butt and potentially time consuming network reconfiguration (if you have no backup). AND it has the added value of keeping your internet up and running so you can entertain yourself during said outage.
New Member
Posts: 7
Registered: ‎11-22-2016

Re: [Help Needed] CloudKey/Controller will not startup after power outage

A UPS doesn't solve the issue of the PoE on a switch port being cycled Man Happy

 

Rather fix the root cause.

Emerging Member
Posts: 66
Registered: ‎12-18-2016
Kudos: 9
Solutions: 1

Re: [Help Needed] CloudKey/Controller will not startup after power outage

I did some further investigation. Boot was interupted by a custom config.gateway.json (this topic for details).

 

Renamed the config file and send a reboot command (shutdown -r now) and the controller came back online.

 

Problem solved, now just to figure out why the config file is not right.

Gateway: UniFi Security Gateway 3P
Switches: UniFi Switch 24 POE-250W / UniFi Switch 8 POE-150W
Access Points: UniFi AP-AC-Lite / UniFi AP-AC-Mesh / UniFi AP-AC-Mesh-Pro
Emerging Member
Posts: 66
Registered: ‎12-18-2016
Kudos: 9
Solutions: 1

Re: [Help Needed] CloudKey/Controller will not startup after power outage

[ Edited ]

@ngoehring@rustyhodge Another interesting discussion is what to do with power outages.

 

I do have an UPS for vital infrastructure devices (well, if you can call a home-infrastructure vital... Out. ). The problem was that the outage duration was longer then the UPS-battery capacity.

 

My servers get a safe-shutdown signal when UPS-battery is going below a certain level. I can't do this with the Ubiquiti devices, so far as I know.

 

Would be nice to know if there are solutions for that?

Gateway: UniFi Security Gateway 3P
Switches: UniFi Switch 24 POE-250W / UniFi Switch 8 POE-150W
Access Points: UniFi AP-AC-Lite / UniFi AP-AC-Mesh / UniFi AP-AC-Mesh-Pro
Reply