• N
    nicholas_saw

    Its running at 16GB.

    I will check the memory usage the next time it happens.

    It happens yesterday but my colleague just rebooted the server.

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    i have alot of APs more than 300 connected.

    Can cloudkey do it?

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    ya..it happen to be on prevous 5.4.x version than i upgrade to 5.4.11.

    But it still happening.

    Im hoping someone able to share some fix for this.

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    Hi,

    Anyone encounter https stop responding and top showing java at 700% with 95% CPU usage.

    After like 1 day the https will not response and i need to reboot the server for the system to resume as normal.

    Here is the screenshot when the system is running without issue.

    top - 20:12:19 up 21:21, 1 user, load average: 2.47, 2.99, 3.21
    Tasks: 168 total, 1 running, 167 sleeping, 0 stopped, 0 zombie
    %Cpu(s): 29.0 us, 0.4 sy, 0.0 ni, 69.6 id, 0.0 wa, 0.0 hi, 1.0 si, 0.0 st
    KiB Mem: 16432336 total, 9793220 used, 6639116 free, 141856 buffers
    KiB Swap: 0 total, 0 used, 0 free. 7830692 cached Mem

    PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
    1318 root 20 0 6188080 1.351g 22416 S 227.8 8.6 3340:50 java
    1343 root 20 0 64.764g 1.550g 1.493g S 9.9 9.9 155:59.05 mongod
    1 root 20 0 33636 4196 2740 S 0.0 0.0 0:01.58 init
    2 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
    3 root 20 0 0 0 0 S 0.0 0.0 0:01.34 ksoftirqd/0
    4 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0
    5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
    7 root 20 0 0 0 0 S 0.0 0.0 1:43.22 rcu_sched
    8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
    9 root rt 0 0 0 0 S 0.0 0.0 0:00.42 migration/0
    10 root rt 0 0 0 0 S 0.0 0.0 0:00.17 watchdog/0
    11 root rt 0 0 0 0 S 0.0 0.0 0:00.16 watchdog/1
    12 root rt 0 0 0 0 S 0.0 0.0 0:00.43 migration/1
    13 root 20 0 0 0 0 S 0.0 0.0 0:01.43 ksoftirqd/1
    14 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/1:0
    15 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/1:+
    16 root rt 0 0 0 0 S 0.0 0.0 0:00.16 watchdog/2

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    And now its not showing again and i still cant do backup.

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    Hi ,

    There seems to be error while repairing.

    2016-06-03T10:36:13.673+0800 [initandlisten] MongoDB starting : pid=4120 port=27117 dbpath=/opt/UniFi/data/db 64-bit
    2016-06-03T10:36:13.674+0800 [initandlisten] db version v2.6.1
    2016-06-03T10:36:13.674+0800 [initandlisten] git version: 4b95b086d2374bdcfcdf2249272fb552c9c726e8
    2016-06-03T10:36:13.674+0800 [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
    2016-06-03T10:36:13.674+0800 [initandlisten] allocator: tcmalloc
    2016-06-03T10:36:13.674+0800 [initandlisten] options: { net: { port: 27117 }, repair: true, storage: { dbPath: "/opt/UniFi/data/db" } }
    2016-06-03T10:36:13.691+0800 [initandlisten] repairDatabase local
    2016-06-03T10:36:13.692+0800 [FileAllocator] allocating new datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/local.ns, filling with zeroes…
    2016-06-03T10:36:13.692+0800 [FileAllocator] creating directory /opt/UniFi/data/db/_tmp_repairDatabase_0/_tmp
    2016-06-03T10:36:13.694+0800 [FileAllocator] done allocating datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/local.ns, size: 16MB, took 0 secs
    2016-06-03T10:36:13.697+0800 [FileAllocator] allocating new datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/local.0, filling with zeroes…
    2016-06-03T10:36:13.697+0800 [FileAllocator] done allocating datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/local.0, size: 64MB, took 0 secs
    2016-06-03T10:36:13.713+0800 [initandlisten] repairDatabase ace
    2016-06-03T10:36:13.714+0800 [FileAllocator] allocating new datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/ace.ns, filling with zeroes…
    2016-06-03T10:36:13.714+0800 [FileAllocator] creating directory /opt/UniFi/data/db/_tmp_repairDatabase_0/_tmp
    2016-06-03T10:36:13.720+0800 [FileAllocator] done allocating datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/ace.ns, size: 16MB, took 0 secs
    2016-06-03T10:36:13.723+0800 [FileAllocator] allocating new datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/ace.0, filling with zeroes…
    2016-06-03T10:36:13.887+0800 [FileAllocator] done allocating datafile /opt/UniFi/data/db/_tmp_repairDatabase_0/ace.0, size: 64MB, took 0.163 secs
    2016-06-03T10:36:13.897+0800 [initandlisten] ace.alarm Assertion failure isOk() src/mongo/db/storage/extent.h 80
    2016-06-03T10:36:13.910+0800 [initandlisten] ace.alarm 0x11c0e91 0x1163109 0x1146f4e 0xee4940 0xee4faf 0xf1da47 0xdedee8 0x75fd8e 0x76242c 0x766b1f 0x7673eb 0x767985 0x767ba9 0x3aa9a1ed1d 0x75eae9
    mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x11c0e91]
    mongod(_ZN5mongo10logContextEPKc+0x159) [0x1163109]
    mongod(_ZN5mongo12verifyFailedEPKcS1_j+0x17e) [0x1146f4e]
    mongod(_ZNK5mongo13ExtentManager9getExtentERKNS_7DiskLocEb+0x60) [0xee4940]
    mongod(_ZNK5mongo13ExtentManager13getNextRecordERKNS_7DiskLocE+0x6f) [0xee4faf]
    mongod(_ZN5mongo12FlatIterator7getNextEv+0x97) [0xf1da47]
    mongod(_ZN5mongo14repairDatabaseESsbb+0x22f8) [0xdedee8]
    mongod(_ZN5mongo11doDBUpgradeERKSsPNS_14DataFileHeaderE+0x5e) [0x75fd8e]
    mongod() [0x76242c]
    mongod(_ZN5mongo14_initAndListenEi+0x5df) [0x766b1f]
    mongod(_ZN5mongo13initAndListenEi+0x1b) [0x7673eb]
    mongod() [0x767985]
    mongod(main+0x9) [0x767ba9]
    /lib64/libc.so.6(__libc_start_main+0xfd) [0x3aa9a1ed1d]
    mongod() [0x75eae9]
    2016-06-03T10:36:13.914+0800 [initandlisten] cleaning up failed repair db: ace path: /opt/UniFi/data/db/_tmp_repairDatabase_0
    2016-06-03T10:36:13.928+0800 [initandlisten] exception in initAndListen: 0 assertion src/mongo/db/storage/extent.h:80, terminating
    2016-06-03T10:36:13.928+0800 [initandlisten] dbexit:
    2016-06-03T10:36:13.928+0800 [initandlisten] shutdown: going to close listening sockets…
    2016-06-03T10:36:13.928+0800 [initandlisten] shutdown: going to flush diaglog…
    2016-06-03T10:36:13.928+0800 [initandlisten] shutdown: going to close sockets…
    2016-06-03T10:36:13.929+0800 [initandlisten] shutdown: waiting for fs preallocator…
    2016-06-03T10:36:13.929+0800 [initandlisten] shutdown: closing all files…
    2016-06-03T10:36:13.929+0800 [initandlisten] closeAllFiles() finished
    2016-06-03T10:36:13.929+0800 [initandlisten] shutdown: removing fs lock…
    2016-06-03T10:36:13.929+0800 [initandlisten] dbexit: really exiting now

    When the mongod started. I can see this error in the logs

    2016-06-03T10:40:22.799+0800 [conn23] ace.stat_hourly Assertion failure false src/mongo/db/structure/btree/key.cpp 433

    At least now, the controller is showing connected Users

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    Hi ,

    The system still not seeing users but able to detect status.

    So i try to run repairs and got the message below.

    2016-06-03T07:48:31.355+0800 [initandlisten] MongoDB starting : pid=7998 port=27117 dbpath=/opt/UniFi/data/db 64-bit host=CentOS-UnifiController02-3
    2016-06-03T07:48:31.355+0800 [initandlisten] db version v2.6.1
    2016-06-03T07:48:31.355+0800 [initandlisten] git version: 4b95b086d2374bdcfcdf2249272fb552c9c726e8
    2016-06-03T07:48:31.355+0800 [initandlisten] build info: Linux build14.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
    2016-06-03T07:48:31.355+0800 [initandlisten] allocator: tcmalloc
    2016-06-03T07:48:31.355+0800 [initandlisten] options: { net: { port: 27117 }, repair: true, storage: { dbPath: "/opt/UniFi/data/db" } }
    2016-06-03T07:48:31.355+0800 [initandlisten] exception in initAndListen: 10310 Unable to lock file: /opt/UniFi/data/db/mongod.lock. Is a mongod instance already running?, terminating
    2016-06-03T07:48:31.355+0800 [initandlisten] dbexit:
    2016-06-03T07:48:31.355+0800 [initandlisten] shutdown: going to close listening sockets…
    2016-06-03T07:48:31.355+0800 [initandlisten] shutdown: going to flush diaglog…
    2016-06-03T07:48:31.355+0800 [initandlisten] shutdown: going to close sockets…
    2016-06-03T07:48:31.356+0800 [initandlisten] shutdown: waiting for fs preallocator…
    2016-06-03T07:48:31.356+0800 [initandlisten] shutdown: closing all files…
    2016-06-03T07:48:31.356+0800 [initandlisten] closeAllFiles() finished
    2016-06-03T07:48:31.356+0800 [initandlisten] dbexit: really exiting now

    I do have the mongod.lock file in the db folder.

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    Hi ,

    I have insufficient space to run the repair.

    Therefore i moved the entire folder to a new system and it able to work without running repairs.

    However, i'm not sure whether it will show error or not as the system is still slowly recovering now.

    I will update when the system is stable again.

    Thanks for the help

    posted in UniFi Wireless read more
  • N
    nicholas_saw

    Hi ,

    My Unifi folder is at /opt/UniFi/data/db

    Im suppose to repair that right?

    posted in UniFi Wireless read more

Looks like your connection to Ubiquiti Networks Community was lost, please wait while we try to reconnect.