Reply
Member
Posts: 192
Registered: ‎08-26-2009
Kudos: 19
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@UBNT-BenBuckley

 

I found many of the "unable to open sock" messages in /var/log/messages of two of the three units running debug firmware.  As I've said before, I think this often indicates a state where clients can connect to wifi, but they can't reach the Internet.

 

Based on this, I rebooted the two units.  After rebooting I was able to see a few debug lines in addition to the "Too much interference" lines.:

 

BZ.bjb-debug-379-odp_ben@toad.ubnt-ut.com.-1# dmesg | grep bjb
[   21.386000] bjb in ath_dev_attach:7590
[   21.396000] bjb default sc_dcs_enabled, val:0
BZ.bjb-debug-379-odp_ben@toad.ubnt-ut.com.-1# dmesg | grep Too
[  151.873000] Too much interference, Trying to change channels
[  181.876000] Too much interference, Trying to change channels
[  211.880000] Too much interference, Trying to change channels
[  241.880000] Too much interference, Trying to change channels

 

 

 

Andris Bjornson
EveryLayer - Broadband for Billions
Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@andris, the "unable to open sock" is something we need to look at for sure.  It is a distinc issue, but since you can reproduce it, I'm going to lean on you to help get me some important info.

 

Next time an AP gets into that state, will you run the following commands:

for a in `pidof mcad`; do echo $a:; ls -l /proc/$a/fd; done

cat /proc/sys/fs/file-nr

 

Also, can I get access to your controller, as well as the problematic AP's?

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@c2h5oh, Pulling you into this thread because @andris is seeing the same error  you are, so I'd like to pursue fixing it here.

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Member
Posts: 192
Registered: ‎08-26-2009
Kudos: 19
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@UBNT-BenBuckley

 

Shoot - now i kind of wish i hadn't rebooted those "unable to open sock" aps!  I will absolutely run these commands next time I see this.

 

Access to the controller is not a problem - should I PM you credentials?  How do you normally do this?

 

Access to the APs is doable but tricky.  There is some natting invovled.  What have you done with other users....port forwards?

Andris Bjornson
EveryLayer - Broadband for Billions
Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@andris yes just pm me the credentials.

 

port forwarding is the easies way to give access to the AP's.  Other options are to give me remote access to a computer that is on the network, then I can ssh from there.

 

Let's do this... if you can set up a port forward easily lets do that.  Otherwise I'll just have you run the commands and share the results.

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@andris@c2h5oh, I have a correction to the command's for you to run when you see the issue manifest on the AP's. Please do the following:

 

for a in `pidof wevent`; do echo $a:; ls -l /proc/$a/fd; done

cat /proc/sys/fs/file-nr

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Member
Posts: 192
Registered: ‎08-26-2009
Kudos: 19
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

[ Edited ]

@UBNT-BenBuckley - Credentials for controller sent via PM.  I will work on port forwarding to the access points later:

 

 

Output of first command was long, i put it in a gist

for a in `pidof wevent`; do echo $a:; ls -l /proc/$a/fd; done

https://gist.github.com/andris/c09a21ef25f0fbdede5c3047f307c311

 

 

Output of second command

BZ.bjb-debug-379-odp_ben@toad.ubnt-ut.com.-1# cat /proc/sys/fs/file-nr
1208   	0      	8192

 

 

Units with the debug firmware that experience these issues are (in order of how frequently the issues occur):

  • CBD-UNION-TOWERS1 /  44:d9:e7:c6:aa:0e
  • CBD-WELBEX-IMENTI / 44:d9:e7:c6:ac:14
  • CBD-KCS BUILDING /  44:d9:e7:c6:a9:c8
Andris Bjornson
EveryLayer - Broadband for Billions
Member
Posts: 229
Registered: ‎05-07-2014
Kudos: 32
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

[ Edited ]

Hello,

 

This is a bit frustrating as well. I have 3 APs that were rebooting or on/off by itself. 

 

Version 3.7.10.5096 - GUI 5.2.3.0

 

Unifi AP Pro - solved by removing minimum RSSI

Unifi AP Outdoor - solved not solved by  removing minimum RSSI ** edited: correction, it started to reboot after 30 minutes (vs 1-5 min before)

Unifi AP Outdoor (another one) - not solved by removing minimum RSSI -- it seems to have the same open socket issue (still reboots every 1-5 minutes)

 

(We are not in a busy environment, but my search in the forum for "valid pRc->validRateIndex" showed this thread)

 

Date	Time	Level	Host Name	Category	Program	Messages
2016-08-23	18:54:44	Information	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.133000] br0: port 4(ath1) entering disabled state
2016-08-23	18:54:44	Information	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.121000] br0.100: port 1(ath2) entering disabled state
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.113000] Set mcast rate : 6000
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.113000] 13 14 15 16 17 18 24 25 26 27
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.113000] valid pRc->validRateIndex :
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.113000] vap: 26:a4:3c:a3:ad:XX
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.112000] Set bcast rate 6000
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.112000] Set mcast rate : 6000
2016-08-23	18:54:44	Information	("U2O,24a43ca2a,v3.7.9.5089")	daemon	init	process '/bin/dropbear -F -d /var/run/dropbear_dss_host_key -r /var/run/dropbear_rsa_host_key -p 22' (pid 1605) exited. Sched
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.112000] vap->iv_mgt_rate 6000
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 81.111000] wlan_get_active_vport_chan: ch=0
2016-08-23	18:54:44	Information	("U2O,24a43ca2ad,v3.7.9.5089")	daemon	init	process '/bin/hostapd /etc/aaa5.cfg' (pid 1603) exited. Scheduling for restart.
2016-08-23	18:54:44	Information	("U2O,24a43ca2ad,v3.7.9.5089")	daemon	init	process '/bin/hostapd /etc/aaa4.cfg' (pid 1602) exited. Scheduling for restart.
2016-08-23	18:54:44	Information	("U2O,24a43ca2ad,v3.7.9.5089")	daemon	init	process '/sbin/udhcpc -f -i br0.100 -V ubnt -A 10 -s /etc/udhcpc/udhcpc_ip_only -p /var/run/udhcpc.br0.100.pid -h "RetreatOut
2016-08-23	18:54:44	Information	("U2O,24a43ca2ad,v3.7.9.5089")	daemon	init	process '/sbin/udhcpc -f -i br0 -V ubnt -A 10 -s /etc/udhcpc/udhcpc -p /var/run/udhcpc.br0.pid -h "RetreatOutdoor"' (pid 1598
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.982000] Set mcast rate : 6000
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.981000] 13 14 15 16 17 18 24 25 26 27
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.981000] valid pRc->validRateIndex :
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.981000] vap: 36:a4:3c:a3:ad:33
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.981000] Set bcast rate 6000
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.981000] Set mcast rate : 6000
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.980000] vap->iv_mgt_rate 6000
2016-08-23	18:54:44	Warning	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.980000] wlan_get_active_vport_chan: ch=0
2016-08-23	18:54:43	Information	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.837000] br0: port 5(ath4) entering disabled state
2016-08-23	18:54:43	Information	("U2O,24a43ca2ad,v3.7.9.5089")	user	libubnt	wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath1: 28:cf:e9:61:02:XX / 0
2016-08-23	18:54:43	Information	("U2O,24a43ca2ad,v3.7.9.5089")	user	kernel	[ 80.828000] br0.100: port 2(ath3) entering disabled state

 

Is there anything I can do to solve this (temporarily?).

 

Cheers,

 

Amaravati

Member
Posts: 229
Registered: ‎05-07-2014
Kudos: 32
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

AP AC Pro is started to play up as well in the syslog:

 

2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4994.226000] 84:89:ad:46:72:00: node vid=0 rsn_authmode=0x00000040, ni_authmode=0x00
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4994.226000] ieee80211_ioctl_setparam: VLANID32 = 0
2016-08-25	19:39:10	Information	("U7P,24a43c02eb8f,v3.7.10.5096")	daemon	hostapd	ath5: STA 84:89:ad:46:72:00 WPA: pairwise key handshake completed (RSN)
2016-08-25	19:39:10	Information	("U7P,24a43c02eb8f,v3.7.10.5096")	user	libubnt	wevent.ubnt_custom_event(): EVENT_STA_JOIN ath5: 84:89:ad:46:72:00 / 5
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4994.051000] 84:89:ad:46:72:00: node vid=0 rsn_authmode=0x00000040, ni_authmode=0x00
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4994.051000] ieee80211_ioctl_setparam: VLANID32 = 0
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4994.048000] 13 14 15 16 23 24 25 26 27 61
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4994.047000] valid pRc->validRateIndex :
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4994.047000] vap: 24:a4:3c:4:eb:8f
2016-08-25	19:39:10	Information	("U7P,24a43c02eb8f,v3.7.10.5096")	daemon	hostapd	ath5: STA 84:89:ad:46:72:00 IEEE 802.11: associated
2016-08-25	19:39:10	Information	("U7P,24a43c02eb8f,v3.7.10.5096")	user	libubnt	wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath5: 84:89:ad:46:72:00 / 0
2016-08-25	19:39:10	Information	("U7P,24a43c02eb8f,v3.7.10.5096")	user	libubnt	wevent.ubnt_custom_event(): EVENT_STA_JOIN ath5: 84:89:ad:46:72:00 / 5
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4993.882000] 84:89:ad:46:72:00: node vid=0 rsn_authmode=0x00000040, ni_authmode=0x00
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4993.882000] ieee80211_ioctl_setparam: VLANID32 = 0
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4993.859000] 84:89:ad:46:72:00: node vid=0 rsn_authmode=0x00000040, ni_authmode=0x00
2016-08-25	19:39:10	Warning	("U7P,24a43c02eb8f,v3.7.10.5096")	user	kernel	[ 4993.859000] ieee80211_ioctl_setparam: VLANID32 = 0

The other AP outdoor that was initially rebooting every few minutes, started to reboot again. (after removing the RSSI setting, it remained 'stable' for 30 minutes. This is the log file:

 

2016-08-25	19:42:21	Information	("U2O,24a43ca293e6,v3.7.10.5096")	daemon	hostapd	ath1: STA bc:4c:c4:a2:2b:a0 IEEE 802.1X: authenticated - EAP type: 25 (PEAP)
2016-08-25	19:42:21	Information	("U2O,24a43ca293e6,v3.7.10.5096")	daemon	hostapd	ath1: STA bc:4c:c4:a2:2b:a0 RADIUS: starting accounting session 0000003F-00000000
2016-08-25	19:42:21	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 114.464000] bc:4c:c4:a2:2b:a0: node vid=0 rsn_authmode=0x00000040, ni_authmode=0x00
2016-08-25	19:42:21	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 114.464000] ieee80211_ioctl_setparam: VLANID32 = 0
2016-08-25	19:42:21	Information	("U2O,24a43ca293e6,v3.7.10.5096")	daemon	hostapd	ath1: STA bc:4c:c4:a2:2b:a0 WPA: pairwise key handshake completed (RSN)
2016-08-25	19:42:20	Information	("U2O,24a43ca293e6,v3.7.10.5096")	user	libubnt	wevent.ubnt_custom_event(): EVENT_STA_JOIN ath1: bc:4c:c4:a2:2b:a0 / 1
2016-08-25	19:42:19	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 113.156000] bc:4c:c4:a2:2b:a0: node vid=0 rsn_authmode=0x00000040, ni_authmode=0x00
2016-08-25	19:42:19	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 113.156000] ieee80211_ioctl_setparam: VLANID32 = 0
2016-08-25	19:42:18	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 111.934000] 13 14 15 16 17 18 19
2016-08-25	19:42:18	Information	("U2O,24a43ca293e6,v3.7.10.5096")	daemon	hostapd	ath1: STA bc:4c:c4:a2:2b:a0 IEEE 802.11: associated
2016-08-25	19:42:18	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 111.934000] valid pRc->validRateIndex :
2016-08-25	19:42:18	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 111.934000] vap: 26:a4:3c:a3:93:e6
2016-08-25	19:41:44	Information	("U2O,24a43ca293e6,v3.7.10.5096")	daemon	hostapd	ath3: STA cc:55:ad:fc:f9:94 IEEE 802.11: deauthenticated due to local deauth request
2016-08-25	19:41:41	Error	("U2O,24a43ca293e6,v3.7.10.5096")	cron	crond	time disparity of 24535840 minutes detected
2016-08-25	19:41:40	Information	("U2O,24a43ca293e6,v3.7.10.5096")	user	libubnt	wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath3: cc:55:ad:fc:f9:94 / 0
2016-08-25	19:41:40	Information	("U2O,24a43ca293e6,v3.7.10.5096")	user	libubnt	wevent.ubnt_custom_event(): EVENT_STA_JOIN ath3: cc:55:ad:fc:f9:94 / 1
2016-08-25	19:41:38	Information	("U2O,24a43ca293e6,v3.7.10.5096")	daemon	hostapd	ath3: STA cc:55:ad:fc:f9:94 IEEE 802.11: associated
2016-08-25	19:41:38	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 72.440000] 13 16 19
2016-08-25	19:41:38	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 72.440000] valid pRc->validRateIndex :
2016-08-25	19:41:38	Warning	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 72.440000] vap: 46:a4:3c:a3:93:e6
2016-08-25	19:41:38	Information	("U2O,24a43ca293e6,v3.7.10.5096")	user	syslog	ace_reporter.reporter_set_managed(): [STATE] enter MANAGED
2016-08-25	19:41:37	Notice	("U2O,24a43ca293e6,v3.7.10.5096")	user	syswrapper	[state is locked] skipping reload
2016-08-25	19:41:35	Notice	("U2O,24a43ca293e6,v3.7.10.5096")	user	syswrapper	[state is locked] waiting for lock
2016-08-25	19:41:34	Information	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 68.344000] br0.100: port 1(ath2) entering forwarding state
2016-08-25	19:41:34	Information	("U2O,24a43ca293e6,v3.7.10.5096")	user	kernel	[ 68.196000] br0.100: port 2(ath3) entering forwarding state

Is this helpful?

Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@andris@c2h5oh,  this firmware should fix the "unable to open sock" issue.   Please try it on your devices and let me know how it goes.  If it works as expected, we will get it merged into the next release.

 

http://dl.ubnt-ut.com/uap-odplus-busy-env2.bin

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Member
Posts: 192
Registered: ‎08-26-2009
Kudos: 19
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

[ Edited ]

@UBNT-BenBuckley

 

(edited to add info from todays testing)

 

Awesome!  Thank you!  We are upgrading have upgraded our 10 test Outdoor+ units in some of our busiest areas now.

 

I confirm I'm no longer seeing "unable to open sock" messages....nice!  I'm seeing a new message I don't think I saw before:  "IOCTL failed" (see below log excerpt)

 

A few questions:

  1. Is the "IOCTL failed" message a sign of something bad?  In one of the logs this looks maybe like a minrssi kick was attempted and failed?
  2. My PicoM2s are currently running 3.7.9.  Any chance to get this firmware for PicoM2 while we wait for these fixes to be rolled into release?  Or is the theory that this issue does not affect PicoM2 unifi?
  3. Just so i fully understand the "Too much Interference" issue:  The fix that was implemented is that units now no longer reboot when "Too much interference" occurs. (which is great!).  I still see the "Too much interference" messages in the logs.  I think that this is normal, right?  Is this just the message shown when the unifi does a channel change?
  4. Previously, I was seeing "Too much interference, trying to change channels" even on units set to a fixed frequency.  This seems unexpected.  This still happens in the current debug firmware, is the expected behavior that units set to a fixed frequency will not show "Too much interference" and will not try to change channels?  I ask because in noisy areas, I may just have to choose a frequency and live with it...rather than jumping around between a bunch of noisy frequencies.  The below log shows an AP that gets "too much interference" messages as freuqently as every 30 seconds - despite being set to a fixed, non auto frequency

 

Thanks again for your assistance

 

log from AP that used to show a lot of "unable to open sock" messages now showing "IOCTL failed" messages:

Aug 26 18:43:18 COMFORT-BusyIntersection user.info libubnt[2467]: wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath0: 8c:77:16:90:a0:82 / 0
Aug 26 18:43:20 COMFORT-BusyIntersection user.warn kernel: [30421.018000] vap: 44:d9:e7:2f:46:88
Aug 26 18:43:20 COMFORT-BusyIntersection user.warn kernel: [30421.018000] valid pRc->validRateIndex :
Aug 26 18:43:20 COMFORT-BusyIntersection user.warn kernel: [30421.018000] 13 14 15 16 17 18 19
Aug 26 18:43:20 COMFORT-BusyIntersection user.err libubnt[2467]: config_sta_qosrule:449; IOCTL failed:
Aug 26 18:43:20 COMFORT-BusyIntersection user.info libubnt[2467]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: d0:9d:ab:5a:ba:d4 / 2
Aug 26 18:43:28 COMFORT-BusyIntersection user.warn kernel: [30428.614000] vap: 44:d9:e7:2f:46:88
Aug 26 18:43:28 COMFORT-BusyIntersection user.warn kernel: [30428.614000] valid pRc->validRateIndex :
Aug 26 18:43:28 COMFORT-BusyIntersection user.warn kernel: [30428.614000] 13 14 15 16 17 18 19
Aug 26 18:43:28 COMFORT-BusyIntersection user.err libubnt[2467]: config_sta_qosrule:449; IOCTL failed:
Aug 26 18:43:28 COMFORT-BusyIntersection user.info libubnt[2467]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: bc:b3:08:e6:62:36 / 8
Aug 26 18:43:50 COMFORT-BusyIntersection user.info syslog: stamgr.kick_sta(): kick-sta-on d0:9d:ab:5a:ba:d4 ath0 (reason:1)
Aug 26 18:43:50 COMFORT-BusyIntersection user.warn kernel: [30450.803000]  ieee80211_ioctl_kickmac[14767]
Aug 26 18:43:50 COMFORT-BusyIntersection user.info libubnt[2467]: wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath0: d0:9d:ab:5a:ba:d4 / 0
Aug 26 18:43:53 COMFORT-BusyIntersection user.info syslog: stamgr.kick_sta(): kick-sta-on 8c:77:16:cb:fa:07 ath0 (reason:1)
Aug 26 18:43:53 COMFORT-BusyIntersection user.warn kernel: [30453.877000]  ieee80211_ioctl_kickmac[14767]
Aug 26 18:43:53 COMFORT-BusyIntersection user.info libubnt[2467]: wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath0: 8c:77:16:cb:fa:07 / 0
Aug 26 18:43:54 COMFORT-BusyIntersection user.warn kernel: [30454.285000] vap: 44:d9:e7:2f:46:88
Aug 26 18:43:54 COMFORT-BusyIntersection user.warn kernel: [30454.285000] valid pRc->validRateIndex :
Aug 26 18:43:54 COMFORT-BusyIntersection user.warn kernel: [30454.285000] 13 14 15 16 17 18 19
Aug 26 18:43:54 COMFORT-BusyIntersection user.err libubnt[2467]: config_sta_qosrule:449; IOCTL failed:
Aug 26 18:43:54 COMFORT-BusyIntersection user.info libubnt[2467]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: 8c:77:16:cb:fa:07 / 2
Aug 26 18:43:54 COMFORT-BusyIntersection user.info syslog: stamgr.kick_sta(): kick-sta-on 8c:77:16:cb:fa:07 ath0 (reason:1)
Aug 26 18:43:54 COMFORT-BusyIntersection user.warn kernel: [30454.946000]  ieee80211_ioctl_kickmac[14767]

 

log from AP that constantly gets "too much interference, trying to change channels" messages every 30 seconds:

 

Jan  1 13:36:31 CBD-WELBEX-IMENTI user.warn kernel: [38191.720000] vap: 44:d9:e7:c7:ac:14
Jan  1 13:36:31 CBD-WELBEX-IMENTI user.warn kernel: [38191.721000] valid pRc->validRateIndex :
Jan  1 13:36:31 CBD-WELBEX-IMENTI user.warn kernel: [38191.721000] 13 14 15 16 17 18 19
Jan  1 13:36:31 CBD-WELBEX-IMENTI user.err libubnt[15196]: config_sta_qosrule:449; IOCTL failed:
Jan  1 13:36:31 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: 04:f1:3e:b0:e3:62 / 18
Jan  1 13:36:39 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath0: 00:08:22:16:d5:fb / 0
Jan  1 13:36:41 CBD-WELBEX-IMENTI user.warn kernel: [38201.965000] vap: 44:d9:e7:c7:ac:14
Jan  1 13:36:41 CBD-WELBEX-IMENTI user.warn kernel: [38201.966000] valid pRc->validRateIndex :
Jan  1 13:36:41 CBD-WELBEX-IMENTI user.warn kernel: [38201.966000] 13 14 15 16 17 18 19
Jan  1 13:36:42 CBD-WELBEX-IMENTI user.err libubnt[15196]: config_sta_qosrule:449; IOCTL failed:
Jan  1 13:36:42 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: 00:08:22:16:d5:fb / 10
Jan  1 13:36:52 CBD-WELBEX-IMENTI user.warn kernel: [38212.685000] Too much interference, Trying to change channels
Jan  1 13:37:13 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath0: 00:08:22:16:d5:fb / 0
Jan  1 13:37:13 CBD-WELBEX-IMENTI user.warn kernel: [38233.839000] vap: 44:d9:e7:c7:ac:14
Jan  1 13:37:13 CBD-WELBEX-IMENTI user.warn kernel: [38233.839000] valid pRc->validRateIndex :
Jan  1 13:37:13 CBD-WELBEX-IMENTI user.warn kernel: [38233.839000] 13 14 15 16 17 18 19
Jan  1 13:37:13 CBD-WELBEX-IMENTI user.err libubnt[15196]: config_sta_qosrule:449; IOCTL failed:
Jan  1 13:37:13 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: 00:08:22:16:d5:fb / 10
Jan  1 13:37:14 CBD-WELBEX-IMENTI user.warn kernel: [38234.190000] vap: 44:d9:e7:c7:ac:14
Jan  1 13:37:14 CBD-WELBEX-IMENTI user.warn kernel: [38234.190000] valid pRc->validRateIndex :
Jan  1 13:37:14 CBD-WELBEX-IMENTI user.warn kernel: [38234.190000] 13 14 15 16 17 18 19
Jan  1 13:37:14 CBD-WELBEX-IMENTI user.err libubnt[15196]: config_sta_qosrule:449; IOCTL failed:
Jan  1 13:37:14 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: bc:44:34:c6:48:51 / 22
Jan  1 13:37:15 CBD-WELBEX-IMENTI user.warn kernel: [38235.510000] vap: 44:d9:e7:c7:ac:14
Jan  1 13:37:15 CBD-WELBEX-IMENTI user.warn kernel: [38235.510000] valid pRc->validRateIndex :
Jan  1 13:37:15 CBD-WELBEX-IMENTI user.warn kernel: [38235.511000] 13 14 15 16 17 18 19
Jan  1 13:37:15 CBD-WELBEX-IMENTI user.err libubnt[15196]: config_sta_qosrule:449; IOCTL failed:
Jan  1 13:37:15 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_JOIN ath0: 78:ff:ca:11:4f:c9 / 24
Jan  1 13:37:21 CBD-WELBEX-IMENTI user.info libubnt[15196]: wevent.ubnt_custom_event(): EVENT_STA_LEAVE ath0: 80:19:67:29:9a:b1 / 0
Jan  1 13:37:22 CBD-WELBEX-IMENTI user.warn kernel: [38242.685000] Too much interference, Trying to change channels
Andris Bjornson
EveryLayer - Broadband for Billions
Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

[ Edited ]

@andris

Is the "IOCTL failed" message a sign of something bad?  In one of the logs this looks maybe like a minrssi kick was attempted and failed?


That has to do with the fix I put in place to fix the socket issue.  I'll have to look into it.

 


My PicoM2s are currently running 3.7.9.  Any chance to get this firmware for PicoM2 while we wait for these fixes to be rolled into release?  Or is the theory that this issue does not affect PicoM2 unifi?

 

Just so i fully understand the "Too much Interference" issue:  The fix that was implemented is that units now no longer reboot when "Too much interference" occurs. (which is great!).  I still see the "Too much interference" messages in the logs.  I think that this is normal, right?  Is this just the message shown when the unifi does a channel change?

 

Previously, I was seeing "Too much interference, trying to change channels" even on units set to a fixed frequency.  This seems unexpected.  This still happens in the current debug firmware, is the expected behavior that units set to a fixed frequency will not show "Too much interference" and will not try to change channels?  I ask because in noisy areas, I may just have to choose a frequency and live with it...rather than jumping around between a bunch of noisy frequencies.  The below log shows an AP that gets "too much interference" messages as freuqently as every 30 seconds - despite being set to a fixed, non auto frequency

 

 


A couple things.  First, the firmware I provided to (for both bugs) were one-off builds in order to confirm that the solution has the expected behavior.  As a result, you are still seeing the "Too much..." beacsue a robust solution has not been fully implemented.  Now that I've confirmed we can fix both issues, I'll implement it in a robust way and you won't see any eronious messages.  

 

Along the same lines, once I merge these changes, they will be available for all UniFi AP's.  However, there is something you should be aware of regarding the PicostationM2.  That device has very scarce resources and doesn't handle newer firmware very well.  You can see the details in this post:

 

http://community.ubnt.com/t5/UniFi-Wireless-Beta/Wireless-Uplink-Station-stock-on-feature-gen1-10-2-...

 

 

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Member
Posts: 192
Registered: ‎08-26-2009
Kudos: 19
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@UBNT-BenBuckley thanks again for the clear info.

 

  • re: IOCTL message - ok, eagerly anticipating more info.
  • re: still seeing "Too much interference" message - thanks - i understand now
  • thanks for the heads up on the pico resource issue.  We use them sparingly, and this is another reason to do so.  I've been able to upgrade to 3.7.9 without issue so far, but have definitely seen they don't perform as well

looking forward to hearing more

Andris Bjornson
EveryLayer - Broadband for Billions
Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

[ Edited ]

@andris,

The IOCTL issue has been resolved.  The error was a result of the update to 10.2 driver for Gen1, but it actually was't causing an error.  So the only thing that needed to happen was to update the error reporting, which has been taken care of and will be in the next Alpha release.

 

Regarding "Too much interference" and it's corresponding reboots; I've also implemented a robust solution and it will be in the next Alpha release (then eventually into a Beta, then GA).  

 

As for now, the debug firmware you have should operate perfectly well even though there will be some erroneous output in you log file.  If you can deal with that, I'll leave it as is.  If you need an updated image with said changes based on 3.7.9, I can get that to you early next week.

 

For any others experiencing these issues on other models than the OD+, let me know and I will get you an image you can use while the the newest release is still in Alpha.

 

P.S.

Thanks for being patient and helping me debug.  It was very valuable and got us to the solution quickly Man Happy

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Member
Posts: 192
Registered: ‎08-26-2009
Kudos: 19
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

Thanks @UBNT-BenBuckley

If a version without the erroneous log messages isn't too much trouble, I'd certainly take you up on that.

I worry the incorrect error messages might mask and other real error messages...

Thanks so much for the your help, looking forward to the alpha!
Andris Bjornson
EveryLayer - Broadband for Billions
Member
Posts: 229
Registered: ‎05-07-2014
Kudos: 32
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

I fixed the rebooting of the AP AC Pro by using the reset button on the AP for 30 seconds. It seems to stop rebooting now for 4 + hours (compared to less than 1 minute previously).

 

I will try that as well with the Outdoor AP later.

Highlighted
Ubiquiti Employee
Posts: 1,224
Registered: ‎12-16-2015
Kudos: 286
Solutions: 9

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

@andris, here is a new binary for you.  This has the fixes that are being released with the next Alpha release, but are based on 3.7.9, which I recall is the version you're currently running.  If I messed that up, let me know:

 

http://dl.ubnt-ut.com/uap-odplus-busy-env3.bin

 

@Amaravati, it looks like you did a factory reset... not really sure how that would solve your issue.  Regardless, here is a binary for the UAP-OD as you requested (this image is also valid for  UAP, UAP-LR, UAP-OD5, PicostatonM2).  Please give it a try and let me know how it goes.

 

http://dl.ubnt-ut.com/uap-busy-env3.bin

If you'd like to try our latest features, consider signing up for beta: https://help.ubnt.com/hc/en-us/articles/204908664-How-To-Sign-Up-for-Beta-Access

If you're trying to debug a network issue, take a look at this help article: https://help.ubnt.com/hc/en-us/articles/221029967-UniFi-Debugging-Intermittent-Connectivity-Issues-on-your-UAP
Regular Member
Posts: 539
Registered: ‎02-13-2013
Kudos: 90
Solutions: 8

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

There is any firmware for the regular Outdoor? I have the same issues with those AP.

SuperUser
Posts: 7,233
Registered: ‎01-05-2012
Kudos: 1902
Solutions: 938

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

Should be the 2nd above ...

Cheers,

jonatha

Member
Posts: 192
Registered: ‎08-26-2009
Kudos: 19
Solutions: 1

Re: Please help, UAP Outdoor+ rebooting every 5 mins in busy wifi environment - "too much inter

Thanks @UBNT-BenBuckley

 

I will get this deployed today.

 

When you say the last binary you provided: uap-odplus-busy-env3.bin is "based on 3.7.9" can you clarify how that differs from the uap-odplus-busy-env2.bin binary....was that also based on 3.7.9 or was that based on a different version?

 

I think if i understand correctly - uap-odplus-busy-env3.bin contains the more robust / permanent solutions you mentioned in earlier posts, where the earlier debug builds were quick/debug implementations and tests of the solutions?

 

Thanks for the help understanding!

Andris Bjornson
EveryLayer - Broadband for Billions
Reply