Scheduled maintenance: Community will be offline Monday June 17th, 1:00 AM - 6:00 AM (PT)
Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

NanoBridge and NanoStation M don't reconnect after minor change

I've had this problem occur twice in the past couple of weeks, so I'm beginning to think it's not a rare random event. We have some 2-point links, typically NanoStation to NanoStation or NanoBridge to NanoBridge, that, when any change is made, simply die. They do not reconnet. The station does not reattach to the AP. This has recently happened with a NanoStation XW running 6.1.3 and a NanoBridge running 6.1.9.

These used to be "trusty" and always reconnected. But I'm losing faith here. The only things that have changed, other than the usual firmware updates, are the band occupancy levels. Both of the failed links have plenty of SNR, and were solid, but Site Survey picks up over 300 beacons. (They're in the city. Lots of cable WiFi.) They were on DFS channels that weren't themselves all that noisy.

Any suggestions to salvage these radios, or are they due for replacement?

 

Ubiquiti Employee
Posts: 11,657
Registered: ‎04-14-2017
Kudos: 2188
Solutions: 335

Re: NanoBridge and NanoStation M don't reconnect after minor change

What firmware versions are you currently using?
Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

Re: NanoBridge and NanoStation M don't reconnect after minor change

(Quoting my original post) This has recently happened with a NanoStation XW running 6.1.3 and a NanoBridge running 6.1.9.

Ubiquiti Employee
Posts: 11,657
Registered: ‎04-14-2017
Kudos: 2188
Solutions: 335

Re: NanoBridge and NanoStation M don't reconnect after minor change

Ok. Can you go to the latest firmware version and see if this reoccurs?
Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

Re: NanoBridge and NanoStation M don't reconnect after minor change

No, I can't. The links are in service, on a critical public safety mission (owned by police, not a WISP), and if they fail, someone has to roll a truck to get there. Maybe in the future when we can add some redundancy to that part of the network.

But we're at 6.1.9 now and it failed. What if anything has changed in the Very Latest to deal with this issue? I am getting a sense, from some other recent problems we've had elsewhere, that NanoBridges are running out of steam in recent releases. Again, it's most noticeable in high sites that see a lot of WiFi traffic in the background.

Regular Member
Posts: 650
Registered: ‎11-13-2017
Kudos: 176
Solutions: 14

Re: NanoBridge and NanoStation M don't reconnect after minor change

your devices are working on what frequency, M2, M5 ?

 

If they didn't reconnect how you solved it? Somebody with a truck need to go there or just restart of the device?

 

Also when the problem occur anything in the log of the devices?

Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

Re: NanoBridge and NanoStation M don't reconnect after minor change

 

> your devices are working on what frequency, M2, M5 ?

 

5 GHz.  (M5)

 

> If they didn't reconnect how you solved it? Somebody with a truck need to go there or just restart of the device?

 

Yes.

 

> Also when the problem occur anything in the log of the devices?

 

Logs clear on reboots, of course. The AP was rebooted before the Station, and the Station didn't connect until it was rebooted. So for the first hour or so of the AP's log, there are no connections being made. Then the station comes on, presumaby being the time that it was power cycled by a technician who had to visit the site.

 

A question UBNT hasn't answered: Is there a limit to how many different (with or without SSID) beacons a Station can handle and still be able to pick up its designated AP? Maybe there's a table overflowing or something. Or maybe a MAC address limit on the 802.11 side.

Ubiquiti Employee
Posts: 1,592
Registered: ‎09-24-2018
Kudos: 315
Solutions: 92

Re: NanoBridge and NanoStation M don't reconnect after minor change

I'm not sure that we have such a limitation by design; typically a well-configured client station will re-scan forever until it finds the correct SSID/Security combo to establish a connection.

I understand that the link is highly critical, so I would recommend at least setting up a testbench recreation of it so you can play with things and prepare to update to new firmwares. You won't be able to reproduce the RF environment perfectly, but you can try to narrow down problems like this, and test firmware for issues that affect the operation of your bridge. It's very important to know if problems are resolved or introduced in specific releases, so updating firmware and testing for the issue again is imperative.

UBNT_Alternate_Logo.png
Ubiquiti Networks airMAX Support Team

Check out our ever-evolving Help Center for answers to many common questions!

FREE UBWA Student Guide-Great RF Primer!

Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

Re: NanoBridge and NanoStation M don't reconnect after minor change

We manage several hundred devices in the same network. Most are now on releases from 6.1.3 to 6.1.9, though there are some old ones. We rarely touch them once they're in. Most are on street lamp posts or traffic light mounts, with some feed points on tall roofs. The ones that give us trouble tend to be on high sites with a lot of RF noise. The most critical one that failed is up about 20 stories. So there's really nothing we can do to reproduce what was a random problem. If I had some kind of device to simulate the outdoor environment, with hundreds of signals all across the band, it might help, but I've never seen such a thing.

 

About 2-3 years ago, we pulled a fairly new 2-point NanoBridge M5 link from a pair of urban high-rise roofs. The station didn't reconnect after an interuption, though it did reconnect after about 2 days. (That delayed reconnection didn't happen on these last two -- one was too critical to wait more than a few minutes beyond DFS Wait before rolling the truck. The other sat about a week before a truck rolled by.) I have that pair here, as a lab radio. But of course it didn't show any issues indoors. The problem was intermittent anyway. The few hundred units on low street poles have been less problematic. But I'm getting worried... the number of APs they hear has skyrocketed. One went in about 4 years ago and heard about 55 beacons (I save Site Surveys). Now it hears over 300, and it's not up high (the AP is on a hill). I may see if, on a "safe" day, I can take one of the high site station sides of a redundantly-connected link (the only kind where I can maniupulate the station side remotely) to see what its CPU load looks like.

Ubiquiti Employee
Posts: 1,592
Registered: ‎09-24-2018
Kudos: 315
Solutions: 92

Re: NanoBridge and NanoStation M don't reconnect after minor change

I haven't seen such a long reconnection failure, but I also haven't worked with an RF environment so polluted as yours. It could be causing the problem with decoding the beacon correctly in a timely fashion.

UBNT_Alternate_Logo.png
Ubiquiti Networks airMAX Support Team

Check out our ever-evolving Help Center for answers to many common questions!

FREE UBWA Student Guide-Great RF Primer!

Regular Member
Posts: 650
Registered: ‎11-13-2017
Kudos: 176
Solutions: 14

Re: NanoBridge and NanoStation M don't reconnect after minor change

@fgoldstein 

 

hi, when technician will be there for a required restart at that point he can also update the firmware, as that can help you and UBNT staff need that for troubleshooting. No UBNT engineer will try find&fix a bug in old firmware.

 

Btw when technician will come for a restart, he should login first and download the log and also check web management if there is anything usable to see. As you figured out already blind devices restarting brings no troubleshooting.

 

I'm curious that 300 beacons are using which frequencies, the indoor ones 5150-5350 or outdoor 5470-5725 MHz ? When you make the scan based on AP's names they are home routers or outdoor radios?

Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

Re: NanoBridge and NanoStation M don't reconnect after minor change

PeterK, please read the original post. The units are pretty close to up to date in firmware (6.1.9 was current at the time, or within a month of the latest). And the technician can't realistically take a log. They do a power cycle. Pulling a lot takes a longer time and they have better things to do.

 

Oh, and aquí en los Estados Unidos, the frequencies are not whatever they are in your country. There are no indoor frequencies. The rules are very complex, and I know them backwards and forwards (it's part of what I do for a living). The cable modems now use both non-DFS bands, at high power, indoors, and the outdoor radios share frequencies with them.

Ubiquiti Employee
Posts: 1,592
Registered: ‎09-24-2018
Kudos: 315
Solutions: 92

Re: NanoBridge and NanoStation M don't reconnect after minor change

The first step will always be the suggestion to upgrade firmware, followed by finding clear spectrum. Especially in your situation, as you have so many beacons to decode, my strong suspicion remains to be almost insurmountable interference.

UBNT_Alternate_Logo.png
Ubiquiti Networks airMAX Support Team

Check out our ever-evolving Help Center for answers to many common questions!

FREE UBWA Student Guide-Great RF Primer!

Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

Re: NanoBridge and NanoStation M don't reconnect after minor change

The first step is probably to identify the problem, not make up excuses.

 

>The first step will always be the suggestion to upgrade firmware,

 

We're at 6.1.9, which is plenty current; 6.1.11 does not change anything we care about (it's mostly non-US stuff). And an upgrade requires a reboot, which means a rescan, and since rescans result in no connect, it probably means a truck roll. And at this point I would suggest that the techs never roll another truck to try to reboot those high sites without a replacement radio in tow, and that might be a different brand. Actually the more critical of the two, the really high site, WAS replaced this week by that other brand. So there's some real competitive loss happening.

 

>followed by finding clear spectrum. Especially in your situation, as you have so many beacons to decode, my strong suspicion remains to be almost insurmountable interference.

 

You again misunderstand the problem. The channel we were using was, in fact, plenty clear -- in at least one case it was DFS, and Comcast does not put its APs on DFS. (It used to be an option that nobody used; they appear to have disabled it.) So the SNR is good; CCQ is 97. The STATION side apparently never decodes the beacon and connects even though it's "loud and clear" enough on its channel. The station is probably getting hung up on the dozens of beacons on 5180, 5200, and 5220 before scanning higher. Note that an  AP can have a frequency list, so it only looks at those when looking for a DFS move channel, and its Site Survey likewise only checks those. A STATION cannot have a corresponding frequency list. It always tries to scan the whole band.

 

(One funny thing about AirMAX Site Survey is that it often stops somewhere around 5805, or maybe touches a beacon or two on 5825. Maybe that too is falling victim to a related issue. Or maybe not. We do however have some devices happily on 5840; their Stations just aren't in locations that see so much.)

Member
Posts: 213
Registered: ‎10-21-2013
Kudos: 90
Solutions: 5

Re: NanoBridge and NanoStation M don't reconnect after minor change

Minor correction - I do see frequency scan list on the station side. I don't generaly use it because it could result in getting out of sync with the AP side, especially on DFS.

Ubiquiti Employee
Posts: 1,592
Registered: ‎09-24-2018
Kudos: 315
Solutions: 92

Re: NanoBridge and NanoStation M don't reconnect after minor change

Given the information I currently read, all I can do is make an educated guess. Your problem sounds exactly like all the other ones with so many competing signals in the area. If you can get logs from a couple devices showing the problem, perhaps just a full support file, then I will be happy to check through it for you.

UBNT_Alternate_Logo.png
Ubiquiti Networks airMAX Support Team

Check out our ever-evolving Help Center for answers to many common questions!

FREE UBWA Student Guide-Great RF Primer!