Upcoming Maintenance Alert:

The UBNT Community will be upgraded at 5pm MDT on April 25th. During this time the community forums will be set to read-only status.

Learn more

×

Radio Shootout Pt.2 - let's try a whole bunch of them...

by ‎04-18-2015 01:55 PM - edited ‎04-23-2015 02:06 PM

Since the results of these tests (see Pt. 1 ) were providing some surprising results against the one Ubiquiti competitor's radio, it seemed like a good idea to see how some other types of radios fared too. And to make it more interesting, how about testing the radios for interference rejection on nearby channels?

ShootoutRadios.jpg

So we put together a group of radios that many WISPs use today or are looking at using – the Mimosa B5C (yes, I'm giving up the thinly veiled “Brand M” thing...), the Cambium EPMP 1000-GPS, the Mikrotik RB922, and from Ubiquiti the AF5X, Rocket AC Lite, and the Rocket AC-PTP (with the Prism Chip).

 

Just as a point of reference, here are the Prices of the various radios as of today:

InterferenceTestRadioMSRPs.png

Here's a diagram of the test setup:

InterferenceTestSetupDiag.png

For those who may not be familiar with more advanced microwave components, some elaboration is in order. To do these tests an interfering signal needed to be introduced at a specifically defined power level without either completely disrupting the link between the radios under test, or harming them by coupling the interfering signal directly to the receiver and blowing something up. This was done with a combination of fixed attenuators and a pair of Directional Couplers. These are devices that are designed to couple or extract a reduced signal level from a RF signal path, but in only one direction and at a specific coupling ratio. The directionality is veery useful when you want to look at things like reflected signals from terminating components (like antennas or amplifiers) and make it easy to add signals to an RF path without disrupting things. They work by arranging specific stripline paths or waveguides with coupling ports into a passive device – for more detailed info see http://www.microwaves101.com/encyclopedias/directional-couplers

The coupling port will either extract a portion of the signal going through the Coupler, or can add signals to the straight through path, at some fixed attenuation level, typically 20-30 dB down. Internally, these are 4 port devices, with the two coupling ports being directionally isolated – the coupling port picks up or inserts the signal in one direction only. For the three-port versions (like used in these tests) the 4th port is just terminated with a 50 ohm load internally. The math behind this is complex, but it works very well.

 

For this test, a pair of directional couplers were set up with the direct through ports connected to the radios being tested, and the interference source connected to the coupling port. This way, a specific level of interference can be injected into the test path.

 

To generate the interference, a standard RocketM5 was used. Atheros (the chipset manufacturer) has a test program which can be used to make the radio do all kinds of things it normally wouldn't do, and in this case it was used to make it transmit continuously with full modulation to simulate very strong interference from a nearby radio. Unless you are doing testing like this, don't try this yourself, and definitely don't do it on a live radio in the field, unless your goal is to interfere with everyone else.

 

The tests

 

For this series of tests, each radio pair was set up on a 20MHz channel at a nominal received signal level, and the interfering signal coupled in at differing levels from the same level as the data channel to 30dB more than the data carrier. Each test series was run with the interfering signal offset from the test radio's carrier frequency by different distances: 10MHz edge-to-edge (30MHz Center-to-Center), 30MHz EtoE (50MHz CtoC), 50MHz EtoE (70MHz CtoC) and 70MHz EtoE (90MHz CtoC) were all tested. The throughput using Iperf in tcp mode was plotted vs. The interfering ratio which is how much stronger in dB the interfering signal is than the received signal. So if the received signal is -60 dBm and the interferer is at -30 at the tested unit's receiver, emulating a very nearby transmitter, and the plotted level would be 30dBc (the worst the test tried, since all the radios were basically unusable by then.)

InterferenceTestSetupChans.png

Each radio pair was tested at the 4 different interferer frequency offset values across a power range of 0dB (interferer same level as the desired signal) to 30dB (30dB more interferer than desired). All 6 radios were plotted together for each frequency offset value, and are shown below.

Throughput_bidir_cfg9.png

 

Throughput_bidir_cfg8.png

Throughput_bidir_cfg7.png

Throughput_bidir_cfg6.png

Results:

The clear winner here is the AF5X, followed by the RocketAC PtP and the Rocket AC Lite. The AF5X's superior filtering ability really shines, with it's ability to ignore interference at all but the closest offset being identical. The AC PtP is actually better at 30MHz offset or more due to the active filtering in the Prism Chip, but it starts to degrade significantly with 10MHz spacing from the main carrier edge.

The MiktoTik is next best with heavier interference levels, and is actually slightly ahead of the AC Lite and the AC PtP at the closest spacing and highest interference levels, although that's of dubious usefulness, as the channel throughput is down under 20Mb at that point.

The Cambium is a surprise here, both for it's relative suceptibility to interferrence at all levels and rather poor performance overall. To look a little deeper as to why, opening it up revealed that it uses the same chipset as the UBNT Rocket5M, so that's probably why. So much for Cambium...

The Mimosa is not too far behind the AC Lite speed wise at lower interference levels, but quickly drops behind the others as the interference level increases. So it's overall performance in heavy interference puts it at the back of the pack, unless you use it's dual channel function to approximately double the throughput (while eating up 2x the bandwidth...) But even doubling the throughput under heavy interference won't get you up to the level of the other radios except the EPMP, which is pretty bad.

 

So what does this tell us? Well, the best performance overall in both throughput and interference handling is the AF5X, followed by the Rocket5 AC PtP and AC Lite. Even at moderately high interference levels, they all outperform the competition handily, and under heavy interference where the other radios have basically given up, the AF5X and PTP are still able to function remarkably well

 

The two biggest surprises here were the B5c and the EPMP, and neither in a good way. For an $839 radio to perform so much worse than a $135 AC Lite was a shock.   Part of this is possibly due to their having to make everything work over such a wide bandwidth all the time to support the dual channel operation, but it's still not good.   And for the EPMP to so drastically underperform across the board, well I suppose if you're a Cambium shop...

 

So in the end, what does this prove?   Well, IMHO it means the Ubiquiti gear is much better engineered, for one thing.   And it's real world performance (when things are kept legal - more on that in a minute) is going to be better than the competition.   I'm not going to say any more than that our decision to keep to Ubiquiti gear keeps being proven to us to be the correct decision over and over again.

 

Now about those folks who have been reporting about how great the B5c radios are working for them - in at least one case I know of that's because they are being run at illegal power levels.   One link in particular I'm aware of is operating at 56dB EIRP on DFS (30dBm EIRP max legal) channels to get 2x80 to work.   Now there are probably people out there doing this with other kinds of radios too, but if that's what it takes to get those "amazing" numbers, then that's not a real solution.   So beware of claims about how well Brand X is working over some huge distance without knowing exactly what the parameters are.

Jim

 

 

EDIT - 

Several people asked for more results for on-channel and adjacent channel interference after I posted this. Initially I wasn't going to do this, but it made sense to include this information as well as all the previous data. So here is the comparison of the AF5X and the B5.

 

To look at a worst case scenario of what happens when the interfering signal is directly adjacent to the operating channels with zero guard space between, in this test, the received signal levels of each test receiver were carefully measured (not from the radio's GUI) to insure they both had a received level of -52dBm. That way any differences or inaccuracies in the radios' conducted power, EIRP calculations or RSSI values would be eliminated.

 

The channel offset (center frequency of the operating channel to the interferer) was varied from 0 (directly on top of each other) to 2 times the channel bandwidth, which leaves a guard band between them of between zero and 1x the bandwidth – 20MHz in this case as that was the channel width used for this test. The interfering signal level was increased until the throughput of the radio channel began to be affected, (in this case a 20% reduction) and was plotted at that point for both the AF5X and the B5. The chart below shows the dB level the interferer needs to be higher than the desired signal at an offset of 1x – no spacing between the two signals at all – literally edge-to-edge – for both above and below the channel.

AF5B5IntChart.png

It's clear that the AF5X can handle quintuple (7dB more or 5x) the interference level of the B5 in both cases. This is very significant in a heavily congested RF environment like we're all finding more and more today.

 

Here is the complete plot of the test results – the x-axis is the channel offset in channel width from right on top of the channel in the center to 2x the channel width above and below the center frequency. The y-axis is the interferer level in dB above the received signal (-52dBm) needed to begin to degrade the throughput of the link.

AF5B5IntPlot.png

So to summarize, the AF5X has far superior (5 times) the zero-guard band interference rejection of the B5 at exactly the same received signal strength. That makes a huge difference in operations at congested sites, and allows for much easier frequency planning as well.

" How can anyone trust Scientists? If new evidence comes along, they change their minds! " Politician's joke (sort of...)

"Humans are allergic to change..They love to say, ‘We’ve always done it this way.’ I try to fight that. "Admiral Grace Hopper, USN, Computer Scientist
Comments
by
on ‎04-20-2015 10:28 AM

@popcorrin there's ack issues in the current AC stuff, others have complained about this as well

by
on ‎04-20-2015 10:30 AM
by
on ‎04-20-2015 10:32 AM

@mhammett "*silence*" - Current UBNT AC Firmware

by Ubiquiti Employee
on ‎04-21-2015 10:15 AM

@Alver  just to make it clear related to the AirFiber line.  We do not use AirPrism in the design of the product.  AirFiber radios are purpose-built from the ground up with a custom design.  Just wanted to make that clear.

by
on ‎04-21-2015 05:38 PM

@UBNT-Alex just on that note, AirPrism on airMAX lines are to improve the filtering on the otherwise off-the-shelf WiFi chipsets used. Does the AirFiber line have equivalent fancy filtering as standard as a part of the purpose-built chips so adding AirPrism in front would change nothing? Or would there still be some advantage to having an AirPrism equivalent sitting in front of the custom AF radio?

by
on ‎04-21-2015 06:31 PM

NVX, In my testing, the AF5X has an even better adjacent channel filter than AirPrism radios do.   Jim's testing here backs my field observations on this.

by Ubiquiti Employee
on ‎04-21-2015 06:40 PM

NVX,

 

I wish I could figure out how to find my old posts on the forum because I addressed this in detail some months ago.  The answer is no.  

 

The AF radio (in this particular case) uses a Zero IF (ZIF) architecture that has a built-in configurable very high order digital filter for selectivity.  It results in a filter passband that is not realizable through analog means (like airPrism) so there would be no point to it.  The filter inside the AFX platform can adapt to different operating conditions (like interference) and has far more equivalent poles than airPrism.  As has been pointed out, airPrism is a way to improve the relatively modest selectivity associated with standard WiFi chips as used by others. The addition of external analog filters enhances the performance and provides a significant performance benefit.  I hope this makes some sense?

 

 

by
‎04-21-2015 07:17 PM - edited ‎04-21-2015 07:20 PM

Makes sense, although I thought the Atheros WiFi chipsets were Zero IF as well, so that wouldn't be the differing factor in this case, but the built in configurable high order digital filter would be (which, from my understanding is what airPrism is anyway, just in a separate chip whereas from the sounds of it AirFiber has it integrated in with the tuner itself?)

 

In any case you guys should really harass your marketing guys so that it's clearer that the AF gear has superior selectivity than airPrism devices, cause the amount airPrism is praised in the airMAX AC datasheets then seeing its absence in the AF datasheets the uneducated could think it's a poorer choice for noisy environments!

 

Edit: Guessing this is the other post you're referring to? http://community.ubnt.com/t5/airFiber/Aifiber-little-spectrum-left/m-p/1181371#M18098

by
on ‎04-22-2015 09:00 AM

Awesome very very good that you did these tests!

by
on ‎04-27-2015 12:16 PM

Great writeup, *but* as has been aluded to, the real world negates some of these results.  Over long distances, the RocketM5 has shown to be superior to the RocketAC-Lite for me as the M5 maintains usable throughput to lower signal levels.  The flip side, the AF5X is many times better at long distance plus noise than the RocketM5 or AC-lite.

 

A brief example is that I replaced an AC-lite link w/ an AF5X recently.  The AC-lite link actually performed worse than the RocketM5 link it recently replaced (about a month previous) only pulling in around 20Mbps aggregate across 13 miles.  The Rocket M5 was around 30Mbps aggregate.  The AF5X is 100Mbps aggregate on the same channel as the RocketM5 (5180) same 40Mhz.  The AC-lite was up in UNII3 for lack of access to 5180Mhz.

 

From my experience the AF5X is even more superior than the tests suggest, and that the RocketAC line's superiority falls off at around 6 miles and the RocketM5 takes over #2 spot.  I've helped install a B5 and a B5c and the owner of those is happy with them, but now that I'm running AF5X I wouldn't consider the B5*.

 

Jim, I think what your tests lack is variables.  Though an excellent test, your decouplers and attenuators are essentially static.  What the real world is throwing in the mix is the variability from random partial-duty cycle links for which my observations tell me the RocketM5 is beating the non-airprism AC-lite, and that the cambium epmp link accells at.  Still a fantastic test, but it's definitely 'lab conditions'