Upcoming Maintenance Alert:

The UBNT Community will be upgraded at 5pm MDT on April 25th. During this time the community forums will be set to read-only status.

Learn more

×
04/18/2015
Radio Shootout Pt.2 - let's try a whole bunch of them...
Description

Since the results of these tests (see Pt. 1 ) were providing some surprising results against the one Ubiquiti competitor's radio, it seemed like a good idea to see how some other types of radios fared too. And to make it more interesting, how about testing the radios for interference rejection on nearby channels?

ShootoutRadios.jpg

So we put together a group of radios that many WISPs use today or are looking at using – the Mimosa B5C (yes, I'm giving up the thinly veiled “Brand M” thing...), the Cambium EPMP 1000-GPS, the Mikrotik RB922, and from Ubiquiti the AF5X, Rocket AC Lite, and the Rocket AC-PTP (with the Prism Chip).

 

Just as a point of reference, here are the Prices of the various radios as of today:

InterferenceTestRadioMSRPs.png

Here's a diagram of the test setup:

InterferenceTestSetupDiag.png

For those who may not be familiar with more advanced microwave components, some elaboration is in order. To do these tests an interfering signal needed to be introduced at a specifically defined power level without either completely disrupting the link between the radios under test, or harming them by coupling the interfering signal directly to the receiver and blowing something up. This was done with a combination of fixed attenuators and a pair of Directional Couplers. These are devices that are designed to couple or extract a reduced signal level from a RF signal path, but in only one direction and at a specific coupling ratio. The directionality is veery useful when you want to look at things like reflected signals from terminating components (like antennas or amplifiers) and make it easy to add signals to an RF path without disrupting things. They work by arranging specific stripline paths or waveguides with coupling ports into a passive device – for more detailed info see http://www.microwaves101.com/encyclopedias/directional-couplers

The coupling port will either extract a portion of the signal going through the Coupler, or can add signals to the straight through path, at some fixed attenuation level, typically 20-30 dB down. Internally, these are 4 port devices, with the two coupling ports being directionally isolated – the coupling port picks up or inserts the signal in one direction only. For the three-port versions (like used in these tests) the 4th port is just terminated with a 50 ohm load internally. The math behind this is complex, but it works very well.

 

For this test, a pair of directional couplers were set up with the direct through ports connected to the radios being tested, and the interference source connected to the coupling port. This way, a specific level of interference can be injected into the test path.

 

To generate the interference, a standard RocketM5 was used. Atheros (the chipset manufacturer) has a test program which can be used to make the radio do all kinds of things it normally wouldn't do, and in this case it was used to make it transmit continuously with full modulation to simulate very strong interference from a nearby radio. Unless you are doing testing like this, don't try this yourself, and definitely don't do it on a live radio in the field, unless your goal is to interfere with everyone else.

 

The tests

 

For this series of tests, each radio pair was set up on a 20MHz channel at a nominal received signal level, and the interfering signal coupled in at differing levels from the same level as the data channel to 30dB more than the data carrier. Each test series was run with the interfering signal offset from the test radio's carrier frequency by different distances: 10MHz edge-to-edge (30MHz Center-to-Center), 30MHz EtoE (50MHz CtoC), 50MHz EtoE (70MHz CtoC) and 70MHz EtoE (90MHz CtoC) were all tested. The throughput using Iperf in tcp mode was plotted vs. The interfering ratio which is how much stronger in dB the interfering signal is than the received signal. So if the received signal is -60 dBm and the interferer is at -30 at the tested unit's receiver, emulating a very nearby transmitter, and the plotted level would be 30dBc (the worst the test tried, since all the radios were basically unusable by then.)

InterferenceTestSetupChans.png

Each radio pair was tested at the 4 different interferer frequency offset values across a power range of 0dB (interferer same level as the desired signal) to 30dB (30dB more interferer than desired). All 6 radios were plotted together for each frequency offset value, and are shown below.

Throughput_bidir_cfg9.png

 

Throughput_bidir_cfg8.png

Throughput_bidir_cfg7.png

Throughput_bidir_cfg6.png

Results:

The clear winner here is the AF5X, followed by the RocketAC PtP and the Rocket AC Lite. The AF5X's superior filtering ability really shines, with it's ability to ignore interference at all but the closest offset being identical. The AC PtP is actually better at 30MHz offset or more due to the active filtering in the Prism Chip, but it starts to degrade significantly with 10MHz spacing from the main carrier edge.

The MiktoTik is next best with heavier interference levels, and is actually slightly ahead of the AC Lite and the AC PtP at the closest spacing and highest interference levels, although that's of dubious usefulness, as the channel throughput is down under 20Mb at that point.

The Cambium is a surprise here, both for it's relative suceptibility to interferrence at all levels and rather poor performance overall. To look a little deeper as to why, opening it up revealed that it uses the same chipset as the UBNT Rocket5M, so that's probably why. So much for Cambium...

The Mimosa is not too far behind the AC Lite speed wise at lower interference levels, but quickly drops behind the others as the interference level increases. So it's overall performance in heavy interference puts it at the back of the pack, unless you use it's dual channel function to approximately double the throughput (while eating up 2x the bandwidth...) But even doubling the throughput under heavy interference won't get you up to the level of the other radios except the EPMP, which is pretty bad.

 

So what does this tell us? Well, the best performance overall in both throughput and interference handling is the AF5X, followed by the Rocket5 AC PtP and AC Lite. Even at moderately high interference levels, they all outperform the competition handily, and under heavy interference where the other radios have basically given up, the AF5X and PTP are still able to function remarkably well

 

The two biggest surprises here were the B5c and the EPMP, and neither in a good way. For an $839 radio to perform so much worse than a $135 AC Lite was a shock.   Part of this is possibly due to their having to make everything work over such a wide bandwidth all the time to support the dual channel operation, but it's still not good.   And for the EPMP to so drastically underperform across the board, well I suppose if you're a Cambium shop...

 

So in the end, what does this prove?   Well, IMHO it means the Ubiquiti gear is much better engineered, for one thing.   And it's real world performance (when things are kept legal - more on that in a minute) is going to be better than the competition.   I'm not going to say any more than that our decision to keep to Ubiquiti gear keeps being proven to us to be the correct decision over and over again.

 

Now about those folks who have been reporting about how great the B5c radios are working for them - in at least one case I know of that's because they are being run at illegal power levels.   One link in particular I'm aware of is operating at 56dB EIRP on DFS (30dBm EIRP max legal) channels to get 2x80 to work.   Now there are probably people out there doing this with other kinds of radios too, but if that's what it takes to get those "amazing" numbers, then that's not a real solution.   So beware of claims about how well Brand X is working over some huge distance without knowing exactly what the parameters are.

Jim

 

 

EDIT - 

Several people asked for more results for on-channel and adjacent channel interference after I posted this. Initially I wasn't going to do this, but it made sense to include this information as well as all the previous data. So here is the comparison of the AF5X and the B5.

 

To look at a worst case scenario of what happens when the interfering signal is directly adjacent to the operating channels with zero guard space between, in this test, the received signal levels of each test receiver were carefully measured (not from the radio's GUI) to insure they both had a received level of -52dBm. That way any differences or inaccuracies in the radios' conducted power, EIRP calculations or RSSI values would be eliminated.

 

The channel offset (center frequency of the operating channel to the interferer) was varied from 0 (directly on top of each other) to 2 times the channel bandwidth, which leaves a guard band between them of between zero and 1x the bandwidth – 20MHz in this case as that was the channel width used for this test. The interfering signal level was increased until the throughput of the radio channel began to be affected, (in this case a 20% reduction) and was plotted at that point for both the AF5X and the B5. The chart below shows the dB level the interferer needs to be higher than the desired signal at an offset of 1x – no spacing between the two signals at all – literally edge-to-edge – for both above and below the channel.

AF5B5IntChart.png

It's clear that the AF5X can handle quintuple (7dB more or 5x) the interference level of the B5 in both cases. This is very significant in a heavily congested RF environment like we're all finding more and more today.

 

Here is the complete plot of the test results – the x-axis is the channel offset in channel width from right on top of the channel in the center to 2x the channel width above and below the center frequency. The y-axis is the interferer level in dB above the received signal (-52dBm) needed to begin to degrade the throughput of the link.

AF5B5IntPlot.png

So to summarize, the AF5X has far superior (5 times) the zero-guard band interference rejection of the B5 at exactly the same received signal strength. That makes a huge difference in operations at congested sites, and allows for much easier frequency planning as well.

Comments
by
on ‎04-18-2015 02:12 PM

WOW Jim this is just blowing my Mind !!! Kudos for that ! Very well done !

by
on ‎04-18-2015 02:40 PM

Cracking read. Very enlightening.

by
on ‎04-18-2015 02:56 PM

An N chip (ePMP) should be expected to perform worse than an AC chip in throughput tests.

by
on ‎04-18-2015 03:23 PM

You do realize the ePMP is the only N radio on the list, right? It should not be such a surprise that an N radio can't match the throughput of an AC radio.

by
on ‎04-18-2015 03:28 PM
N or not, it degrades much faster than anything but the B5C.
by
on ‎04-18-2015 03:55 PM

And it "costs" more than a "custom radio" (AF5X) or both "AC Radios" from UBNT Biggrin

 

Leaving

by
‎04-18-2015 04:43 PM - edited ‎04-18-2015 04:45 PM

Jim, this is an awesome writeup. But your results don't hold up in the real world when you throw distance into the equation. AFx is a clear winner. Airmax AC falls on its face when distance comes into the equation.

 

However, adjacent channel info is spot on.

by
on ‎04-18-2015 05:32 PM

I am surprised to see how well the lite held up to some noise!

by
on ‎04-18-2015 05:46 PM

Thanks for doing this writeup Jim! Great insight!

by
on ‎04-18-2015 10:13 PM

Very valuable data, few people have a chance compare all of them!