Upcoming Maintenance Alert:

The UBNT Community will be upgraded at 5pm MDT on April 25th. During this time the community forums will be set to read-only status.

Learn more

×
04/18/2015
Radio Shootout Pt.2 - let's try a whole bunch of them...
Description

Since the results of these tests (see Pt. 1 ) were providing some surprising results against the one Ubiquiti competitor's radio, it seemed like a good idea to see how some other types of radios fared too. And to make it more interesting, how about testing the radios for interference rejection on nearby channels?

ShootoutRadios.jpg

So we put together a group of radios that many WISPs use today or are looking at using – the Mimosa B5C (yes, I'm giving up the thinly veiled “Brand M” thing...), the Cambium EPMP 1000-GPS, the Mikrotik RB922, and from Ubiquiti the AF5X, Rocket AC Lite, and the Rocket AC-PTP (with the Prism Chip).

 

Just as a point of reference, here are the Prices of the various radios as of today:

InterferenceTestRadioMSRPs.png

Here's a diagram of the test setup:

InterferenceTestSetupDiag.png

For those who may not be familiar with more advanced microwave components, some elaboration is in order. To do these tests an interfering signal needed to be introduced at a specifically defined power level without either completely disrupting the link between the radios under test, or harming them by coupling the interfering signal directly to the receiver and blowing something up. This was done with a combination of fixed attenuators and a pair of Directional Couplers. These are devices that are designed to couple or extract a reduced signal level from a RF signal path, but in only one direction and at a specific coupling ratio. The directionality is veery useful when you want to look at things like reflected signals from terminating components (like antennas or amplifiers) and make it easy to add signals to an RF path without disrupting things. They work by arranging specific stripline paths or waveguides with coupling ports into a passive device – for more detailed info see http://www.microwaves101.com/encyclopedias/directional-couplers

The coupling port will either extract a portion of the signal going through the Coupler, or can add signals to the straight through path, at some fixed attenuation level, typically 20-30 dB down. Internally, these are 4 port devices, with the two coupling ports being directionally isolated – the coupling port picks up or inserts the signal in one direction only. For the three-port versions (like used in these tests) the 4th port is just terminated with a 50 ohm load internally. The math behind this is complex, but it works very well.

 

For this test, a pair of directional couplers were set up with the direct through ports connected to the radios being tested, and the interference source connected to the coupling port. This way, a specific level of interference can be injected into the test path.

 

To generate the interference, a standard RocketM5 was used. Atheros (the chipset manufacturer) has a test program which can be used to make the radio do all kinds of things it normally wouldn't do, and in this case it was used to make it transmit continuously with full modulation to simulate very strong interference from a nearby radio. Unless you are doing testing like this, don't try this yourself, and definitely don't do it on a live radio in the field, unless your goal is to interfere with everyone else.

 

The tests

 

For this series of tests, each radio pair was set up on a 20MHz channel at a nominal received signal level, and the interfering signal coupled in at differing levels from the same level as the data channel to 30dB more than the data carrier. Each test series was run with the interfering signal offset from the test radio's carrier frequency by different distances: 10MHz edge-to-edge (30MHz Center-to-Center), 30MHz EtoE (50MHz CtoC), 50MHz EtoE (70MHz CtoC) and 70MHz EtoE (90MHz CtoC) were all tested. The throughput using Iperf in tcp mode was plotted vs. The interfering ratio which is how much stronger in dB the interfering signal is than the received signal. So if the received signal is -60 dBm and the interferer is at -30 at the tested unit's receiver, emulating a very nearby transmitter, and the plotted level would be 30dBc (the worst the test tried, since all the radios were basically unusable by then.)

InterferenceTestSetupChans.png

Each radio pair was tested at the 4 different interferer frequency offset values across a power range of 0dB (interferer same level as the desired signal) to 30dB (30dB more interferer than desired). All 6 radios were plotted together for each frequency offset value, and are shown below.

Throughput_bidir_cfg9.png

 

Throughput_bidir_cfg8.png

Throughput_bidir_cfg7.png

Throughput_bidir_cfg6.png

Results:

The clear winner here is the AF5X, followed by the RocketAC PtP and the Rocket AC Lite. The AF5X's superior filtering ability really shines, with it's ability to ignore interference at all but the closest offset being identical. The AC PtP is actually better at 30MHz offset or more due to the active filtering in the Prism Chip, but it starts to degrade significantly with 10MHz spacing from the main carrier edge.

The MiktoTik is next best with heavier interference levels, and is actually slightly ahead of the AC Lite and the AC PtP at the closest spacing and highest interference levels, although that's of dubious usefulness, as the channel throughput is down under 20Mb at that point.

The Cambium is a surprise here, both for it's relative suceptibility to interferrence at all levels and rather poor performance overall. To look a little deeper as to why, opening it up revealed that it uses the same chipset as the UBNT Rocket5M, so that's probably why. So much for Cambium...

The Mimosa is not too far behind the AC Lite speed wise at lower interference levels, but quickly drops behind the others as the interference level increases. So it's overall performance in heavy interference puts it at the back of the pack, unless you use it's dual channel function to approximately double the throughput (while eating up 2x the bandwidth...) But even doubling the throughput under heavy interference won't get you up to the level of the other radios except the EPMP, which is pretty bad.

 

So what does this tell us? Well, the best performance overall in both throughput and interference handling is the AF5X, followed by the Rocket5 AC PtP and AC Lite. Even at moderately high interference levels, they all outperform the competition handily, and under heavy interference where the other radios have basically given up, the AF5X and PTP are still able to function remarkably well

 

The two biggest surprises here were the B5c and the EPMP, and neither in a good way. For an $839 radio to perform so much worse than a $135 AC Lite was a shock.   Part of this is possibly due to their having to make everything work over such a wide bandwidth all the time to support the dual channel operation, but it's still not good.   And for the EPMP to so drastically underperform across the board, well I suppose if you're a Cambium shop...

 

So in the end, what does this prove?   Well, IMHO it means the Ubiquiti gear is much better engineered, for one thing.   And it's real world performance (when things are kept legal - more on that in a minute) is going to be better than the competition.   I'm not going to say any more than that our decision to keep to Ubiquiti gear keeps being proven to us to be the correct decision over and over again.

 

Now about those folks who have been reporting about how great the B5c radios are working for them - in at least one case I know of that's because they are being run at illegal power levels.   One link in particular I'm aware of is operating at 56dB EIRP on DFS (30dBm EIRP max legal) channels to get 2x80 to work.   Now there are probably people out there doing this with other kinds of radios too, but if that's what it takes to get those "amazing" numbers, then that's not a real solution.   So beware of claims about how well Brand X is working over some huge distance without knowing exactly what the parameters are.

Jim

 

 

EDIT - 

Several people asked for more results for on-channel and adjacent channel interference after I posted this. Initially I wasn't going to do this, but it made sense to include this information as well as all the previous data. So here is the comparison of the AF5X and the B5.

 

To look at a worst case scenario of what happens when the interfering signal is directly adjacent to the operating channels with zero guard space between, in this test, the received signal levels of each test receiver were carefully measured (not from the radio's GUI) to insure they both had a received level of -52dBm. That way any differences or inaccuracies in the radios' conducted power, EIRP calculations or RSSI values would be eliminated.

 

The channel offset (center frequency of the operating channel to the interferer) was varied from 0 (directly on top of each other) to 2 times the channel bandwidth, which leaves a guard band between them of between zero and 1x the bandwidth – 20MHz in this case as that was the channel width used for this test. The interfering signal level was increased until the throughput of the radio channel began to be affected, (in this case a 20% reduction) and was plotted at that point for both the AF5X and the B5. The chart below shows the dB level the interferer needs to be higher than the desired signal at an offset of 1x – no spacing between the two signals at all – literally edge-to-edge – for both above and below the channel.

AF5B5IntChart.png

It's clear that the AF5X can handle quintuple (7dB more or 5x) the interference level of the B5 in both cases. This is very significant in a heavily congested RF environment like we're all finding more and more today.

 

Here is the complete plot of the test results – the x-axis is the channel offset in channel width from right on top of the channel in the center to 2x the channel width above and below the center frequency. The y-axis is the interferer level in dB above the received signal (-52dBm) needed to begin to degrade the throughput of the link.

AF5B5IntPlot.png

So to summarize, the AF5X has far superior (5 times) the zero-guard band interference rejection of the B5 at exactly the same received signal strength. That makes a huge difference in operations at congested sites, and allows for much easier frequency planning as well.

Comments
by
on ‎04-18-2015 10:23 PM

Hi,  Im considered a non experted user and with very basics about RF.  But Im a user of cambium and ubnt gear.

 

My question is why to use/choose the more expensive radio from ubnt , i guess the sincronization feature do not provides any advantage on ths scenario , and prices is almos the 3 times againts the non GPS radio. the epmp  radio is not more like a rocket m5 radio ? 

 

Whata are the results from a Rocket M5 radios results on this test ?

Thanks in advanced.

 

 

 

 

 

by
on ‎04-18-2015 11:07 PM

Could you please list what firmware you use on each radio, along with some of the basic configurations on each link... e.g. depending on what ePMP firmware you're running there are 3 possible PtP modes, each with different performance and latency characteristics.

by
‎04-19-2015 02:00 AM - edited ‎04-19-2015 02:06 AM

 eejimm in these  tests simply measured  Adjacent Channel Rejection  of various radios  . ACR is important parameter, but it does not give any estimation of interference resistance  of ePMP , ubnt  and Mikrotik radio. Really  these tests showed following :

- ACR  of Atheros  802.11n  and ubnt/mikrotik  АС  radio  at 2.5  adjacent channel - 30 MHz guard interval (GI)  in 20MHz channel bandwidth , roughtly  are  about  15-17 dB. It is not good, but is not so  bad.

- Mimosa  radio ACR  is much worse – less than  12 dB at 30 MHz guard interval. It is  not  good.

- ACR of ubnt AF5X   it seems due to Airprizm filter is about 30 dB at  (GI  >30 MHz). But we see that at GI= 20 MHz  and less  ACR is the same as  other systems  without filter. So airprizm does not works at 1st  and probably at 2nd  adjacent channels, so  it makes this type of filter at  low GI  unefficient.

In regards to measure of thoughput.

 ePMP aggregate throughput  60-70 mbps without interference ( 70 mHz GI) is not correct. It should have been 110-120Mbps. Something   wrong   in configuration, may be   DL/UL ratio  is 50/50 and  simplex ( one way) speed  is measured .   Also   ePMP works at  64QAM  and  comparison    with  other system at   256QAM  is not correct.  

So these tests  are   usual  throughput  comparison  at  64QAM 5/6 and 256QAM modulation and ACR measurement  various systems and nothing more.

 eejimm should provide tests  with   interference  at  the same frequency channels  (co-channel interference)   with   C/I = 10, 20,30 dB. These results will be more interesting and usefull

by
on ‎04-19-2015 02:07 AM

epmp is a suprise here , but ok gsp model is not needed in this secenario so price is more like rocketM5 or aclite, and still in wide use ... meMan Happy. I woud like to se RocketM5 in coparations. All radios are atheros wifi based except arifiber .. I think !?

 

 

by
on ‎04-19-2015 05:37 AM

@Makoto Mimosa is not Atheros based, it is Quantenna based.

 

I would like to see the PMP450 and PTP650 series run through the same tests. yes, I know they're expensive and no, I don't have any to donate to the test. Maybe someone else can?

by
on ‎04-19-2015 07:59 AM

Everyone - I understand some of the frustration of only seeing limited testing of a limited number of radios here, but I do have a business to run, and a lot of other things take up the majority of my time (which is why I'm writing this on Sunday morning - only so many hours in the week...)   So there's no way I can do more tests on more radios right now - this just happened to be somethig that was started for some other reasons, and was fairly easy to do at the time.

 

Please feel free to do whatever additional testing/comparisons you feel approrpiate, and post your results too - more data is always better than less.  I'm sure I'll do more in the future - this really started by wanting to compare AF5X with B5c, and sort expanded from there - we're going to be doing some testing on long links in the field soon, and I'll post those results when I get them.

Jim

by
on ‎04-19-2015 08:02 AM

@eejimm Agreed that more data is better than less and that time is precious. Not everyone has your skills in creating whatever devices it is you have in the middle. Hopefully others can piece these things together and do some further testing.

by
on ‎04-19-2015 02:00 PM

Awesome tests.

 

I'd be curious to see what the results are of on-channel interference as well (albeit at much lower levels I'm sure). Would  be a great reference to point people to when they think that airPrism is going to solve their on-channel interference issues. Man Very Happy - I also have a suspicion that the AF5x would cope better than the airMAX AC units with on-channel interference, but it would be good to actually have some numbers rather than a gut feeling on the matter haha.

by
‎04-19-2015 02:28 PM - edited ‎04-19-2015 07:48 PM

How close do you think the Cambium proximates a standard Rocket M5 or Airmax N radio since they use the same chipsets? Do you think the Cambium results would be about the same for the "N" Airmax line? Like if you replaced the Cambium with another rocket m5 radio?

by
on ‎04-20-2015 06:00 AM

I like seeing test comparing multiple vendors equipment. It really helps the end user decide on which equipment to use because we all know you can't trust marketing. And not to take anything away from Jim, like he said, he has a business to run and this was done on his spare time, but from my experience these results don't relate well to real world performance.  At least for the Rocket AC Lite and the epmp 1000.  We've tested the mikrotik netmetal, epmp 1000, rocket AC Lite, and now most recently the AF5X on real world links ranging from 5 miles to just shy of 20 miles and while the AF5x blows the rest of them out of the water, the next best performing radio is the epmp 1000, followed by the mikrotik, and then the AC Lite(mind you we didn't test the AC PTP or Brand "M").  

Our 20 mile link had very little noise and on a 20mhz channel the rocket ac lite would only push 70mbps tcp, only slightly better than the rocket M it replaced.  The epmp 1000 pushed 110mbps tcp.   On a 10 mile link with heavy noise, the AC lite would barely pass traffic, maybe 10 mbps,  whereas the mikrotik netmetal was able to push 60mbps tcp.   Maybe it's a software issue or something to do with ACK calculation but the AC Lites left us disappointed, on the other hand we were very impressed with the epmp considering it has an 802.11N chipset and it was outperforming AC radios.

 

In the end, it really doesn't matter though, I can't imagine using any 5ghz radio for a backhaul other than the AF5x or AF5 at this point.