Radio Shootout Pt 1 - AC Lite vs. Brand M

by ‎04-17-2015 03:49 PM - edited ‎04-17-2015 04:10 PM

Mim-AC.JPG

I was initially going to compare the AF5X with the Brand M radio (in the picture above) so that everyone could see how the two really compare.   But once I did some baseline throughput testing on the bench, it became obvious that Brand M wasn't really in the same league as the AF5X, and was really more like the AC radios (which is actually what it is - it even says "a/n/ac" in one of it's wireless mode selections).   Below is a quick look at the max speeds of a RocketM, and AC Lite, an AF5X (actually on early firmware, so the ones people are getting now are actually faster) and Brand M.   This is a quick and dirty baseline test - the tests farther below were much more thorough and better set up, but all the radios were treated equally for this first one.

BaselineShootout.png

What became immediately obvious was that except with an 80MHz channel the Brand M radio, comparing apples-to-apples (with just one of it's 2x2 radio channels operating) is significantly slower than the AF5X, and still slower than the RocketAC Lite.   

So at this point I abandoned the idea of head-to-head testing (at least for throughput) with the AF5X, which really outclasses the Brand M, and instead stick with the ACs.

 

One thing that should be finally put to bed here - if anyone out there still thinks there's some magic going on inside the Brand M radio for it's "160MHz channel" other  than effectively two AC radios in one box, here's the proof - an AirView of the radios actually running:  

B5cDualRadio.png

If you look at the areas directly adjacent to the two 40MHz channels, you can see how much IM junk is generated in the PA - this is on the bench, so there's no interference from outside radios.

 

Anyway, after this I just concentrated on comparing it to an AC Lite.   Following is a series of automated tests comparing the AC Lite and the Brand M radios.   These are all using iperf in tcp mode, with 10 streams each, and with a complete tests running in each direction at once, so this is testing actual throughput both directions simultaneously for each radio.   Note that the PHY rate (dotted line) is the actual theoretical hardware encoding rate possible at that MCS and channel width, and the actual throughput of the radios is always lower than that - often much lower.

Throughput_bidir_R5AC-LITE,MIMO-B5C,['1x20']1429223748.pngThroughput_bidir_R5AC-LITE,MIMO-B5C,['1x40']1429223942.png

Throughput_bidir_R5AC-LITE,MIMO-B5C,['1x80']1429223960.png

You can see that in 2x2 mode at 80MHz the Brand M is faster than the AC Lite - at that channel width, the AC Lite is at the edge of it's chipset's capabilities, plus it was designed to do more narrow channels.   And who has clear 80MHz chunks of spectrum anyway?

 

Since the Brand M radio has the capability (as seen in the AirView above) to run as two 2x2 MIMO radios on two different channels, and combine the data streams together, the analogous setup for an AC radio is to use two discrete radios, each on it's own channel.   So this was done also (the setup actually used an EdgeRouter in load balancing mode to create the two streams for the two radios, and then another one to recombine them on the other end):

Throughput_bidir_R5AC-LITE,MIMO-B5C,['2x20']1429223709.png

You will note in the above test that the Brand M radio shifted speed part way through the test.   I saw this kind of behavior in other areas too - changing power output to see what effect that had on throughput, I found I could never quite get back to the performance level I initially saw.   And some times the performance would drop off after the radio heated up - this may be indicative of some thermal management issues in their power amplifiers - the actual quality of the PA , or maybe they just need a bigger heatsink...

Throughput_bidir_R5AC-LITE,MIMO-B5C,['2x40']1429223662.pngThroughput_bidir_R5AC-LITE,MIMO-B5C,['2x80']1429223987.png

Note that in the dual 80MHz case the AC Lites are faster than Brand M, unlike in the single 2x2 test before.  My guess is the PA is reaching the limits of it's IM cababilities trying to linearly amplify the two separate frequencies, and therefore the modulation rates go down.

 

Since the test was already set up, here are the AC Lite's results for all the other channel widths it can do:

Throughput_bidir_R5AC-LITE_1429224656.png

And here it is with dual radios:

Throughput_bidir_R5AC-LITE_1429224655.png

Now it's legitimate to ask if using two complete AC radios and an EdgeRouter Lite on each end is a fair comparison in the real world.   Well, economically, a pair of Brand M radios costs $1678, vs $740 for 4 AC Lites plus two EdgeRouters, a less than half the price of the Brand M radios alone.   And yes, to make this work you'd need 2 more antennas for the ACs, but this is a pretty corner case solution anyway - where but in the middle of nowhere or over a very short link are you going to be able to do this (especially at 80MHz channels) and in those cases you can probably do better with AF5s or AF24s anyway.

 

So what's the upshot of all this?   Well, things don't look so good for the Brand M radios so far in this shootout...

 

Part two will look at some other radios (plus these two again) and how they fare in interference situations. 

Jim

" How can anyone trust Scientists? If new evidence comes along, they change their minds! " Politician's joke (sort of...)
"Humans are allergic to change..They love to say, ‘We’ve always done it this way.’ I try to fight that. "Admiral Grace Hopper, USN, Computer Scientist
":It's not Rocket Science! - Oh wait, Actually it is... "NASA bumper sticker
":The biggest problem in tech I see right now is that most users don't want to do things that are hard. That doesn't bode well for the industry or the society.": (me. actually ;-)
Comments
by
on ‎04-17-2015 04:01 PM

Thanks for the detailed write-up.

by
on ‎04-17-2015 04:09 PM

Nice write up.  Thanks for the great info!

by
on ‎04-17-2015 04:11 PM

I used to take pride in being a hack, now I want to grow up to be @eejimm 

by
‎04-17-2015 04:15 PM - edited ‎04-17-2015 04:17 PM

  >>   I used to take pride in being a hack, now I want to grow up to be @eejimm   

 

Thanks, today I needed that...

Jim

by
on ‎04-17-2015 04:19 PM

Great...

by
on ‎04-17-2015 04:22 PM

As always an excellent review !!! Superb Jim !!

Where is the hats off to you emoticon ?

by
on ‎04-17-2015 04:24 PM

Jim,

 

I am looking forward to the hard data on how the Rocket AC handels interference.  I think it will open the eyes of people thinking they are going to use 80 wide channels.

 

Wayne

by
on ‎04-17-2015 04:26 PM

@wtm Smiley Very Happy

 

Great write up Jim.  You're the man!

by Deleted Account
on ‎04-17-2015 04:27 PM
+1, as usual Jim, great and detailed writeup!
by
on ‎04-17-2015 05:23 PM

Exactly what I wanted to see! Thanks Jim

by
on ‎04-17-2015 05:31 PM

Can someone explain what 'M' mean when they say 4x4:4 mimo?

by
on ‎04-17-2015 06:33 PM

Thanks Jim!  I'm seeing similar results in the field and trying to get a handle on it.   Nice to see this extensive lab work to make sense of it all.  

by
‎04-17-2015 08:45 PM - edited ‎04-17-2015 08:50 PM

What their 4x4 MIMO is?   Briefly, MIMO is a way to create multiple streams of data in an RF signal, where the payload data is spread across more than one stream.   In 2x2 MIMO there are two streams or "chains" and each one is encoded in a separate RF carrier.   In most radio systems (like UBNT) the two radio channels are transmitted in the same direction by the antenna, but in orthagonal polarities (90 degrees apart) so they can be kept separate, since the two streams are using the same radio frequency.   If there are more than 2 streams, that just means there have to be additional ways to keep them separate.   In Ms radio, they actually use a 4x4 MIMO chipset, but separate it into 2 2x2 streams by putting them on 2 different frequencies.   You can theoretically create as many streams as you want - the practical limit is around 8 or so - but 3x3 is used in higher-end WiFi systems.

Jim

by
on ‎04-18-2015 01:16 AM

Thanks Jim, I guess I should have been a bit more specific - I'm familiar with 2x2, 3x3 etc, but they write it as 4x4:4 - Whats the last 4??

by
on ‎04-18-2015 01:24 AM

@eejimm Great writeup! Very good work, you're the man!

by
on ‎04-18-2015 02:45 AM

@eejimm Great and detailed writeup.

 

Alot of time spent on this... Kudos

 

/Paetur

by Ubiquiti Employee
on ‎04-18-2015 07:58 AM

Awesome writeup Jim! Thank you for sharing your results

by
on ‎04-18-2015 08:31 AM

can someone please tell me how to power my PTZ 24 volt camera and Ubiquiti NanoStation M5 NSM5 with a 12 o2 24 volt car battery

by
‎04-18-2015 08:36 AM - edited ‎04-18-2015 09:36 AM

by 2m ago

can someone please tell me how to power my PTZ 24 volt camera and Ubiquiti NanoStation M5 NSM5 with a 12 o2 24 volt car battery 


Post that question here:  

http://community.ubnt.com/t5/Installation-Troubleshooting/bd-p/airMAX_Troubleshooting

 

And, you will get some good help.  You intrupted a thread with your oftopic question.

by
‎04-18-2015 09:03 AM - edited ‎04-18-2015 09:03 AM
by 8 hours ago

Thanks Jim, I guess I should have been a bit more specific - I'm familiar with 2x2, 3x3 etc, but they write it as 4x4:4 - Whats the last 4??

 

OK, the nomenclature is kind of repetative, but it goes like this:

 

TxR : S   Where T is the number of transmit chains, R the number of Receive chains, and S the number of Spatial streams.   Not sure who came up with that, but there it is.

Jim

by
on ‎04-18-2015 09:40 AM

Great Write Up! But I need to see results at 34 miles.

by
on ‎04-18-2015 09:56 AM

Nice,

 

Regarding AirfiberX,  is seams to me that this product is more equivalent to Cambium ptp650 !?

I dont have experience with neither of them, but that test woud be nice

by
on ‎04-18-2015 11:13 AM

pcmcnary - don't have a link that long, but we are going to be testing their radios against AF5Xs on a 21 mile link later this week.

 

Makoto - Yeah, the ones based on the  old Orthogon radios - don't have one of those to test either...

Jim

 

by
on ‎04-18-2015 11:40 AM

What was the framing of M 2/4/8 ms?

You can't compare the frequency reuse case in lab.

M has working GPS Sync. So you can use bigger channels

as you might use them 3 times at one tower.

by
on ‎04-18-2015 12:11 PM
 
by 28m ago

What was the framing of M 2/4/8 ms?

You can't compare the frequency reuse case in lab.

M has working GPS Sync. So you can use bigger channels

as you might use them 3 times at one tower.

 

The test was for 4ms    And the AF5X has GPS sync, which can make channel reuse of as many radios as you want.   And we're doing that in the field right now.

Jim

by
‎04-19-2015 03:34 AM - edited ‎04-19-2015 08:27 AM

These tests show max throughput vs channel bandwidth and MIMO streams  at 256QAM modulation, that is measured  in lab.     Average result – it is about  70MBps MIMO 2x2 simplex in 10 MHz channel bandwidth for 802.11АС . Is is not new information. 

 We   see  new, that AF5X provides   max about 85 MBps simplex in 10 MHz channel.  Mimoza  AC radio AC ( it seems it  is latest  wave 2  AC chipset MU-MIMO) provides max 60-70Mbps MIMO 2 x2 ( 2 streams),  that is less that rocket AC.

But it is max throughput. What will  we get in real word?

2 jim , you have set up with interference sourse. Provide tests with co-channel interference.

We already have seen  real links  on  Powerbeam  АC  at 12 km distance LOS( 40 MHz BW, RSSI -57 dBm,  CINR= C/(I+N) =30 dB,  256QAM ). PB AC is not able to give more that 160-180Mbps TCP aggregate ( UL+DL)  at  duplex ( two way) traffic, without latency degradation above avg 20 mc.  So  max 70 Mbps  simplex   in each 10 MHz channel bandwidth ubnt AC in lab condition became 40-45 Mbps (UL+DL) in  each 10 MHz in real links.

I suppose that MU-MIMO chip  ( that  is probably used  in Mimoza)  has better BER at 256QAM and lower CINR  value, that  is required for 256QAM, than AC wave 1 ( rocket AC).  I think  that in spite  of  new AC chip gives slightly less throughput in lab, it may be able to keep high  speed in real links. Please test Mimoza  and Rocket  AC at 256QAM  at  CINR 30 dB ( when required CINR level for 256QAM  is 34-36 dB).

Also the similar test  will be interesting with AF5X. I suppose, that AF5 radio and protocol is not able to resist enough against   interference and multipath ib 5 GHz. I suppose that AF5X throughput in real word may l be in twice or more worse than in lab. You may check it just now,  please, do it.

by
on ‎04-20-2015 07:10 AM

by
on ‎04-20-2015 09:38 AM

Am I misreading the data which shows the RocketAC being faster than the AF5X in terms of throughput vs channel size?

by
on ‎04-20-2015 10:52 AM

@wispWizardThe AirFiber and Brand-M radios use a version of TDD where the airtime for upload and download are fixed in size. AirMAX uses TDMA which allows for dynamic airtime allocation.  This means that in one way speed tests the AirMAX radios can dedicate virtually all of the airtime to that one direction, but the other radios are restricted by their settings.  This is why eejim added the aggregate column on the end of the tables for the AirFiber and the Brand M radios.

by
on ‎04-20-2015 11:11 AM

Thanks for that info.

 

by
on ‎07-23-2015 01:24 PM

Jim,

 

Can you please point me to a config to set up a dual backhaul link with an ERL on each side?  I have a lab set up and have been fumbling through load balance config examples for dual WAN balancing but what I need to do is combing/balance 2 ptp RF links that are already and running.

 

Thanks,

 

JW

by
on ‎05-12-2016 06:05 AM

Hi guys,

 

I do not agree 100% with this test. tested in a real environment the aforementioned radio and mimosa B5c fared better in these conditions REAL. in without interference environment is one thing, now in real life is another.
use very AF5X in our network and AF5, and now a few more B5c due to the high that radio bandwidth capacity.
We spoke by burning problems with the rocket ac lite and other problems that we had with B5c of mimosa. who knows and properly confgura the Rados know what I'm saying. but it is a great test to have as a base, now will work or not for each provider is another story.

by
on ‎12-08-2016 12:48 AM

Tnx eejimm

 

 

 

 

 

 

M_Brand_Shoot.jpg

by
on ‎12-08-2016 04:15 AM
por
3 horas atrás

Tnx eejimm

 

 

 

 

 

 

by
on ‎12-12-2016 12:49 AM

Still facing some performance issue (high PER even in cleanest enviroment causing lower modulation)  and most importand high sensitivy of the ethernet interface (link state/flapping).

 

 

Did you notice it has over the air frame losses (drop frames) ?