Reply
SuperUser
Posts: 4,683
Registered: ‎05-02-2008
Kudos: 3977
Solutions: 35

The new and improved low throughput thread

@Josh_SPITwSPOTSsuggested that we needed a new thread so here we go ...

has anybody at ubnt been able to reproduce the the double upload bug ? Buehler, anyone, @UBNT-Matt , Buehler ???


rockhead wrote:

Fiber->Cisco2821->Ligo(backhaul)->TS8

This Cisco replaced a 'tik five days ago, with my computer plugged into the TS8 I pull symmetric 160/160 which is what the Ligo's will do because I detuned them looking for an RF interference issue that wasn't there.  When connected at a cpe that is fed from a Rocket5M I get 17/35 and have since the system went live nearly a year ago.  The TS8 also feeds another Ligo at GigE.

The other end of the second Ligo-hop is another TS8 it has been forever since I've been to that site so can't report on what exactly comes outa that switch for up/down.

Two hops (West and South {Rocket-Beam, PB-Beam}) later we hit another TS8 and sitting at virtually any cpe downstream from here as well as plugged into the switch there is highly repeatable reverse-double traffic flow.


 My observations above indicate that it is a backhaul and router agnostic issue. So please ubnt, duplicate the issue, then you can suggest an appropriate setup to cause the ToughSwitch to perform better.

If it was easy it would already be done.
Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

I'm in the process of setting up a testbed in the backyard.

1000 port CPU -> ERL3 router -> Toughswitch/dumbswitch -> Nano M5 Loco -> Nano M5 Loco -> 100 port CPU

Going to run Iperf tests through it, and start changing configurations, to hopefully see what gives the best download speeds through the entire system..........

(May have to push that off a day, huge Monsoon dust storm hit Phoenix last night, and I have to fix roofing today)

Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

Thanks Rockhead!    (By the way, what Ligo equipment do you have ?)

I am planning on setting up a testbed to do some experimentation.

1000 port CPU -> ERL3 router -> Toughswitch/Dumb Switch -> Nano M5 Loco -> Nano M5 Loco -> 100 port CPU

And then run Iperf tests through it, then start changing things to find the best Download speeds possible?

(Might have to put this off a day, as Phoenix got hit hard with a Monsoon dust storm last night, and I have to replace roofing shingles today-bummer)

I was wondering.......might the radios have to use "Half-duplex" run ports rather than "Full duplex" ports ? The radios are only half duplex anyway?

Also, I have noticed that even when I get a speed test that shows say 10 down/20 up, that I can still get traffic through the link with higher download than what the speed test shows ???  I think that the Upload speed is more accurate to what the link will process, and the Download speeds are being limited by "something"????

SuperUser
Posts: 4,683
Registered: ‎05-02-2008
Kudos: 3977
Solutions: 35

Re: The new and improved low throughput thread

My Ligo's are the UnityPTP-Pro.

Oh, wrong day to start a thread and crack the whip eh ?  Happy 4th of July Cheers2 !  Once you're done rubbing fireworks residue out of the corners of your eyes I expect results Icon Razz

If it was easy it would already be done.
Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

[ Edited ]

Just awoke from my nap AFTER fixing the shingles on the roof that blew off!  Still up in the 100's here, but now the humidity is around 50+, meaning it is miserable! More storms on the horizon.......

I am starting to get the radios ready to hook up and then program them. See how it goes. Wife up visiting her Mom, so I have all weekend to work on it.

 

Hey Rockhead, we need a @UBNT-Buehler  !!!

SuperUser
Posts: 4,683
Registered: ‎05-02-2008
Kudos: 3977
Solutions: 35

Re: The new and improved low throughput thread

Buehler Buehler @UBNT-Matt Buehler

If it was easy it would already be done.
Ubiquiti Employee
Posts: 7,391
Registered: ‎11-27-2007
Kudos: 4204
Solutions: 167
Contributions: 45

Re: The new and improved low throughput thread

Can you provide some more details on what "reverse-double traffic flow" is? 

Can you post some screenshots or examples of what you're seeing? 

Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

[ Edited ]

OK, attached is an Excel spreadsheet on speed tests.

Setup is as follows:  ERL-3 ports both set to Auto (1000)

CPU (1000 Nic port) -> Ubnt ERL3 -> Toughswitch 5 -> Nano Loco M5 -> Nano Loco M5 -> CPU (100 Nic port)

Radios were setup for a 40 mhz wide channel, about 50 feet apart, and the radios were turned 180 degrees to give about -70 to 65 on the signal strength.   Radio Lan ports were left as default 100/full/Auto

Iperf 3 was installed on the computers.  Download speeds are the ONLY ones tested.....Downloading from ERL3 to radios.

Rough radio speed test showed @ 20 Mbps download to @ 40 Mbps Upload (typical of 2/1 we are seeing on the Up/Down stuff)      "Reverse Double traffic flow"

Figured I would start on the Toughswitch first, as that seems to be where the existing problem is!

As expected, it went about as I figured till I got to the "Half Duplex" setup. All I can deduct is that the radios are really half duplex anyway, so that is what they need?

I will be further testing the connections on both the ERL3 and the Radios to see what happens further, but this is already a big help!

Wayne

 

 

 

 

 

 

Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

[ Edited ]

@UBNT-Matt 

I have found a programming glitch on the TS-5 firmware 1.3 when connected to the ERL-3 v1.5.0

For Port 1, if you manually set the speed to 100 Full, it shows 100 full on the setup page, but in the status page, it connects as 100 Half. If you setup it to 100 half, it shows 100 half, but in the status page, it connects as 100 Full !!

This is ONLY when the ERL3 LAN port (feeding the TS-5) is taken out of Auto and put in 100 Full or Half

Port 2 seems to work as it should.

This may effect my test !!!!!!!

(Later Edit....You have to unplug and re-plug the Lan port connection to make sure it sets up the correct configuration. If you just change it via the software, it will not neccessarily change to what you set it too.)

Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

[ Edited ]

Additional testing...........

Setup the TS-5 for Auto on both the In and Out ports, Flow Control OFF on both.

Then changed the ERL-3 on the LAN port for 100 Full or Half. (Full or Half did not seem to matter)

Iperf test showed 69.3 Mbps for the Download speeds on the test setup.  Best Yet!


I am going to pick up a "Dumb switch", and try it on that...........

Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

[ Edited ]

Further testing.............On radio link equipment LAN ports

Setup:    ERL3 =Auto (1000),  TS5 In = Auto (1000), TS5 Out= Auto (100)

AP radio LAN was set to 100/Half    Speed throughput = @24 Mbps (Flow Control=Off on TS5 In Port)

                                                                              = @ 25 Mbps (Flow Control=On on TS5 In Port)

AP radio LAN set to Auto, STN radio LAN set to 100/Half       = @ 2.26 Mbps

Both AP and STN radio's LAN set to 100/Half               = @ 2.35 Mbps

Established Member
Posts: 1,195
Registered: ‎11-05-2008
Kudos: 504
Solutions: 46

Re: The new and improved low throughput thread

[ Edited ]

I still think its related to latency issues.
I have ER-PRO-> TS1 -> TS2

Latency ping -f test to TS1 management IP

ping -f 10.0.0.1
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.
.^C
--- 10.0.0.1 ping statistics ---
259915 packets transmitted, 259914 received, 0% packet loss, time 116121ms
rtt min/avg/max/mdev = 0.192/0.524/312.099/4.799 ms, pipe 27, ipg/ewma 0.446/0.460 ms

1 packet out of 259915 missed.

Latency ping -f test to TS2 management IP

ping -f  10.0.0.2
PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data.
...........................................................................................................................................................................................................................................................................^C
--- 10.0.0.2 ping statistics ---
371149 packets transmitted, 370931 received, 0% packet loss, time 211156ms

218 packets lost out of 370,931

Conditions: SW1 is using between 100-200Mbps

Note: the packet loss gets worse with flow control on and better if you change the uplink ports to 100-full.  I have not tested further.

Note2: Sw1 and sw2 have a mixture of 100 and 1000 links (mostly rockets/nanobridges/1 gbit nanobeam)

This would probably result in download/upload speeds.

##ubnt - chat.freenode.net
Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

OK, put in a "Dumb Switch" (Netgear GS-108) in place of the TS-5 switch.

Set up the following:

ERL3 LAN   = Auto (1000)        ->   Speed test results= 14.7 Mbps

                    = 100 (Full)                                               = 10.2 Mbps

                    = 100 (Half)                                              = 69.5 Mbps      (Ding, Ding, Ding, Ding- we have a WINNER)

Looks like the TS unit is not the real problem, except maybe the flow control. The real problem is if you have a connection over 100 Mbps on your Upload side, that the Router HAS to set it down to 100 Mbps connection for the radio units.

I will next be replacing the ERL3 with a Mikrotik RB-1100AH which we have handy, and see if it stays the same results?

SuperUser
Posts: 4,683
Registered: ‎05-02-2008
Kudos: 3977
Solutions: 35

Re: The new and improved low throughput thread

Interesting ... I have no ER lite or otherwise.

If it was easy it would already be done.
Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

[ Edited ]

Here is the info on using a Mikrotik router (RB1100-AH)

As long as the Mikrotik Input port can negotiate 100 Mbps connection, (need to de-advertise 1000 Mbps), and the Output port is 100 Mbps (De-advertise 1000 Mbps) Leave both in "Auto" negotiate. Then it will pass the best possible throughput, which is this case was 69 Mbps.

IF, the Mikrotik tries to connect to a 1000 Mbps (1 Gig) port on the Input, it will only pass @ 25-30 Mbps down to the Output port.  You can even use a "Dumb Switch" between the 1000 Mbps device and the Mikrotik Input port, and then it will work correctly and pass the higher throughput (69 Mbps in this case).

Mikrotik does not seem to want to interface with the 1000 Mbps ports well !  ONLY 100 Mbps connections ............

BIG Eye opener here, as we will be doing some re-programming, and changing of some equipment on the system.

If you are using Mikrotik routers, and AirFiber, Rocket TI, NanoBeam 400 backhaul, then you have to make sure you have a "Dumb Switch" between those 1 Gig devices, and the input to the Mikrotik unit.

As far as the Edge Router (at least the ERL3 unit), it will connect up to a 1000 Mbps input port, and then set the output port to 100 Mbps (Half Duplex), which then seems to give the best possible throughput (69 Mbps in the test setup) for 100 Mbps connections, such as Rockets, NanoStations, Nano Locos, and Nanobridges.

 

HOPE this helps?

Wayne

SuperUser
Posts: 4,683
Registered: ‎05-02-2008
Kudos: 3977
Solutions: 35

Re: The new and improved low throughput thread


UBNT-Matt wrote:

Can you provide some more details on what "reverse-double traffic flow" is? 

Can you post some screenshots or examples of what you're seeing? 


 

Upload being twice the download is reverse double flow

If it was easy it would already be done.
Established Member
Posts: 1,563
Registered: ‎11-03-2009
Kudos: 416
Solutions: 6

Re: The new and improved low throughput thread

[ Edited ]

So uhhh basically the EdgeRouter has the same problem as the ToughSwitch with Gbps uplinks? Seriously, who is testing these things!?

 

EDIT:  Well I guess it is on the downstream side... but still...

SuperUser
Posts: 4,683
Registered: ‎05-02-2008
Kudos: 3977
Solutions: 35

Re: The new and improved low throughput thread

[ Edited ]

All switches between me and the fiber now on 1.3 ( I wanted a quiet moment on the network to update ) no change.

I reaaaaaaaaalllllly would like to see anybody at ubnt acknowledge that they can duplicate this issue. Buehler ? Buehler ? @UBNT-Matt 

If it was easy it would already be done.
Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

After my tests yesterday, I changed the configuration on the main ERL3 coming into my system from the Fiber. It immediately showed an additional 10 Mbps of download coming into the system.  Only thing I needed to do was change the Output LAN port (going to the link radios) to be 100 Mbps (Half Duplex).  Any flow control was turned Off.

The ERL does not seem to be as bad as the Mikrotik on connecting to a 1 gig fiber, as the 'Tik will NOT interface and pass good throughput when it sees a 1 Gig input connection. Have to have something in front of it to bring it down to 100 Mbps connection first.

Of course, you can ONLY see how this happens, when you setup an entire testbed, including a radio link, to test COMPLETELY from the router ALL the way down to the end user.

Unfortunately, I do not have any of the 1 gig UBNT devices, such as Air Fiber to do a test with. I would assume that those will change things a bit on the entire setup. Not looking forward to figuring that out when we upgrade our backhaul to 1 Gig in the future.

Veteran Member
Posts: 7,486
Registered: ‎04-21-2011
Kudos: 2599
Solutions: 167

Re: The new and improved low throughput thread

Jake_craner:
So uhhh basically the EdgeRouter has the same problem as the ToughSwitch with Gbps uplinks? Seriously, who is testing these things!? EDIT: Well I guess it is on the downstream side... but still...

 I don't think it is the ERL or TS by itself. It is the "interfacing" with the rest of the "System".

The Rockets, Nano's, etc, are all 100 Mbps LAN ports, and the radios themselves are ONLY half-duplex throughput.

SO, have to get the 1 Gig connection down to 100 Mbps, and then format it for Half-Duplex throughput.

Basically, if you have a higher throughput Main connection than what the Radio link will allow, then you are wasting your money. We have 200 Mbps available on a 1 Gig connection (Special year end deal with fiber provider), but we can ONLY process under 100 Mbps through to the radios at this point.  We will be doing upgrading to the newer 1 Gig port radios in the future, and HOPEFULLY, at that point we can use all we are paying for?

BUT, that then shoves this problem down to the lower end 100 Mbps port radios, and the Routers connected to the AP's.

We will have to do these tests all over again at that time to figure out how to maintain the best throughput on the ENTIRE system.


Really need a small CPU device that can just run an IPERF3 server unit, to temporarily stick someplace in the network, and then test to it. Having 2 LAN ports, 1 - 1 Gig port, and 1 - 100 Mbps port.

Reply