Reply
New Member
Posts: 12
Registered: ‎08-05-2018
Kudos: 1

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!


@RADHAD wrote:

@UBNT-afomins 

 

> Has the speed decrease people have mentioned being addressed in the upcoming 2.0.1? Seems many people have noticed throughout decrease, and haven’t seen you acknowledge the issue.
Yeah, there have been some reports when throughput degrades after about 1 month of uptime if offloading is enabled on Cavium-based routers. Unfortunately I was not able to reproduce it - I setup a router in the lab and was running 1G traffic for a week but throughput was stable. Now I will try a different scenario with more diverse IP flows that represent real life traffic pattern.

 

thats weird  i can reprudce this directly on my edgemax lite3. on 1.10.8  get a round 980 mbits up / down and 2.0.0 drops to 670+- mbit.

 

I have connected edgemax lite direct to my FTU (Genexis (FXP-100) i have a simpel routersetup running default config from the edgeos wizard.

 

And IPV4 offloading enabled

 

 


Yup, It's just like a light switch for me too, simple config right out of the wizard with only some static ip's setup for some devices on the network, full speed with 1.10.8 then it from a couple hundred mbits less on the download once you update to 2.0.0. It's only a Comcast gigabyte connection so the upload is uneffected.

Highlighted
New Member
Posts: 7
Registered: ‎08-06-2017
Kudos: 9

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

@UBNT-afomins 

> will 2.0.1 fix hwnat & ipsec offload for the ERX or is that in a future release?
Unfortunately no, IPSec offload for ER-X will not be added in v2.0.1

 

Could you please tell us a bit more why the initial assumptions have been changed?

 

The first post of this thread states as below:

 

See release notes here -> https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-EdgeRouter-software-release-v2-0-0/ba-p/2...

 

 

NOTE: There are some known hwnat related issues on ER-X/ER-X-SFP that will be fixed in upcoming v2.0.1 firmware. 

  • [LoadBalancing] - LoadBalancing randomly fails if hwnat offloading is enabled on ER-X and ER-X-SFP models. LoadBalancing watchdog randomly reports false-positive interface-failure events and switches to backup link when it should not. Workaround is to disable hwnat offloading.
  • [PPPoE] - PPPoE client interface randomly fails to reconnect with PPPoE server when hwnat offloading is enabled on ER-X and ER-X-SFP router models. This issue was noticed only with LoadBalancing or ECMP setups. Workaround is to disable hwnat offloading.
  • [Offloading] - IPSec offloading does not work on ER-X and ER-X-SFP

 

Your feedback is highly appreciated.

 

I'm worrying, that now you are dropping the IPSec offloading from the roadmap for 2.0.1, and tomorrow we will see that you will drop the whole hwnat offload (or maybe this is what you mean already?) - https://help.ubnt.com/hc/en-us/articles/115006567467-EdgeRouter-Hardware-Offloading.

 

I can see that IPSec offload related issues has starter more thatn 1.5 year ago:

https://community.ubnt.com/t5/EdgeRouter/1-9-7-VPN-not-working-both-IPSec-and-l2tp/td-p/2008597

https://community.ubnt.com/t5/EdgeRouter/ER-X-l2tp-ipsec-w-HW-offload/td-p/2041722

 

I bought recently 5 units of ER-X, configured s2s vpn, l2tp/ipsec and next wanted to enable hwoffload and then I saw this:

 

 

WARNING : IPsec offload on ER-X platform is causing problems to L2TP remote-access VPN.
          and IPV6 site-to-site IPSec VPN
          You should *not* enable IPsec offload if you are using any of above.

          Other VPN modes are not affected by this issue:
             * IPv4 site-to-site IPsec VPN is working correctly with IPsec offload.
             * PPTP VPN is working correctly with IPsec offload.

          Only ER-X/ER-X-SFP/EP-R6 models are affected by this issue.

          This issue is to be fixed in future release.

 

I bouht the ER-X because their specification offers hwoffload. When such important feature is unavailable for almost 2 years this feature should be removed from device specification as it misleads people as it is simply a lie. I think that there are many, many owners of ER-X here, waiting for this specific feature to be fixed. Please, treat us seriously, we are your customers, very ofter loyal, commited and actively involved in development. I think that we deserve more detailed information than just yes or no.

 

New Member
Posts: 14
Registered: ‎07-13-2017

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

[ Edited ]

@bartaspoz 

 

Both hwnat & ipsec offload work for ipsec s2s on 1.18.0 on ERX :-) what does not work is is the remote-access vpn offloading; which is a client-access VPN solution.

Even then; ipsec s2s offload will only work on certain ciphers. default aes ciphers work fine for example. Its all in the knowledgebase if you need to refer to some documentation!

Ubiquiti Employee
Posts: 1,231
Registered: ‎07-20-2015
Kudos: 1453
Solutions: 81

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

@Blunix94
> After update from v1.10.5 to these release the edgerouter pro it stays here
> A start job is running for UBNT Routing Daemon (15min 23s / no limit)
> It frozen an hour and twenty minutes at that point.
Could you please run following oneliner in shell and then report back whether it improved boot time:

sudo sed -i -e 's/iptables -L/iptables -nL/g' /opt/vyatta/sbin/firewall.init
Emerging Member
Posts: 43
Registered: ‎11-19-2018
Kudos: 1

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

@bartaspoz I use no ipsec or vpn services and offloading causes a broadcast storm in my network. I switched to ubnt because of the hardware off loading, updated to 2.0.x per support and bricked my first router, this second one I've kept on 1.10.8 but kept offloading off and now am only capable of using a half of my gig connection. Makes me curious why the air has changed since first post, I would have held off buying a er-x and got 12 or a different brandon.

 

@UBNT-afomins any and all information on this would be great. *hwnat since many of us have been waiting for the .1 release due to what was released.

New Member
Posts: 14
Registered: ‎07-13-2017

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

@bpendleton28 out of curiosity; was this a clean config when running the 2.0 build? I installed 2.0 then hardreset the ERX after making sure it was running the latest bootloader. Set up the ERX and was running fine. This is a simple 1LAN setup running a s2s to a usg4 offsite. Manually upgraded to the 2.0.1 nightly posted earlier and again no real issues or any of the problems some of the guys have been experiencing.

I know downgrading back down to 1.10.8 caused a weird hiccup with the S2S dropping off every couple of hours; I hardreset again and set it up from scratch on 1.10.8 - flicked hwnat & ipsec offloading back on and thats where I am now.

If it wasnt for the lack of offloading on the e50 in 2.0.x then I would have stayed there. No random reboots or crashes of the sort - but then again I have a very basic setup. No VLANs for example; just PoE passthrough for the AP.
Ubiquiti Employee
Posts: 1,231
Registered: ‎07-20-2015
Kudos: 1453
Solutions: 81

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

@gritech
> The Cavium offload bug takes almost exactly 30 days to manifest for me
Yeah, I saw multiple reports saying that something wrong happens with offloading on Cavium-based routers after exactly one month of uptime. I'm now testing offload engine by skewing system time back and forth in order ot simulate large uptime but so far I was not able to reproduce throughput degradation.

 

@RADHAD
> Has the speed decrease people have mentioned being addressed in the upcoming 2.0.1?
> I can reprudce this directly on my edgemax lite3. on 1.10.8 get a round 980 mbits up / down and 2.0.0 drops to 670+- mbit.
Does throughput drop from 980 to 670 happens instantly? Or it slowly degrades during some time?

 

@bartaspoz
> There are some known hwnat related issues on ER-X/ER-X-SFP that will be fixed in upcoming v2.0.1 firmware.
> Unfortunately no, IPSec offload for ER-X will not be added in v2.0.1
IPSec offloading for hwnat was not provided by the vendor for v4.x kernel that's why we need to port it ourselves from older kernel. This is time consuming task and we wont be able to finish it by v2.0.1 release.

 

 

New Member
Posts: 12
Registered: ‎08-05-2018
Kudos: 1

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

[ Edited ]

@UBNT-afomins wrote:

 

@RADHAD
> Has the speed decrease people have mentioned being addressed in the upcoming 2.0.1?
> I can reprudce this directly on my edgemax lite3. on 1.10.8 get a round 980 mbits up / down and 2.0.0 drops to 670+- mbit.
Does throughput drop from 980 to 670 happens instantly? Or it slowly degrades during some time?

 

 


I am one of the people that this happens to and I can say that if i go from 1.10.8 to 2.0.0 the decrease is instant from one firmware to the other. My config is bone stock from the wizard with the exception of some static dhcp reservations. Flash it back to 1.10.8 it goes back to normal 950 on 2.0.0 it drops to about 750 to 850 for my download. I have no issue with upload since Comcast has that capped at 40 down. Also to add I  am using an Edgerouter 12.

Member
Posts: 229
Registered: ‎04-22-2018
Kudos: 26
Solutions: 2

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

[ Edited ]

@bartaspoz wrote:

@UBNT-afomins 

> will 2.0.1 fix hwnat & ipsec offload for the ERX or is that in a future release?
Unfortunately no, IPSec offload for ER-X will not be added in v2.0.1

 

Could you please tell us a bit more why the initial assumptions have been changed?

 

The first post of this thread states as below:

 

See release notes here -> https://community.ubnt.com/t5/EdgeMAX-Updates-Blog/EdgeMAX-EdgeRouter-software-release-v2-0-0/ba-p/2...

 

 

NOTE: There are some known hwnat related issues on ER-X/ER-X-SFP that will be fixed in upcoming v2.0.1 firmware. 

  • [LoadBalancing] - LoadBalancing randomly fails if hwnat offloading is enabled on ER-X and ER-X-SFP models. LoadBalancing watchdog randomly reports false-positive interface-failure events and switches to backup link when it should not. Workaround is to disable hwnat offloading.
  • [PPPoE] - PPPoE client interface randomly fails to reconnect with PPPoE server when hwnat offloading is enabled on ER-X and ER-X-SFP router models. This issue was noticed only with LoadBalancing or ECMP setups. Workaround is to disable hwnat offloading.
  • [Offloading] - IPSec offloading does not work on ER-X and ER-X-SFP

 

Your feedback is highly appreciated.

 

I'm worrying, that now you are dropping the IPSec offloading from the roadmap for 2.0.1, and tomorrow we will see that you will drop the whole hwnat offload (or maybe this is what you mean already?) - https://help.ubnt.com/hc/en-us/articles/115006567467-EdgeRouter-Hardware-Offloading.

 

I can see that IPSec offload related issues has starter more thatn 1.5 year ago:

https://community.ubnt.com/t5/EdgeRouter/1-9-7-VPN-not-working-both-IPSec-and-l2tp/td-p/2008597

https://community.ubnt.com/t5/EdgeRouter/ER-X-l2tp-ipsec-w-HW-offload/td-p/2041722

 

I bought recently 5 units of ER-X, configured s2s vpn, l2tp/ipsec and next wanted to enable hwoffload and then I saw this:

 

 

WARNING : IPsec offload on ER-X platform is causing problems to L2TP remote-access VPN.
          and IPV6 site-to-site IPSec VPN
          You should *not* enable IPsec offload if you are using any of above.

          Other VPN modes are not affected by this issue:
             * IPv4 site-to-site IPsec VPN is working correctly with IPsec offload.
             * PPTP VPN is working correctly with IPsec offload.

          Only ER-X/ER-X-SFP/EP-R6 models are affected by this issue.

          This issue is to be fixed in future release.

 

I bouht the ER-X because their specification offers hwoffload. When such important feature is unavailable for almost 2 years this feature should be removed from device specification as it misleads people as it is simply a lie. I think that there are many, many owners of ER-X here, waiting for this specific feature to be fixed. Please, treat us seriously, we are your customers, very ofter loyal, commited and actively involved in development. I think that we deserve more detailed information than just yes or no.

 


That’s why I sold my ER-X. Not sure why the community can’t get a response on this. Seems like they skip the post and remain silent about it.  You can see @UBNT-afomins @posting since you asked the question.  Maybe sell the ER-X and get ER4? Still a shame there’s is nothing but silence 


AP AC LITE
UAP nanoHD (x2)
Edgerouter 4
Regular Member
Posts: 489
Registered: ‎10-16-2016
Kudos: 60
Solutions: 13

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

Than use a older firmware and your done ... to flame around isnt very helpful. Its hard work to keep such things working and costs hundreds of hours sometimes ... I know the complexity of such kernel related issues
New Member
Posts: 7
Registered: ‎10-28-2018
Kudos: 6

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

Well, there wouldn't be any flame around this if we can get some official statement.


I bought multiple ER-X a few months ago and I'm thinking about RMA because of broken IPSec offloading. Using older firmware really isn't proper advice.

Ubiquiti Employee
Posts: 1,231
Registered: ‎07-20-2015
Kudos: 1453
Solutions: 81

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

@gritech
> The Cavium offload bug takes almost exactly 30 days to manifest for me
After reviewing source code I finally was able to find root cause of this issue - this bug was introduced in v1.10.2. We shall provide fix in v2.0.1 and v1.10.9 firmwares.

 

@jiber
> Well, there wouldn't be any flame around this if we can get some official statement.
> Using older firmware really isn't proper advice.
Old firmware was not abandoned, we will continue to provide security patches and major bug fixed for v1.10.x firmware until all features will be ported to v2.0.x

New Member
Posts: 7
Registered: ‎10-28-2018
Kudos: 6

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

@UBNT-afomins 

 

thank you for response. But issue with ipsec-offload was introduced in 1.9.0 in 2016. It is not working since then.

 

see:  this comment.

Emerging Member
Posts: 53
Registered: ‎04-14-2017
Kudos: 28
Solutions: 1

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

there is more than 2 offload issue.

hwnat

and

ipsec

Emerging Member
Posts: 45
Registered: ‎02-11-2018
Kudos: 6
Solutions: 1

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

Now that v2.0.0 is out, will 1.10.x still see updates? From scanning this thread it sounds like v2 has ways to go before it's solid. I see no good reason to move to v2, unless critical/security updates are no longer provided for 1.10.x

Established Member
Posts: 1,486
Registered: ‎07-07-2014
Kudos: 326
Solutions: 101

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!


@xwired wrote:

Now that v2.0.0 is out, will 1.10.x still see updates? From scanning this thread it sounds like v2 has ways to go before it's solid. I see no good reason to move to v2, unless critical/security updates are no longer provided for 1.10.x


Three posts above yours (and numerous before that!) indicate yes, 1.10.x will continue to receive bug fixes and security updates.

New Member
Posts: 4
Registered: ‎04-02-2015
Kudos: 1

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

It seems Traffic Analysis is enabled after reboot even if it’s set to disabled? I did “Clear Data” and now it doesn’t seem to populate the graphs.

XQ7LOLE

Member
Posts: 184
Registered: ‎02-08-2017
Kudos: 27
Solutions: 3

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

So I heard that 2.x would bring LACP offload, any word on that?
Emerging Member
Posts: 98
Registered: ‎12-21-2018
Kudos: 53

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

https://help.ubnt.com/hc/en-us/articles/115006567467-EdgeRouter-Hardware-Offloading

As far as I could tell, it is IPv4 only, so not very useful now.

Member
Posts: 184
Registered: ‎02-08-2017
Kudos: 27
Solutions: 3

Re: EdgeMAX EdgeRouter software version v2.0.0 has been released!

I'm asking about LCAP, which is layer 2 not layer 3, so IPv4 or v6 shouldn't matter.
Reply