Reply
Highlighted
New Member
Posts: 19
Registered: ‎05-11-2013
Kudos: 5
Solutions: 1

OpenVPN Performance/Throughput

I wanted to ask for any experience with the OpenVPN performance of the ERLight.

I have a EdgeMax with various things running: NAT, zone-based Firewall, PBR, VLAN, etc. and OpenVPN as client.

I got quite disappointing Throughput over the encrypted OpenVPN-interface. It is about 6Mbit down/8 Mbit up (On a 150/10 Mbit Connection). Any idea what the problem could be?

New Member
Posts: 15
Registered: ‎08-16-2013
Kudos: 254
Solutions: 1

Re: OpenVPN Performance/Throughput

I've got a similar setup (ERL as OpenVPN client with a 100/2 mbps connection) and I see similar throughput, around 10-15 mbps down (how I wish I could get that sort of upload speed on my connection...but I digress). Having had a look at some other threads it seems that's about what you can expect because OpenVPN isn't hw offloaded.  Ipsec is offloaded to the hardware though, and apparently people have acheived >200mbit site-to-site connections with ipsec.  

I have also seen one of the developers mention that it's a possibility they'll offload it to the hardware (which would potentially give us similar speeds that are seen with ipsec connections as I understand it) if they have the time/resources.  I don't know where to put my vote in for that, but if anyone of consequence reads this you can count my vote for that.

Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5464
Solutions: 1656
Contributions: 2

Re: OpenVPN Performance/Throughput

6 Mbps is pretty low, there might be some useful information in previous discussions on the forum, for example here. You can also monitor the CPU usage on the router while the throughput is maxed out, to see if the router CPU is the bottleneck.

Regarding offload, as mentioned before it is important to note that IPsec and OpenVPN are very different. The data path for IPsec is within the kernel, but for OpenVPN every data packet needs to go from kernel to userspace for processing and then send back to the kernel, which incurs much higher overhead (relative to IPsec). As mentioned, one test to see this is to configure the OpenVPN tunnel to disable encryption and HMAC (for example overriding the configuration with "--cipher none --auth none"), and then do the performance test. This should pretty much eliminate the crypto overhead, so the result should be pretty close to the upper bound even if crypto offload is available.

New Member
Posts: 19
Registered: ‎05-11-2013
Kudos: 5
Solutions: 1

Re: OpenVPN Performance/Throughput

[ Edited ]

I'm running the following configuration:

mode client
 openvpn-option "--resolv-retry infinite"
 openvpn-option "--ns-cert-type server"
 openvpn-option --nobind
 openvpn-option "--cipher AES-256-CBC"
 openvpn-option --comp-lzo
 openvpn-option "--verb 3"
 openvpn-option "--explicit-exit-notify 5"
 openvpn-option --route-nopull
 protocol udp
 remote-host xxx.org
 remote-port 443
 tls {
     ca-cert-file /config/user-data/ca.crt
     cert-file /config/user-data/airvpn.crt
     key-file /config/user-data/airvpn.key
 }

 It seems like the CPU isn't the bottleneck: With 6 MBit throughput i got 25%. Could it be that AES-256-CBC is too complicated to encrypt or would i see this in CPU-usage?

I know the thing about the hw acceleration, i never expected 200 MBit, but 6... i don't know Man Wink

Do you think its clever to switch to IPSec?

Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5464
Solutions: 1656
Contributions: 2

Re: OpenVPN Performance/Throughput

You are right that 6 Mbps is certainly low. You could check the previous discussions on the forum, there might be some useful information. If possible you could also try setting cipher/auth to none as mentioned. Also, how are you testing the performance, e.g., how are the endpoints connected to the router etc.?

New Member
Posts: 15
Registered: ‎08-16-2013
Kudos: 254
Solutions: 1

Re: OpenVPN Performance/Throughput

You'll definitely get better throughput with ipsec.  As far as I understand though the ERL only supports site-to-site connections with ipsec, and in my case I need a client connection (basically I need to route around my isp's international bottleneck from Australia to the rest of the world...it's a long story that makes me mad) to connect to a commercial VPN provider so ipsec on the ERL won't work.  Even at ~10mbps it's faster than my normal international links.


Similarly I never see CPU usage above 60% so that's not the bottleneck, so I might try to optimise my openvpn connection when I've got some time too.

New Member
Posts: 19
Registered: ‎05-11-2013
Kudos: 5
Solutions: 1

Re: OpenVPN Performance/Throughput

Since i'm only the client i can't change a lot, only udp/tcp (which is nearly the same in throughput).

Setup is like this:

PC <-> Gigabit Switch <-> VLAN <-> Router <-> WAN <-> VPN-Server

 But i did doublecheck: Same VPN-Connections runs faster if encrypted directly on my PC.

New Member
Posts: 19
Registered: ‎05-11-2013
Kudos: 5
Solutions: 1

Re: OpenVPN Performance/Throughput

I changed to another VPN-Provider which uses another authorisation and as cipher 128-BlowFish-CBC. Now the throughput is around 12 MBit. This should work for now, maybe i'm going to set up a stand-alone-box which is used as VPN-Client and route the traffic through it.

Veteran Member
Posts: 5,417
Registered: ‎03-12-2011
Kudos: 2707
Solutions: 128

Re: OpenVPN Performance/Throughput


petervogt wrote:

I changed to another VPN-Provider which uses another authorisation and as cipher 128-BlowFish-CBC. Now the throughput is around 12 MBit. This should work for now, maybe i'm going to set up a stand-alone-box which is used as VPN-Client and route the traffic through it.


I'm not sure what's up with your particular setup but I have had an openvpn connection across a ~50/50mbit wireless link (ERL's on each end) with negligible difference in speed through the openpvn tunnel vs without it.

Do you have another endpoint to test against (Like another ERL connected via gigabit ethernet)? I saw another thread in here with poor openvpn performance when using a vpn provider. Might be a latency thing adversely affecting performance perhaps? (My wireless link naturally is fairly low latency compared to your average trip out to the greater internets).

New Member
Posts: 15
Registered: ‎08-16-2013
Kudos: 254
Solutions: 1

Re: OpenVPN Performance/Throughput

Yeah, I'm definitely at the ~12mbps rate along with petervogt.  Maybe it has something to do with using it as a client rather than in a site-to-site configuration.  Anything special you remember about setting up your link?  

Unfortunately I don't have another endpoint to test it against...maybe petervogt will.  I'll fiddle with settings later this week, but I suspect it might require hw offload to get it moving much quicker than that (I'm happy to be wrong about that btw).

New Member
Posts: 19
Registered: ‎05-11-2013
Kudos: 5
Solutions: 1

Re: OpenVPN Performance/Throughput

No, i don't have another endpoint. Since i'm interested in a real-world-setting with a tunnel into the "greater internet", this is the speed which i'm interested in. But anyway: Which cipher did you use in your site-to-site-setting? Do you get the same speed if using a client-server-configuration?

Actualy the latency isn't that bad, there are around 30ms between the router and the VPN-Server. This shouldn't be a problem.

Veteran Member
Posts: 5,417
Registered: ‎03-12-2011
Kudos: 2707
Solutions: 128

Re: OpenVPN Performance/Throughput


petervogt wrote:

No, i don't have another endpoint. Since i'm interested in a real-world-setting with a tunnel into the "greater internet", this is the speed which i'm interested in. But anyway: Which cipher did you use in your site-to-site-setting? Do you get the same speed if using a client-server-configuration?

Actualy the latency isn't that bad, there are around 30ms between the router and the VPN-Server. This shouldn't be a problem.


I have only tried site-to-site and I have changed my configuration so that it doesn't use the openvpn tunnel any more (I had originally needed the tunnel as I ran it over the internet, when I got a dedicated ptp link up for it I just ran the tunnel over that instead until I got around to re-jigging the routing to route over the new ptp link directly without a tunnel).

30ms doesn't sound like much, but consider it's still an order of magnitude larger than the latency I had. Might see if I can scrounge up some hardware and run some tests.

New Member
Posts: 13
Registered: ‎09-16-2013
Kudos: 4

Re: OpenVPN Performance/Throughput


petervogt wrote:

I changed to another VPN-Provider which uses another authorisation and as cipher 128-BlowFish-CBC. Now the throughput is around 12 MBit. This should work for now, maybe i'm going to set up a stand-alone-box which is used as VPN-Client and route the traffic through it.


For reference, that's inline with what I get. I run a client-server tunnel with ERLs on both ends and get around the same with 128-bit blowfish. AES is a bit slower. The latency over the link is around 5ms.

The bottleneck appears to be the CPU (presumably for the encyption), which hits ~90% during transfer. I guess blowfish is a little less CPU intensive.

Regarding CPU monitoring on the ERL, keep in mind that there are two cores. The Web GUI seem to show the total CPU usage over both, but I presume the openvpn encryption runs on a single core only. So it may not appear that CPU is the bottleneck when in fact it is. If I check the CPU usage on the Web gui during transfer, I see ~45%, but if I check using the top command over SSH, I see 90+% on one core.

Reply