Reply
Highlighted
New Member
Posts: 33
Registered: ‎03-01-2014
Kudos: 10
Solutions: 1

Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

Would it be possible to get Kernel IPVS support added? This would mean that we could use the ER to handle load-balancing of incoming connections.

Kernel options:

CONFIG_IP_VS=m
CONFIG_IP_VS_IPV6=y
CONFIG_IP_VS_TAB_BITS=12 # might want to tweak to 13 or 14
CONFIG_IP_VS_PROTO_TCP=y
CONFIG_IP_VS_PROTO_UDP=y
CONFIG_IP_VS_PROTO_AH_ESP=y
CONFIG_IP_VS_PROTO_ESP=y
CONFIG_IP_VS_PROTO_AH=y
CONFIG_IP_VS_PROTO_SCTP=y # maybe, depends if we have SCTP support
CONFIG_IP_VS_RR=m
CONFIG_IP_VS_WRR=m
CONFIG_IP_VS_LC=m
CONFIG_IP_VS_WLC=m
CONFIG_IP_VS_LBLC=m
CONFIG_IP_VS_LBLCR=m
CONFIG_IP_VS_DH=m
CONFIG_IP_VS_SH=m
CONFIG_IP_VS_SED=m
CONFIG_IP_VS_NQ=m
CONFIG_IP_VS_SH_TAB_BITS=8
CONFIG_IP_VS_NFCT=y
CONFIG_IP_VS_PE_SIP=m # maybe

Debian packages

ipvsadm

I have NOT included keepalived in the packages to be included, as that probably best belongs in a separate module; just do non-dynamic IPVS for now

Previous Employee
Posts: 10,504
Registered: ‎06-09-2011
Kudos: 3126
Solutions: 945
Contributions: 16

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

keepalived is already part of the system for vrrp.

EdgeMAX Router Software Development
New Member
Posts: 33
Registered: ‎03-01-2014
Kudos: 10
Solutions: 1

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

Ok, so then this is just adding the kernel options and the ipvsadm package, not invasive at all.

New Member
Posts: 40
Registered: ‎03-01-2013
Kudos: 17

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

This would be some really nice functionality to have Man Happy it'd open up the EdgeRouter platform to being used as a low cost load balancer (something that doesn't really exist on the market).

Member
Posts: 122
Registered: ‎02-09-2013
Kudos: 29
Solutions: 1

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

I don't think inbound LB has any application on a router. Also would this be for IP, application? Sessions? What about SSL Offload - that'll be requested since it's now a LB.

It's easy enough setting up a  small VM with HAProxy, or if hardware required Foundry Server Iron's are sub $100 these days second hand, they are enterprise grade... HAProxy can do SSL offload, so can some of the ServerIron's.

New Member
Posts: 33
Registered: ‎03-01-2014
Kudos: 10
Solutions: 1

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

The difference is that IPVS is layer 4 load balancing, not layer 7, and much simpler than haproxy, varnish et al.

IPVS has several modes for handling traffic to the real servers: direct routing (DR), NAT, tunnel; and runs ideally in the router for the network, esp if used in the NAT mode (which is built entirely on top of netfilter, right down to conntrack table sync between LB/routers).

If anybody wants SSL offload or content awareness (HTTP headers), then yes, they belong in a layer 7 load balancer.

If you don't want to support the rest of the application side (just the ipvsadm package), please do enable the kernel modules.

I could run a pair of redundant VM hosts and VMs in each with a load balancer, but if I already have redundant ER's, and I think this really does fit nicely into the existing ER model.

Veteran Member
Posts: 5,417
Registered: ‎03-12-2011
Kudos: 2711
Solutions: 128

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

Hmm, sounds like an interesting avenue for ubnt, especially considering the functionality can be done without any hardware changes. Man Very Happy

Previous Employee
Posts: 13,551
Registered: ‎06-10-2011
Kudos: 5465
Solutions: 1656
Contributions: 2

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

[ Edited ]

We could look into adding those kernel modules into the default config and perhaps the ipvsadm package too. For now though it should be pretty straightforward to build those modules externally and copy them to the router, in fact I've given it a quick try and attached a tarball (extract in "/lib/modules/3.4.27-UBNT/kernel/net/netfilter", "depmod -a", etc.). Also should be straightforward to install the Debian package, for example:

curl -O http://ftp.us.debian.org/debian/pool/main/i/ipvsadm/ipvsadm_1.25.clean-1_mips.deb                                                       
curl -O http://ftp.us.debian.org/debian/pool/main/libn/libnl/libnl1_1.1-6_mips.deb
dpkg -i libnl1_1.1-6_mips.deb ipvsadm_1.25.clean-1_mips.deb

Of course installing them doesn't mean it actually works, so obviously some actual testing will be required Icon Smile

Attachment
Veteran Member
Posts: 5,417
Registered: ‎03-12-2011
Kudos: 2711
Solutions: 128

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections


@UBNT-ancheng wrote:

We could look into adding those kernel modules into the default config and perhaps the ipvsadm package too. For now though it should be pretty straightforward to build those modules externally and copy them to the router, in fact I've given it a quick try and attached a tarball (extract in "/lib/modules/3.4.27-UBNT/kernel/net/netfilter", "depmod -a", etc.). Also should be straightforward to install the Debian package, for example:


Did I mention you guys are awesome? Man Happy

New Member
Posts: 33
Registered: ‎03-01-2014
Kudos: 10
Solutions: 1

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

I've just been slammed with work this week, so I haven't tested the modules yet, but hope to soon

New Member
Posts: 33
Registered: ‎03-01-2014
Kudos: 10
Solutions: 1

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

Sorry for the delay, meant to report back that these worked great at a kernel level.

 

The keepalilved side needs a lot of work, but please include the IPVS modules in the 1.6 releases if you haven't already!

New Member
Posts: 23
Registered: ‎06-14-2013
Kudos: 13
Solutions: 1

Re: Add Kernel IP Virtual Server (IPVS) to allow load-balancing incoming connections

I am currently testing this on VyOS and once I have the direct-routing config manipulating ipvsadm I can test on EdgeOS as well. After that I will expand to support NAT and TUN mode.

 

 

 

Reply