Reply
Emerging Member
Posts: 85
Registered: ‎04-17-2014
Kudos: 89
Solutions: 1

ES-48 LAG... something seems wrong

 

 

I have an ES-48 Lite for quite some time now and am very happy with it. I have 5 LAG, each consisting of 2 Gb links, running from servers and UAP. The other ports are regular switching ports, with end user equipments. I also have just one VLAN configured.

 

Everything seems to work fine, except I have discoevered soemthing weird...

 

My UTM (FGT-61E), configured on a single Ethernet port, keeps on receiving packets on its interface that should never reach it, as they should reach the correct MAC on the local subnet.

 

To be more accurate, it seems that packets coming from LAG equipments are broadcasted to all ports, instead of just reaching the correct MAC on the correct port. 

 

My interpretation is that teh quipment will not find the destination MAC, so it talks to the router. But why is that ??

 

Am I missing some configuration ? I would relly appreciate some help. It is litteraly millions of packets per day being discarded by the UTM...

The configuration is somewhere wrong.

Emerging Member
Posts: 85
Registered: ‎04-17-2014
Kudos: 89
Solutions: 1

Re: ES-48 LAG... something seems wrong

OK... 

 

I did an ARP manually from one of the servers using LAG. 

The request just won't go through, and I have no answer for this interface.

 

Forcing the request foing through a non LAG interface works fine....

 

Why do my ARP request won't go through a LAG port ??

 

 

server:~ philippe$ arp 192.168.0.12

? (192.168.0.12) at 40:6c:8f:36:3b:e0 on en2 ifscope [ethernet]

server:~ philippe$ arp -i bond0 192.168.0.12

192.168.0.12 (192.168.0.12) -- no entry on bond0

Emerging Member
Posts: 85
Registered: ‎04-17-2014
Kudos: 89
Solutions: 1

Re: ES-48 LAG... something seems wrong

Rebooting the EdgeSwitch seems to have solved the issue for now.

 

That is so weird.

Member
Posts: 139
Registered: ‎01-25-2012
Kudos: 17
Solutions: 5

Re: ES-48 LAG... something seems wrong

Don't take this the wrong way as I don't know you or your experience and I'm only sharing for mine. For a long time I erroneously assumed that a LAG was the equivalent of a singe piece of wire/fiber, just with more bandwidth. Like an equal load sharing kind of thing. My experience with LAGs on other platforms (Dell, Brocade, Juniper) is that the traffic is divided, based on whatever priority you set, in such a way that MAC address A will always go down Interface 1 and MAC B will always go down Interface 2. Or traffic from IP 1 will always go down Interface 1, and IP2 will always go down Interface 2.

We learned this when one half of a LAG went down but certain traffic was still getting through and the rest wasnt. It was based on the table the device had to divide the traffic. Until we rebooted the switch and got the LAG back up it was a mystery. Maybe this is what happened to you. And I could be totally wrong about my understanding of a LAG as I've only gotten into them this year and some Google-fu.
Highlighted
Member
Posts: 212
Registered: ‎05-01-2014
Kudos: 56
Solutions: 9

Re: ES-48 LAG... something seems wrong


@corbywanwrote:
Don't take this the wrong way as I don't know you or your experience and I'm only sharing for mine. For a long time I erroneously assumed that a LAG was the equivalent of a singe piece of wire/fiber, just with more bandwidth. Like an equal load sharing kind of thing. My experience with LAGs on other platforms (Dell, Brocade, Juniper) is that the traffic is divided, based on whatever priority you set, in such a way that MAC address A will always go down Interface 1 and MAC B will always go down Interface 2. Or traffic from IP 1 will always go down Interface 1, and IP2 will always go down Interface 2.

We learned this when one half of a LAG went down but certain traffic was still getting through and the rest wasnt. It was based on the table the device had to divide the traffic. Until we rebooted the switch and got the LAG back up it was a mystery. Maybe this is what happened to you. And I could be totally wrong about my understanding of a LAG as I've only gotten into them this year and some Google-fu.

This largely depends on the type of LAG you have setup.  LACP and LBFO LAGs in general solve this problem quite well, as the link will have the bandwidth of both connections and also be redundant if a link fails.  But both switches need to be configured to support it.

Emerging Member
Posts: 85
Registered: ‎04-17-2014
Kudos: 89
Solutions: 1

Re: ES-48 LAG... something seems wrong

To start wirh, I don't take any of your comments in the wrong way, and experience and knowledge sharing is always welcome. And at last, someone answers ! Man Happy

 

On my side I've been playing with 802.1ad for a few years now, with different vendors  Cisco, HP, Apple, so it is all LACP, but I would qualify myself as an expert of any sort !

 

In this case, any link can fail without any interruption to the traffic, and bringing the link back brings the full bandwidth back. It has been totally transparent. I've been really used to such behavior, and never had an issue with MAC tables.

 

I replaced my HP switch with the ES-48 about 8 months ago, keeping a very similar setup. At least, I did not change the way aggregates were setup on the client and server sides. 

 

Ubiquiti terminlogoy (LAG) and documentation is a bit light to me and I just wonder if I'm missing a configuration point.

Static vs dynamic ? Load balance modes ? They may seem obvious, but don't really match other vendor's wording.

And I guess the GUI is misleading... Working from the CLI is necessary.

 

What I saw is however the most bizarre thing for a switch : it forwarded packets to the wrong interfaces ! I don't know why, I had not change the configuration for many months...

Member
Posts: 139
Registered: ‎01-25-2012
Kudos: 17
Solutions: 5

Re: ES-48 LAG... something seems wrong

Yeah. Interesting. UBNT uses the term "Port Channels" but also does LACP. They seem to function differently. Maybe working with LACO specifically and not LAG/Port Channels will give you better results?
Member
Posts: 212
Registered: ‎05-01-2014
Kudos: 56
Solutions: 9

Re: ES-48 LAG... something seems wrong


@phastierwrote:

To start wirh, I don't take any of your comments in the wrong way, and experience and knowledge sharing is always welcome. And at last, someone answers ! Man Happy

 

On my side I've been playing with 802.1ad for a few years now, with different vendors  Cisco, HP, Apple, so it is all LACP, but I would qualify myself as an expert of any sort !

 

In this case, any link can fail without any interruption to the traffic, and bringing the link back brings the full bandwidth back. It has been totally transparent. I've been really used to such behavior, and never had an issue with MAC tables.

 

I replaced my HP switch with the ES-48 about 8 months ago, keeping a very similar setup. At least, I did not change the way aggregates were setup on the client and server sides. 

 

Ubiquiti terminlogoy (LAG) and documentation is a bit light to me and I just wonder if I'm missing a configuration point.

Static vs dynamic ? Load balance modes ? They may seem obvious, but don't really match other vendor's wording.

And I guess the GUI is misleading... Working from the CLI is necessary.

 

What I saw is however the most bizarre thing for a switch : it forwarded packets to the wrong interfaces ! I don't know why, I had not change the configuration for many months...


Glad to hear it, sounds like what the sharing of knowledge of experience is supposed to be :-)

 

I don’t consider myself to be an expert either, but I have a fair amount of experience with ubiquiti’s implementation of LAGs. We use them at our facility, and have them setup between Cisco switches as well, and all our links work correctly as LBFO.

 

That being said, it was definitely confusing at first coming from other vendors and seeing how UBNT implemented LAGs in edgeswitches. The LACP settings should be left alone for the most part (though make sure it is set to enabled) and only apply to the capabilities of individual ports, not the LAGs. The LAGs need to be set to dynamic not static, if that’s what you have already, you might have another issue.

 

Can you post a piece of your switch configuration with the relevant a ports and port-channel configs?

Reply