01/03/2016
Do people use VLANs for the right things? Pt.1
Description

This story is in two parts for a reason - Pt 2 is about using EdgePoint-S16s and R8s for building complex sites   Pt 1 covers how VLANs work (which is important for Pt2).   Forgive me if you already know all this, but most users don't so going through it will be beneficial to many (I hope). If you are already an expert on this, Pt2 deals with a real world example of using VLANs between switches we will be putting into production soon, so feel free to skip Pt 1.

EP_testBench.jpg

 

To understand VLANs you have to understand how switches in modern Ethernet systems work. When two computers need to talk to each other they do so on an Ethernet network using the MAC address which is generally hardcoded into the network interface card in the computer or device. Somehow we have to translate the MAC address of the device to the IP address of addigned to that device so we can talk to it beyond the local area network and out into the Internet . This is done through the address resolution protocol or ARP.

 

When a device needs to talk to another device by IP it sends out an arp broadcast with a request for that IP. That request is broadcast over the local network and received by all of the devices on that LAN segment. Assuming that only one of them uses that IP address it will respond by saying “I'm here and this is my MAC address”. Now that the two devices know each other's MAC addresses they can send data between them via ethernet, which is only capable of using the MAC address to send packets locally (Layer-2 in Network Speak).

 

The original Ethernet design used a single physical cable segment to connect all computers on the local area network together. This meant that devices sending data packets to each other could actually have their packets collide with each other on the cable and be corrupted . So a collision protocol was developed to prevent this from occurring . You can't actually prevent collisions with this design, but you can handle them. When a collision is sensed, both devices stop sending, and they each wait a random amount of time before trying again. Since the time they wait is random, one will “win” and it's data will get through first. This is still how half-duplex communication on a LAN using a hub works.

 

The advent of switches removed this problem by creating a device which would not only connect each computer by its own cable but would also keep track of what MAC addresses were connected to what ports on the switch. Switches do this by monitoring the ARP traffic and the MAC addresses and keeping a table of which MAC addresses are on which ports. So no collisions ever occur because the switch doesn't let them happen. This also allows full duplex operation where each device can send and receive at the same time, speeding things up by a factor of two. If needed, packets can be buffered in the switch, and sent on to their destination from there.

VLANswitchBldgFloors.png

VLANs were created on top of this design . The initial problem they were designed to address was in a larger office type environment where you would have several different groups of users who wanted to keep their LANs separate. However if these different groups are located on different floors of a building , running individual cables between the floors for each group caused logistical problems (too many cables) and incurred additional costs . So they came up with a way to allow the different groups to keep their traffic separate while still talking through a single switch. Why might they want to keep their respective LAN segments separate? Maybe they each demand their own DHCP server that they control (politics, politics...) This was done by adding additional information to the packet header in what we now call the VLAN tag. One downside of this is that it made the packet larger which for very large packets could make the overall packet size larger than the LAN's MTU size , which for some devices would cause them to discard the packets. One way around this was to keep all of the VLAN traffic inside the switch itself, where the MTU size wouldn't matter. In fact managed VLAN capable switches tag every single packet that enters them with a VLAN tag even if the outside users are not using VLANs at all.

VLANswitchDiag1.png

Let's look at how a switch actually functions. There are several subsections of a switch which include the management section which will keep track of the MAC addresses and other things, the physical port interfaces that connect to the outside world, a port management section which connects the physical ports in to the switch fabric, and the switch fabric itself. The switch fabric is like a giant crosspoint switch which packets tagged with MAC addresses and possibly VLAN tags are flung into and the switch fabric, using the lookup tables for MAC addresses and VLANs, figures out which ports to send that traffic too.

 

One thing that is seldom discussed with managed switches is that inside the switch fabric, every single packet is VLAN tagged. By default, every packet entering the switch is given a default tag, And all ports use the same tag by default, so they appear to not be doing anything at all with VLANs, but in fact they are. Let's look at a few cases of this in operation.

VLANswitchDiag2.png

This is a drawing of a switch with no additional VLANs beyond the single default VLAN set up. We are showing four ports labeled 1, 2, 11 and 12 . Assuming that the device on port 1 needs to talk to the device on port 12 and the device on port 2 needs to talk to the device on port 11, and assuming the ARP protocol has already connected the devices to each other logically and has "told" the switch which MAC addresses are connecterd to which ports, these devices can communicate through the switch. Upon entering the switch into port 1 a packet without a VLAN tag goes into the port interface system and has the default tag (in this case VLAN 1) added to it. Then it is sent into the switch fabric where it is routed to port 12 because the switch knows based on the MAC table that that is where that device is. On its way out to port 12 the interface system strips off the default tag leaving a packet without any tag at all to be sent out port 12 to the device that is there. This works the same way for all the default ports in a switch which does not have any VLAN tags other than the default set. A port that behaves this way is called “untagged” since it adds or removes the internal tags automatically and does not present any tags to the outside world.

VLANswitchDiag3.png

Let's look at a case with a second VLAN set in the switch. Here we have left the default VLAN on ports 1 and 12, but added untagged VLAN 2 on ports 2 and 11. Packets coming in Port 1 behave just as before and are routed to port 12, however packets coming in Port 2, which still do not have tags from the outside world, have tag 2 added and are then sent in to the switch fabric. The switch fabric will use the MAC addresses it has found to send this packet in to port 11, where that tag is stripped off and the untagged packet sent out . This ends up keeping traffic on port 1 and 12 separate from ports 2 and 11 due to the VLAN tags. What we've actually done here is effectively segmented a single switch into behaving like two separate Ethernet switches. This in itself is extremely useful in a case where you need separate broadcast domains for your network but don't want to add separate physical switches. By using untagged VLAN ports within the switch you are effectively creating separate broadcast domains which will not allow traffic to go between them into a different domain. And the only limit on how you define these ports and VLANs is the internal constraints of the switch and how many VLANs you can create. We use this to segment switches into different operations based on what the groups of ports need to do. For instance, one group of ports can be used to interface between a carrier's fiber optic feed and several routers, while another group of the same switch's ports are used to keep inter-server traffic separate. A third group of ports (with a third untagged VLAN setting) could keep all the broadcast Access Points separate from the rest of the network. (this is a very simple version of what we actually do in our network). And it's all done in a single large physical switch. And since it's all software programmable, it can be changed any time more or less ports need to be associated with one of the domains.

VLANswitchDiag4.png

Now let's look at a different type of port tagging , which is given the name “tagged port”. Tagged ports do not add or remove the VLAN tag – they keep it intact in the packet and let it in or pass it to the outside world. And you can tell the switch which tags to allow in and out of which ports. Here we have set port 11 to be untagged as VLAN 2, so like before packets coming in port 2 will have VLAN tag 2 added to them. The Fabric then routes the packet to port 11, but this time does not remove the VLAN tag, but sends the tagged packet out port 11. In order to be useful whatever device(s) are connected to port 11 will need to understand the VLAN tag; if they don't they may actually discard the packet thinking it's corrupted. But if the device understands VLANs, it will be able to receive the packet successfully and use it.

VLANswitchDiag5.png

In this example we have added port 3 which is an untagged port on VLAN 3 . However we've changed port 11 to be a tagged port tagged as both VLAN 2 and 3 . What will happen now is packets entering physical ports 2 and 3 will show up on port 11 but instead of stripping the VLAN tag it is left in the packet . This means that whatever device is connected to port 11 must be able to handle the larger packet with the VLAN tag in it, as well as understanding what that VLAN tag means . What we are effectively doing here is combining the two broadcast domains in VLANs 2 and 3 to show up on a single port 11 which can now be used outside of the switch by devices that themselves can separate those two VLANs .

VLANswitchDiag6.png

The reverse function also works the same way. If tagged packets show up at port 11 with VLAN tags 2 or 3 on them , the switch fabric will separate them and send them to the proper ports which are untagged VLAN2 and untagged VLAN3. It's easy to see how two physically separate switches each set up this way and each talking to each other on port 11 would actually send port 2 or port 3 traffic to each other and keep that traffic separate using the VLAN tags. Since this can be expanded as much as the switch will physically allow , this is a way to multiplex multiple domains onto a single port for a backbone connection. In our first discussion of why VLANs were created (in a multi-group office environment) this allows for trunking between floors between two switches while keeping the domains for the various work groups completely separated .

 

It's actually quite complex how this is all accomplished, and the details can be interesting for those so inclined – here is an article with some more basic information on the 802.1Q protocol - http://www.dummies.com/how-to/content/how-virtual-local-area-networks-vlans-work.html

 

In the next part of this story I'll look at how this gets implemented with real switches.

Jim

Do people use VLANs for the right things? Pt.1

by ‎01-03-2016 01:24 PM - edited ‎01-03-2016 08:09 PM

This story is in two parts for a reason - Pt 2 is about using EdgePoint-S16s and R8s for building complex sites   Pt 1 covers how VLANs work (which is important for Pt2).   Forgive me if you already know all this, but most users don't so going through it will be beneficial to many (I hope). If you are already an expert on this, Pt2 deals with a real world example of using VLANs between switches we will be putting into production soon, so feel free to skip Pt 1.

EP_testBench.jpg

 

To understand VLANs you have to understand how switches in modern Ethernet systems work. When two computers need to talk to each other they do so on an Ethernet network using the MAC address which is generally hardcoded into the network interface card in the computer or device. Somehow we have to translate the MAC address of the device to the IP address of addigned to that device so we can talk to it beyond the local area network and out into the Internet . This is done through the address resolution protocol or ARP.

 

When a device needs to talk to another device by IP it sends out an arp broadcast with a request for that IP. That request is broadcast over the local network and received by all of the devices on that LAN segment. Assuming that only one of them uses that IP address it will respond by saying “I'm here and this is my MAC address”. Now that the two devices know each other's MAC addresses they can send data between them via ethernet, which is only capable of using the MAC address to send packets locally (Layer-2 in Network Speak).

 

The original Ethernet design used a single physical cable segment to connect all computers on the local area network together. This meant that devices sending data packets to each other could actually have their packets collide with each other on the cable and be corrupted . So a collision protocol was developed to prevent this from occurring . You can't actually prevent collisions with this design, but you can handle them. When a collision is sensed, both devices stop sending, and they each wait a random amount of time before trying again. Since the time they wait is random, one will “win” and it's data will get through first. This is still how half-duplex communication on a LAN using a hub works.

 

The advent of switches removed this problem by creating a device which would not only connect each computer by its own cable but would also keep track of what MAC addresses were connected to what ports on the switch. Switches do this by monitoring the ARP traffic and the MAC addresses and keeping a table of which MAC addresses are on which ports. So no collisions ever occur because the switch doesn't let them happen. This also allows full duplex operation where each device can send and receive at the same time, speeding things up by a factor of two. If needed, packets can be buffered in the switch, and sent on to their destination from there.

VLANswitchBldgFloors.png

VLANs were created on top of this design . The initial problem they were designed to address was in a larger office type environment where you would have several different groups of users who wanted to keep their LANs separate. However if these different groups are located on different floors of a building , running individual cables between the floors for each group caused logistical problems (too many cables) and incurred additional costs . So they came up with a way to allow the different groups to keep their traffic separate while still talking through a single switch. Why might they want to keep their respective LAN segments separate? Maybe they each demand their own DHCP server that they control (politics, politics...) This was done by adding additional information to the packet header in what we now call the VLAN tag. One downside of this is that it made the packet larger which for very large packets could make the overall packet size larger than the LAN's MTU size , which for some devices would cause them to discard the packets. One way around this was to keep all of the VLAN traffic inside the switch itself, where the MTU size wouldn't matter. In fact managed VLAN capable switches tag every single packet that enters them with a VLAN tag even if the outside users are not using VLANs at all.

VLANswitchDiag1.png

Let's look at how a switch actually functions. There are several subsections of a switch which include the management section which will keep track of the MAC addresses and other things, the physical port interfaces that connect to the outside world, a port management section which connects the physical ports in to the switch fabric, and the switch fabric itself. The switch fabric is like a giant crosspoint switch which packets tagged with MAC addresses and possibly VLAN tags are flung into and the switch fabric, using the lookup tables for MAC addresses and VLANs, figures out which ports to send that traffic too.

 

One thing that is seldom discussed with managed switches is that inside the switch fabric, every single packet is VLAN tagged. By default, every packet entering the switch is given a default tag, And all ports use the same tag by default, so they appear to not be doing anything at all with VLANs, but in fact they are. Let's look at a few cases of this in operation.

VLANswitchDiag2.png

This is a drawing of a switch with no additional VLANs beyond the single default VLAN set up. We are showing four ports labeled 1, 2, 11 and 12 . Assuming that the device on port 1 needs to talk to the device on port 12 and the device on port 2 needs to talk to the device on port 11, and assuming the ARP protocol has already connected the devices to each other logically and has "told" the switch which MAC addresses are connecterd to which ports, these devices can communicate through the switch. Upon entering the switch into port 1 a packet without a VLAN tag goes into the port interface system and has the default tag (in this case VLAN 1) added to it. Then it is sent into the switch fabric where it is routed to port 12 because the switch knows based on the MAC table that that is where that device is. On its way out to port 12 the interface system strips off the default tag leaving a packet without any tag at all to be sent out port 12 to the device that is there. This works the same way for all the default ports in a switch which does not have any VLAN tags other than the default set. A port that behaves this way is called “untagged” since it adds or removes the internal tags automatically and does not present any tags to the outside world.

VLANswitchDiag3.png

Let's look at a case with a second VLAN set in the switch. Here we have left the default VLAN on ports 1 and 12, but added untagged VLAN 2 on ports 2 and 11. Packets coming in Port 1 behave just as before and are routed to port 12, however packets coming in Port 2, which still do not have tags from the outside world, have tag 2 added and are then sent in to the switch fabric. The switch fabric will use the MAC addresses it has found to send this packet in to port 11, where that tag is stripped off and the untagged packet sent out . This ends up keeping traffic on port 1 and 12 separate from ports 2 and 11 due to the VLAN tags. What we've actually done here is effectively segmented a single switch into behaving like two separate Ethernet switches. This in itself is extremely useful in a case where you need separate broadcast domains for your network but don't want to add separate physical switches. By using untagged VLAN ports within the switch you are effectively creating separate broadcast domains which will not allow traffic to go between them into a different domain. And the only limit on how you define these ports and VLANs is the internal constraints of the switch and how many VLANs you can create. We use this to segment switches into different operations based on what the groups of ports need to do. For instance, one group of ports can be used to interface between a carrier's fiber optic feed and several routers, while another group of the same switch's ports are used to keep inter-server traffic separate. A third group of ports (with a third untagged VLAN setting) could keep all the broadcast Access Points separate from the rest of the network. (this is a very simple version of what we actually do in our network). And it's all done in a single large physical switch. And since it's all software programmable, it can be changed any time more or less ports need to be associated with one of the domains.

VLANswitchDiag4.png

Now let's look at a different type of port tagging , which is given the name “tagged port”. Tagged ports do not add or remove the VLAN tag – they keep it intact in the packet and let it in or pass it to the outside world. And you can tell the switch which tags to allow in and out of which ports. Here we have set port 11 to be untagged as VLAN 2, so like before packets coming in port 2 will have VLAN tag 2 added to them. The Fabric then routes the packet to port 11, but this time does not remove the VLAN tag, but sends the tagged packet out port 11. In order to be useful whatever device(s) are connected to port 11 will need to understand the VLAN tag; if they don't they may actually discard the packet thinking it's corrupted. But if the device understands VLANs, it will be able to receive the packet successfully and use it.

VLANswitchDiag5.png

In this example we have added port 3 which is an untagged port on VLAN 3 . However we've changed port 11 to be a tagged port tagged as both VLAN 2 and 3 . What will happen now is packets entering physical ports 2 and 3 will show up on port 11 but instead of stripping the VLAN tag it is left in the packet . This means that whatever device is connected to port 11 must be able to handle the larger packet with the VLAN tag in it, as well as understanding what that VLAN tag means . What we are effectively doing here is combining the two broadcast domains in VLANs 2 and 3 to show up on a single port 11 which can now be used outside of the switch by devices that themselves can separate those two VLANs .

VLANswitchDiag6.png

The reverse function also works the same way. If tagged packets show up at port 11 with VLAN tags 2 or 3 on them , the switch fabric will separate them and send them to the proper ports which are untagged VLAN2 and untagged VLAN3. It's easy to see how two physically separate switches each set up this way and each talking to each other on port 11 would actually send port 2 or port 3 traffic to each other and keep that traffic separate using the VLAN tags. Since this can be expanded as much as the switch will physically allow , this is a way to multiplex multiple domains onto a single port for a backbone connection. In our first discussion of why VLANs were created (in a multi-group office environment) this allows for trunking between floors between two switches while keeping the domains for the various work groups completely separated .

 

It's actually quite complex how this is all accomplished, and the details can be interesting for those so inclined – here is an article with some more basic information on the 802.1Q protocol - http://www.dummies.com/how-to/content/how-virtual-local-area-networks-vlans-work.html

 

In the next part of this story I'll look at how this gets implemented with real switches.

Jim

" How can anyone trust Scientists? If new evidence comes along, they change their minds! " Politician's joke (sort of...)
"Humans are allergic to change..They love to say, ‘We’ve always done it this way.’ I try to fight that. "Admiral Grace Hopper, USN, Computer Scientist
"It's not Rocket Science! - Oh wait, Actually it is... "NASA bumper sticker
"Just because you can do something doesn't mean you should."my mantra in the Programming classes I used to teach once upon a time...
Comments
by
on ‎01-03-2016 02:57 PM

Jim,

 

As always enjoyed your article, and really appreciate the effort and time you spent to share your knowledge (in a way everybody should understand) to the community!

Here is some nice info for other forum members, so they can understand the mechanics behind vlans.

 

Thanks!

 

David

by Ubiquiti Employee
on ‎01-03-2016 02:57 PM

Excellent explanation! It's true that there are still many users that have never used VLANs.

by
on ‎01-03-2016 04:19 PM
Great writeup Jim! Think you typoed a but though - mac address stuff is on Layer 2 (L1 being the cables, etc. for the physical interconnects).
by
on ‎01-03-2016 08:10 PM

dpurget -

 

Thanks, you're right - fixed it.   Guess if that's the only major typo things aren't too bad...

Jim

by
‎01-04-2016 07:49 AM - edited ‎01-04-2016 07:57 AM

God read, you can put words to facts.

 

One thing i miss is a short intro paragraph on ethernet frames and ip packets, what are they, where are they used, when does a packet turn into a frame and vice versa. Cisco's docs on frames and packets is not an easy read.

 

Otherwise nice article Man Happy.

by
on ‎01-14-2016 09:32 AM

I believe there is a typo in the third to last paragraph Man Happy

 

Now let's look at a different type of port tagging , which is given the name “tagged port”. Tagged ports do not add or remove the VLAN tag – they keep it intact in the packet and let it in or pass it to the outside world. And you can tell the switch which tags to allow in and out of which ports. Here we have set port 11 to be untagged as VLAN 2, so like before packets coming in port 2 will have VLAN tag 2 added to them. The Fabric then routes the packet to port 11, but this time does not remove the VLAN tag, but sends the tagged packet out port 11. In order to be useful whatever device(s) are connected to port 11 will need to understand the VLAN tag; if they don't they may actually discard the packet thinking it's corrupted. But if the device understands VLANs, it will be able to receive the packet successfully and use it.

I think it is supposed to read Port 2 was untagged VLAN2, Port 11 was set tagged VLAN2.

by
on ‎01-14-2016 09:34 AM

And thanks for this, I can't wait to show this to my employee's!

by
on ‎12-04-2017 10:04 AM

A beautifully written and illustrated article!

 

Here's my more plebian description of how to setup VLANs

in a UniFi routing/switching environment; hope it's useful:
     https://community.ubnt.com/t5/UniFi-Routing-Switching/UniFi-VLANs-simple-explanation-reposted/m-p/21...

 

 

by
on ‎12-21-2018 12:12 PM

Thanks

by
a month ago

excellent

 

Thank you