09/04/2018
Cisco who?
Used Products
×2
×2
×7
×10
×2
Location
Amsterdam, Netherlands
Description

TLDR summary:  Unifi is on the cusp of being enterprise-ready out of the box.  Below is the story of how I got it to work with links to helpful resources.  These are the pitfalls I encountered and the workarounds I used to get it working quickly. 

 

I currently work for a small niche biomedical company that provides antibodies for cancer research.  They are starting to feel the growing pains of expanding their business on the global market from an IT perspective. Each site was an island and supporting them was difficult. I was hired to streamline the network globally and consolidate administration out of our US office.

 

So, there I was...

 

Sitting in a conference room with senior leadership, explaining why we needed to pay for Cisco's SmartNET contract if we want continued support and software upgrades of our networking equipment.  In a big business where a network down could cost the business millions of dollars per hour, that support contract may be worth it.  During the meeting, it was impressed upon me the need to reduce costs and explore non-Cisco options.  They tasked me with designing and building an office in four months with one requirement:  "Anything but Cisco."  So, I jumped on the challenge, and after reviewing Juniper, HP, Aruba and other enterprise vendors, all had the same cost structure.  My direct manager joked about putting in Unifi gear in the offices and calling it a day.


I knew Unifi’s user base is mostly home and small, single-network businesses, etc.  When I looked at them last, they were not enterprise ready.  However, during this project I revisited Ubiquiti.  I ended up getting some products in the Unifi line to test…  and test I did. 

 

The Unifi line is known for wireless.  While Ubiquiti has several other lines that may be more robust for networks, but they don’t utilize the same management platform.  I wanted one pane of glass for my network, especially for remote sites, so I focused on the Unifi line of gear. 

 

The project was to upgrade both the office and a nearby warehouse.  The requirements were to move all the servers to the office and make the smallest footprint I could for the warehouse.  Since the warehouse does all the shipping, we needed a dedicated circuit due to the amount of traffic and size of the printed labels. As part of a global network, I needed to configure BGP at each location to work with the VPN infrastructure we had already put in. I required redundant Internet to keep the sites up and connected to our global network if one circuit went down.  As time grew closer to the office build-out, we found that we had three circuits at the office, two internet, and one point to point to the warehouse.  To take advantage of that additional circuit, I decided to make sure the VoIP phones were routed in and out of that circuit only while Data routed out the primary circuit.  In case of one circuit failure, automatic failover would occur with the circuit failing back again when the failed circuit came back up.

 

The design looked like this:networkdesign-ubnt.PNG

Things I needed to test:

  • BGP
  • Policy routing
  • Load balancing between circuits
  • DHCP relay
  • Radius Authentication of Wireless
  • Mesh Wireless
  • Geo Filtering
  • Failure of equipment
  • Remote access to equipment
  • IPS / Firewall 

 

If you didn’t know already, in addition to live chat and support forums, Ubiquiti has a Discord and Reddit channel.  The number of helpful people on the Discord channel is staggering.  I spent a month just watching and reading other people’s questions.  I eventually jumped in and started asking questions.  The majority of the answers I got were: “Just use Cisco” or “Use EdgeMax for your edge devices” save yourself the trouble of working with the Unifi line, they aren’t ready for the things you are looking to do.  However, enough people said that it could work, even though it is a hassle to customize the gear.  That was all I needed to let me keep going down the path of using only the Unifi line of products to complete this buildout.  In no particular order, a big shout out to the people in Discord that helped me through this journey:

  @vidplace7@Kapyrna@Brontide@jonbloom, @wifiholic, 

@jardin espanol, @t.c.o.a, @ckd@brielle@macgeek312

@jaffe[UniFi-R&S], @CWizard@ilkevinli, @adamD[UniFi-R&S]

 

Discord:  https://discord.me/page/ubnt

 

Jumping around the list a little, I will explain what I did, how easy/hard it was, and provide some helpful links and pictures.  The easy stuff first.

 

IPS
IPS was ruled out due to the reduction of throughput and the fact it is in Beta still.  Host-based security on the endpoints allowed for us not to need this specific feature turned on.IPS-ubnt.PNG

 

Geo Filtering
I didn’t need to turn this one due to host-based security doing the same thing, but I did.  Blocked China and the Russian Federation as being the main offenders of Internet attacks.  Does this stop attackers? Not really but it reduces the threat footprint a little bit.  What I would like to see is a whitelist exception to put in here.

Geofiltering-ubnt.PNG

 

DHCP Relay 
Sure, the USG PRO could handle giving out DHCP to all devices, but I already had an IPAM solution that allows me to track and identify clients.  The relay is in Beta still, but it works beautifully.  After selecting DHCP relay in the Network section go to the Services and configure your relay.

dhcprelay-ubnt.JPG

 

Radius Authentication 
Users accessing corporate wireless back to an AD environment is a breeze.  Create a RADIUS server under Settings > Profiles.  Then under the Wireless Networks select the RADIUS server from the drop-down box. On your Radius server, you will need to add the AP, the Unifi switch and the USG depends on where the authentication server is. The logs will show who is making the request.  A helpful article can be found here:  https://blog.ubnt.com/2016/11/04/managing-radius-authentication-unifi/ 

radiusserver-ubnt.PNG

 

Mesh Wireless
Mesh Wireless is pretty easy once you understand what you need to do.  Connect both APs you are using directly to a PoE switch that can reach the controller and adopt them.  Once connected, select each AP you want to use in this scenario.  I had three access points wired and only one in a spot that I didn’t have cabling, but I did have power.  Under devices on the main page, select each access point.  Click on Config and then scroll down to Wireless Uplinks.  Select “allow meshing to another access point.”  Again, this is in Beta, but it works great.  If you have multiple wired access points, the mesh will roam between them to maintain the quality of the network.  It is also important to note that meshed access points only provide half the original bandwidth due to the backhaul channel needed to maintain the connection.  A more detailed article on this can be found here:  https://help.ubnt.com/hc/en-us/articles/115002262328

wirelessmesh2-ubnt.PNG

 

Now for the difficult part of the project:  BGP, Load Balancing, and Policy Routing.  For my network design, I had four routing decisions to make: 3 Internet paths and one VPN path between the two sites.  BGP is used to route the internal network as well as routing Internet out both sites in case one site went fully down.  The load balancing was between WAN1 and WAN2 for redundant circuits at the primary site.  Finally with policy routing to ensure both Internet circuits are active by routing data out one and voice out the other.  Each section configured via JSON files on the controller.  The Unifi and EdgeMax lines are the same hardware and software in most models.  Staying with the Unifi line, I saw that they are still actively working on the GUI to provide more and more functionality to match what is in the EdgeMAX series already.  

 

A couple of notes of caution:

  1. The unfortunate part of making JSON configurations is you don’t see it in the GUI so if another person comes along to manage the site they may break something inadvertently. 
  2. Just because you can issue set commands via the CLI doesn't mean they will translate well into the JSON file.
  3. Doing the command “mca-ctrl -t dump-cfg” doesn’t show all the code in JSON format.  They are making strides though.  I mentioned this only to let you know you aren’t crazy if you can’t find it in the output. 
  4. Configuration is done on the controller, not the USG, an important distinction in my mind.  Anything you configure on the USG will lose the configuration on reboot/policy push.

USG Advanced Configuration, or how to do something that isn’t in the GUI


This article will show you how to create a JSON file on the controller:   https://help.ubnt.com/hc/en-us/articles/215458888-UniFi-How-to-further-customize-USG-configuration-with-config-gateway-json

 

In essence, you create the commands on the USG using set commands in the CLI.  Save and commit the configuration then run “mca-ctrl -t dump-cfg” to see the JSON format.  In theory, you should be able to cut and paste the section you need into the config.gateway.json file.  It is good to upload your config file to an online JSON editor to troubleshoot and verify.

 

Ubiquiti recommends going to https://jsonlint.com/, but I would recommend looking at http://jsoneditoronline.org/ also.  The latter link shows section trees to help you to determine if that "{" or "," put you in the right section.  The struggle is real if you are not familiar with scripting/programming like I am.  After hours of trying to troubleshoot the error message of missing a “}” or an “:” only to find out it was a trick question, I was missing “,” several lines above where I was looking.

 

jsoneditoronline-ubnt.PNG

 

BGP Routing 
Configuring BGP wasn’t hard, or so I thought.  You can find an EdgeMax article on BGP here:  https://help.ubnt.com/hc/en-us/articles/205222990-EdgeRouter-Border-Gateway-Protocol
The main issue I had was when the routes flipped from a WAN1 to LAN2 at my warehouse. Existing traffic stopped while new traffic routed correctly.  This issue was my biggest block for using Unifi gear.  It took me three weeks of bugging a lot of people in Discord, the forums and support before I figured it out.  It turned out that the NAT table was holding the existing traffic.  If I let the traffic drop for about five minutes, it would switch over to the new route.  If it didn’t, it would stay in the table indefinitely or until the route switched back.  To force traffic over I could issue the command “clear connection-tracking” on the USG.

 

I created a post on the forums showing the issue and the solution located here:  https://community.ubnt.com/t5/UniFi-Routing-Switching/Existing-traffic-blocked-when-BGP-routes-changed-Solved/m-p/2401590 

 

Load Balancing and Policy Routing
Next came load balancing the circuits and policy routing.  Using the following article, I was able to set everything up:  https://help.ubnt.com/hc/en-us/articles/360005460813-UniFi-USG-Advanced-Policy-Based-Routing-.  The article showcases what you can do with advanced routing.  Just need to translate that into JSON afterward.


Unfortunately, the load balancing feature didn’t work for me as intended.  Again, I had issues with connection tracking. Another issue is that when the primary circuit failed all traffic routed out the secondary circuit except the original traffic going out that circuit. It would stop passing traffic for my voice networks going out that interface.loadbalance2-ubnt.PNG

 

Working with the support team, we decided to go down an EdgeMAX configuration for Policy-Based Routing found here:  https://help.ubnt.com/hc/en-us/articles/204952274-EdgeRouter-Policy-Based-Routing The article worked out pretty well, and it worked with the BGP configuration also.  I was able to shorten a couple of commands by creating address groups in the GUI.  If you drop down into the CLI, the address groups will show as a string of numbers.  It allows me to add and remove networks in the GUI without affecting the JSON configuration.

 

addressgroup-ubnt.PNG

 

Hardware failure and replacement
My greatest worry with the Unifi line is that nothing is redundant.  The equipment doesn’t have dual power supplies, and I can’t configure VRRP/HSRP or even switch stacking.  Coming from a world where everything was high availability and redundant this was the biggest mental obstacle to overcome.  Being able to set my expectations with an acceptable downtime allowed me to move forward with this project.  I ended up buying at least one of everything between the two sites, to have on site, in case something failed.


Two main scenarios needed to be resolved for my site to work.

  • A switch failure, PoE or XG
  • USG failure

Each switch has unique configurations to them.  Some ports have just data networks; some have a voice network also.  Wireless ports only have mgmt and the wireless networks on the ports.  So what happens when I lose a switch and forget the configuration?  When replacing a failed switch, the default configuration of the ports is set to “all.”  The failed switch is easy to replace outside of the physical part of moving all the cables.  Remove the old switch, install the new switch.


Adopt the switch then go to Config > Manage Device.  At the top, you will see “copy configuration.”  Click in the box to list the other switches including the failed switch which will be in red.  Select the failed switch and click on Apply.

failedswitch2-ubnt.PNG

 

USG failure is tricky.  If I was onsite, it is just a matter of replacing the old with the new physically.  Inside your devices window, you will need to forget the failed USG.  You can’t have two gateways on one site.  Look in your device window to adopt it.  If the USG doesn’t show up, connect a console cable to it and issue the command “set-inform HTTP://<Name-or-IP-of-controller>:8080/inform” and look at the device window again to adopt.  The USG gets all its configuration from the controller.


Unfortunately, the site is remote, and I only have remote hands to work with, so I had to cheat a little if the gateway fails.  I ended up getting an Out-of-Band modem that I could bypass the USG completely to get me on the internal network.  The added insurance of being able to support a location was worth it.  The USG controls everything; I have three circuits coming into a site but it all worthless if the USG fails.

 

Wishlist

 

Things I would wish for that may or may not already be on the roadmap for Unifi. 

  • Routing configurations in the GUI. Obvious I know.
  • Being able to configure email alerts.  The existing alerts waste a lot of space in the email by showing a color header and the picture of a cloud key.  All I care about is the error msg and from what device.
  • Redundancy on the USG.  VRRP or even active/passive failover.  So much relies on that one device.
  • Put a real plug type on the back of the USG.  The three-prong laptop plug is nonstandard in a data center, and it is hard to find online when in a pinch.

Final thoughts

 

The site has been up now for two weeks, and it seems rock solid.  My journey to learn about the Unifi line over the last few months was challenging.  Easy to learn, hard to master some would say.  I now know a lot about the inner workings of the product, but I know there is still much more to learn.


I would recommend installing Unifi in small to medium size offices if you have the expectations set.  When you come down to it, do you need high availability?  If you can have small outages during maintenance windows, I think you can get away with it.  The cost savings of moving from a Cisco product line to Unifi is huge.  My next project is to retrofit a larger site here in the US with Unifi gear for the same cost the SmartNET contract would be.  I’m working on that budget next.

 

 

Some extra pictures:

 

Traffic-ubnt.JPGDay One Traffic

IMG_20180821_122024.jpgOffice Wiring

 

IMG_20180821_122230.jpgRack Layout Front

 

IMG_20180826_111247.jpgRack Layout Rear

 

 

 

 

Cisco who?

by ‎09-04-2018 03:46 PM - edited ‎09-04-2018 04:07 PM

TLDR summary:  Unifi is on the cusp of being enterprise-ready out of the box.  Below is the story of how I got it to work with links to helpful resources.  These are the pitfalls I encountered and the workarounds I used to get it working quickly. 

 

I currently work for a small niche biomedical company that provides antibodies for cancer research.  They are starting to feel the growing pains of expanding their business on the global market from an IT perspective. Each site was an island and supporting them was difficult. I was hired to streamline the network globally and consolidate administration out of our US office.

 

So, there I was...

 

Sitting in a conference room with senior leadership, explaining why we needed to pay for Cisco's SmartNET contract if we want continued support and software upgrades of our networking equipment.  In a big business where a network down could cost the business millions of dollars per hour, that support contract may be worth it.  During the meeting, it was impressed upon me the need to reduce costs and explore non-Cisco options.  They tasked me with designing and building an office in four months with one requirement:  "Anything but Cisco."  So, I jumped on the challenge, and after reviewing Juniper, HP, Aruba and other enterprise vendors, all had the same cost structure.  My direct manager joked about putting in Unifi gear in the offices and calling it a day.


I knew Unifi’s user base is mostly home and small, single-network businesses, etc.  When I looked at them last, they were not enterprise ready.  However, during this project I revisited Ubiquiti.  I ended up getting some products in the Unifi line to test…  and test I did. 

 

The Unifi line is known for wireless.  While Ubiquiti has several other lines that may be more robust for networks, but they don’t utilize the same management platform.  I wanted one pane of glass for my network, especially for remote sites, so I focused on the Unifi line of gear. 

 

The project was to upgrade both the office and a nearby warehouse.  The requirements were to move all the servers to the office and make the smallest footprint I could for the warehouse.  Since the warehouse does all the shipping, we needed a dedicated circuit due to the amount of traffic and size of the printed labels. As part of a global network, I needed to configure BGP at each location to work with the VPN infrastructure we had already put in. I required redundant Internet to keep the sites up and connected to our global network if one circuit went down.  As time grew closer to the office build-out, we found that we had three circuits at the office, two internet, and one point to point to the warehouse.  To take advantage of that additional circuit, I decided to make sure the VoIP phones were routed in and out of that circuit only while Data routed out the primary circuit.  In case of one circuit failure, automatic failover would occur with the circuit failing back again when the failed circuit came back up.

 

The design looked like this:networkdesign-ubnt.PNG

Things I needed to test:

  • BGP
  • Policy routing
  • Load balancing between circuits
  • DHCP relay
  • Radius Authentication of Wireless
  • Mesh Wireless
  • Geo Filtering
  • Failure of equipment
  • Remote access to equipment
  • IPS / Firewall 

 

If you didn’t know already, in addition to live chat and support forums, Ubiquiti has a Discord and Reddit channel.  The number of helpful people on the Discord channel is staggering.  I spent a month just watching and reading other people’s questions.  I eventually jumped in and started asking questions.  The majority of the answers I got were: “Just use Cisco” or “Use EdgeMax for your edge devices” save yourself the trouble of working with the Unifi line, they aren’t ready for the things you are looking to do.  However, enough people said that it could work, even though it is a hassle to customize the gear.  That was all I needed to let me keep going down the path of using only the Unifi line of products to complete this buildout.  In no particular order, a big shout out to the people in Discord that helped me through this journey:

  @vidplace7@Kapyrna@Brontide@jonbloom, @wifiholic, 

@jardin espanol, @t.c.o.a, @ckd@brielle@macgeek312

@jaffe[UniFi-R&S], @CWizard@ilkevinli, @adamD[UniFi-R&S]

 

Discord:  https://discord.me/page/ubnt

 

Jumping around the list a little, I will explain what I did, how easy/hard it was, and provide some helpful links and pictures.  The easy stuff first.

 

IPS
IPS was ruled out due to the reduction of throughput and the fact it is in Beta still.  Host-based security on the endpoints allowed for us not to need this specific feature turned on.IPS-ubnt.PNG

 

Geo Filtering
I didn’t need to turn this one due to host-based security doing the same thing, but I did.  Blocked China and the Russian Federation as being the main offenders of Internet attacks.  Does this stop attackers? Not really but it reduces the threat footprint a little bit.  What I would like to see is a whitelist exception to put in here.

Geofiltering-ubnt.PNG

 

DHCP Relay 
Sure, the USG PRO could handle giving out DHCP to all devices, but I already had an IPAM solution that allows me to track and identify clients.  The relay is in Beta still, but it works beautifully.  After selecting DHCP relay in the Network section go to the Services and configure your relay.

dhcprelay-ubnt.JPG

 

Radius Authentication 
Users accessing corporate wireless back to an AD environment is a breeze.  Create a RADIUS server under Settings > Profiles.  Then under the Wireless Networks select the RADIUS server from the drop-down box. On your Radius server, you will need to add the AP, the Unifi switch and the USG depends on where the authentication server is. The logs will show who is making the request.  A helpful article can be found here:  https://blog.ubnt.com/2016/11/04/managing-radius-authentication-unifi/ 

radiusserver-ubnt.PNG

 

Mesh Wireless
Mesh Wireless is pretty easy once you understand what you need to do.  Connect both APs you are using directly to a PoE switch that can reach the controller and adopt them.  Once connected, select each AP you want to use in this scenario.  I had three access points wired and only one in a spot that I didn’t have cabling, but I did have power.  Under devices on the main page, select each access point.  Click on Config and then scroll down to Wireless Uplinks.  Select “allow meshing to another access point.”  Again, this is in Beta, but it works great.  If you have multiple wired access points, the mesh will roam between them to maintain the quality of the network.  It is also important to note that meshed access points only provide half the original bandwidth due to the backhaul channel needed to maintain the connection.  A more detailed article on this can be found here:  https://help.ubnt.com/hc/en-us/articles/115002262328

wirelessmesh2-ubnt.PNG

 

Now for the difficult part of the project:  BGP, Load Balancing, and Policy Routing.  For my network design, I had four routing decisions to make: 3 Internet paths and one VPN path between the two sites.  BGP is used to route the internal network as well as routing Internet out both sites in case one site went fully down.  The load balancing was between WAN1 and WAN2 for redundant circuits at the primary site.  Finally with policy routing to ensure both Internet circuits are active by routing data out one and voice out the other.  Each section configured via JSON files on the controller.  The Unifi and EdgeMax lines are the same hardware and software in most models.  Staying with the Unifi line, I saw that they are still actively working on the GUI to provide more and more functionality to match what is in the EdgeMAX series already.  

 

A couple of notes of caution:

  1. The unfortunate part of making JSON configurations is you don’t see it in the GUI so if another person comes along to manage the site they may break something inadvertently. 
  2. Just because you can issue set commands via the CLI doesn't mean they will translate well into the JSON file.
  3. Doing the command “mca-ctrl -t dump-cfg” doesn’t show all the code in JSON format.  They are making strides though.  I mentioned this only to let you know you aren’t crazy if you can’t find it in the output. 
  4. Configuration is done on the controller, not the USG, an important distinction in my mind.  Anything you configure on the USG will lose the configuration on reboot/policy push.

USG Advanced Configuration, or how to do something that isn’t in the GUI


This article will show you how to create a JSON file on the controller:   https://help.ubnt.com/hc/en-us/articles/215458888-UniFi-How-to-further-customize-USG-configuration-w...

 

In essence, you create the commands on the USG using set commands in the CLI.  Save and commit the configuration then run “mca-ctrl -t dump-cfg” to see the JSON format.  In theory, you should be able to cut and paste the section you need into the config.gateway.json file.  It is good to upload your config file to an online JSON editor to troubleshoot and verify.

 

Ubiquiti recommends going to https://jsonlint.com/, but I would recommend looking at http://jsoneditoronline.org/ also.  The latter link shows section trees to help you to determine if that "{" or "," put you in the right section.  The struggle is real if you are not familiar with scripting/programming like I am.  After hours of trying to troubleshoot the error message of missing a “}” or an “:” only to find out it was a trick question, I was missing “,” several lines above where I was looking.

 

jsoneditoronline-ubnt.PNG

 

BGP Routing 
Configuring BGP wasn’t hard, or so I thought.  You can find an EdgeMax article on BGP here:  https://help.ubnt.com/hc/en-us/articles/205222990-EdgeRouter-Border-Gateway-Protocol
The main issue I had was when the routes flipped from a WAN1 to LAN2 at my warehouse. Existing traffic stopped while new traffic routed correctly.  This issue was my biggest block for using Unifi gear.  It took me three weeks of bugging a lot of people in Discord, the forums and support before I figured it out.  It turned out that the NAT table was holding the existing traffic.  If I let the traffic drop for about five minutes, it would switch over to the new route.  If it didn’t, it would stay in the table indefinitely or until the route switched back.  To force traffic over I could issue the command “clear connection-tracking” on the USG.

 

I created a post on the forums showing the issue and the solution located here:  https://community.ubnt.com/t5/UniFi-Routing-Switching/Existing-traffic-blocked-when-BGP-routes-chang... 

 

Load Balancing and Policy Routing
Next came load balancing the circuits and policy routing.  Using the following article, I was able to set everything up:  https://help.ubnt.com/hc/en-us/articles/360005460813-UniFi-USG-Advanced-Policy-Based-Routing-.  The article showcases what you can do with advanced routing.  Just need to translate that into JSON afterward.


Unfortunately, the load balancing feature didn’t work for me as intended.  Again, I had issues with connection tracking. Another issue is that when the primary circuit failed all traffic routed out the secondary circuit except the original traffic going out that circuit. It would stop passing traffic for my voice networks going out that interface.loadbalance2-ubnt.PNG

 

Working with the support team, we decided to go down an EdgeMAX configuration for Policy-Based Routing found here:  https://help.ubnt.com/hc/en-us/articles/204952274-EdgeRouter-Policy-Based-Routing The article worked out pretty well, and it worked with the BGP configuration also.  I was able to shorten a couple of commands by creating address groups in the GUI.  If you drop down into the CLI, the address groups will show as a string of numbers.  It allows me to add and remove networks in the GUI without affecting the JSON configuration.

 

addressgroup-ubnt.PNG

 

Hardware failure and replacement
My greatest worry with the Unifi line is that nothing is redundant.  The equipment doesn’t have dual power supplies, and I can’t configure VRRP/HSRP or even switch stacking.  Coming from a world where everything was high availability and redundant this was the biggest mental obstacle to overcome.  Being able to set my expectations with an acceptable downtime allowed me to move forward with this project.  I ended up buying at least one of everything between the two sites, to have on site, in case something failed.


Two main scenarios needed to be resolved for my site to work.

  • A switch failure, PoE or XG
  • USG failure

Each switch has unique configurations to them.  Some ports have just data networks; some have a voice network also.  Wireless ports only have mgmt and the wireless networks on the ports.  So what happens when I lose a switch and forget the configuration?  When replacing a failed switch, the default configuration of the ports is set to “all.”  The failed switch is easy to replace outside of the physical part of moving all the cables.  Remove the old switch, install the new switch.


Adopt the switch then go to Config > Manage Device.  At the top, you will see “copy configuration.”  Click in the box to list the other switches including the failed switch which will be in red.  Select the failed switch and click on Apply.

failedswitch2-ubnt.PNG

 

USG failure is tricky.  If I was onsite, it is just a matter of replacing the old with the new physically.  Inside your devices window, you will need to forget the failed USG.  You can’t have two gateways on one site.  Look in your device window to adopt it.  If the USG doesn’t show up, connect a console cable to it and issue the command “set-inform HTTP://<Name-or-IP-of-controller>:8080/inform” and look at the device window again to adopt.  The USG gets all its configuration from the controller.


Unfortunately, the site is remote, and I only have remote hands to work with, so I had to cheat a little if the gateway fails.  I ended up getting an Out-of-Band modem that I could bypass the USG completely to get me on the internal network.  The added insurance of being able to support a location was worth it.  The USG controls everything; I have three circuits coming into a site but it all worthless if the USG fails.

 

Wishlist

 

Things I would wish for that may or may not already be on the roadmap for Unifi. 

  • Routing configurations in the GUI. Obvious I know.
  • Being able to configure email alerts.  The existing alerts waste a lot of space in the email by showing a color header and the picture of a cloud key.  All I care about is the error msg and from what device.
  • Redundancy on the USG.  VRRP or even active/passive failover.  So much relies on that one device.
  • Put a real plug type on the back of the USG.  The three-prong laptop plug is nonstandard in a data center, and it is hard to find online when in a pinch.

Final thoughts

 

The site has been up now for two weeks, and it seems rock solid.  My journey to learn about the Unifi line over the last few months was challenging.  Easy to learn, hard to master some would say.  I now know a lot about the inner workings of the product, but I know there is still much more to learn.


I would recommend installing Unifi in small to medium size offices if you have the expectations set.  When you come down to it, do you need high availability?  If you can have small outages during maintenance windows, I think you can get away with it.  The cost savings of moving from a Cisco product line to Unifi is huge.  My next project is to retrofit a larger site here in the US with Unifi gear for the same cost the SmartNET contract would be.  I’m working on that budget next.

 

 

Some extra pictures:

 

Traffic-ubnt.JPGDay One Traffic

IMG_20180821_122024.jpgOffice Wiring

 

IMG_20180821_122230.jpgRack Layout Front

 

IMG_20180826_111247.jpgRack Layout Rear

 

 

 

 

{"location":{"title":" Amsterdam, Netherlands","placeId":"ChIJp-HtVLkJxkcR-zpu8-GfH6c"},"addedProducts":[{"id":"unifi-security-gateway-pro","count":2},{"id":"unifi-switch-16-xg-beta","count":2},{"id":"unifiswitch-48-750w","count":7},{"id":"unifi-ap-ac-shd","count":10},{"id":"unifi-cloud-key","count":2}],"solved":"","numbers":"","description":"","mainImage":"173342i1C4B8E288A99FB3D"}

Comments
by
on ‎09-04-2018 03:59 PM

Absolutly excellent write up and network setup!! Good luck!

by
‎09-05-2018 12:39 AM - edited ‎09-05-2018 12:41 AM

@kenlambert => really really nice implementation ! thumbs up !

A little question ; is the VM server on the POE switch or the USW-16-GX (from the picture, it looks like not on the core USW-16-XG) ?

Just to reduce latency and switches downlink/uplink towards it.

 

by
on ‎09-05-2018 02:38 AM

Nice job, great writeup.

How you deal with the heat, I see you have space between switches.

 

Regards,

 

by
‎09-05-2018 05:43 AM - edited ‎09-05-2018 05:50 AM

@kenlambert=> thanks of the explainations, and yes, keep the 10Gbps port free for better usage if required (change in destination)

One last question : as you hare having 2 sites, with identical hardware listing : USG-Pro, USW-48-500W, UAP-AC-??? and spare parts

Don't you think the simplicity of the management and deployment overcome the complexity of a highly redundant system which anyway, on certain aspect, especially end user, are not (any desktop phone, wifi or desktop with redundant connectivity ?), as I see this ias the primary driver ofr most of the companies to continue to pay large amount of support contract fees, which require at the end hardware replacement, and most of hte time, much more time than what you need to simply replace a USG, a USW or a UAP.
Note :  I suppose that you don't have spare parts from your current/before UBNT provider (Cisco I mean here)

by
on ‎09-05-2018 05:49 AM

@verisarioc - Thank you!

 

@di3 - I have space on the XG for the VM server to upgrade to 10gb ports.  The need isn't there yet.  The majority of corporate services are back in the US office.

 

@danmero - The Unifi line listed heat tolerance is around 104 F.  During my testing I put the switches in a raised temperature environment with low airflow causing the internal temperature to be between 110-120 F for over two months.  I had no issues or failures during that time.  The location where they are now is in a cool datacenter with a hot and cold aisle setup. 

In the picture, you will see there are about 5 inches of space on either side of the PoE switches.  Also may notice in the rack, spacers that fill in the empty spaces.  Cold air to be sucked in from the front of the switch and exhausted out the back into the hot aisle.  

by
on ‎09-05-2018 06:29 AM

@di3

Buying extra hardware is insurance. I highly doubt I will have to use the equipment in the next two years. If a piece goes down, I have an SLA of 4 hours to replace the failed device. I had this same policy with any other vendor in place that I didn't have a support contract for. There are different parts of the network that require different levels of attention. End users, I can get away with this level of support. Core or devices facing the Internet have a higher level of support on them depending on the impact they would have if they failed. Support contracts earn their keep in my book with software upgrades, new features, and security updates.

 

 

 

 

by
on ‎09-06-2018 04:04 AM

Excellent writeup.  Great to follow your thoughts, analysis, issues and solutions.  

by
on ‎09-06-2018 10:45 AM

Great job.  We are currently going throgh some of the pains with the BGP routing (completed) and the failover / load-balancing @kotoritech

by
on ‎09-06-2018 12:47 PM

@OzPHB - Ty.  My gripe with Ubiquiti stories is that no one really dives into what they did to make it work.

 

@dmkjr - Ty.  I hear you on the struggle.  Looks like you are all sorted out now, great job!

by
on ‎09-06-2018 05:09 PM

I find Visual Studio Code to be a great JSON editor. Highlighting, auto-completion, validation with line/column references you can click to jump to the location, and formatting. Lets you collapse sections so you can quickly narrow down your focus. Would definitely have saved you the time chasing your tail on the missing comma.

 

As others have mentioned, great write-up. We've been messing with PBR, and have taken the same approach to redundancy as you have (configure it where practical, but mostly just standardise and keep spares on hand). Remote sites we often put a 3G/4G router with some prepaid data in as our fallback option.

 

Would you be able to speak to the savings in more detail? Not looking for actual money spent, as I imagine that's not free to share, but an indication of the % saving would be great. The first school I manage that used UniFi at all was back in 2011 when we switched wi-fi from Meru to UBNT. We were paying around 10% of the cost per AP with UniFi, and while they didn't have the same performance the licensing costs dropped the overall cost for equivalence to around 10% as well.