02/01/2019
10G Home Data Center
Used Products
×1
×1
×1
×1
×1
×1
×1
×1
×1
×1
×1
Location
Glendale, OH 45246, USA
Description

I couldn't figure out how to cross-post, so I duplicated (also in video).  If someone can do that, this can be deleted.

 

This is an update to a previous story.  I moved, and seriously upgraded my home network to a full on 10G dual fiber Data Center.  The full story is on my wiki, but I took a spare 17'x11' room in the basement, UniFi'ed the crap out of it, added a bunch of power and AC, and below is the result.

 

It's a biggish house, and I wanted to go all out, no compromises with this build since this is our "forever" home.  I'm also going big into home automation, so I wanted a very reliable network with as many things on Ethernet as I could to free up WiFi spectrum.  I'm very pleased with the outcome, my WiFi is providing 400Mbps download speeds.

 

* Cincinnati Bell 1G down/250M up FTTH primary, 200M/6M Spectrum cable fail-over WAN

* XG-8 10G Router with full IPS/IDS enabled

* 4 XG-16 core switches, dual 10G SFP+ LAG uplinks

* 3 US-48 (1 750W POE, 1 48-L2-POE, 1 US-48) + 1 US-16 POE Distribution switches

* 8 US-8 (some POE, some powered by POE) local switches

* 6 UAP-AC-HD WiFi access points

* 1 UAS-XG UniFi and UVC server

* 5 UVC-Pro

* 4 UVC-G3 Cameras

* 5 UVC-Micro cameras

* 5 Ring Floodlight cameras

* 2 Ring Doorbell cameras

* 7 Sonos Connect Amp for audio

* 3 Tivo Bolt for TV

* 2 HD HomeRun for local TV

* 3 Dell rack mount severs (Proxmox and FreeNAS)

* 1 Rack mount Mac Mini server

* 7 VLAN

* All private services (web, email, wiki, Plex, NextCloud)

* Sonnet 10G copper NIC for local PCs

* Dual 10G fiber to 4 PC locations

* Cat6a cabling throughout (80 drops)

* 1.5 ton mini-split AC

* 6 1500VA CyberPower UPS 

 

Network_top.jpgNetwokring Top

 

 

 

 

 

Network_bottom.jpgNetworking Bottom

 

 

Server_top.jpgServer Top

 

 

Entertain_bottom.jpgEntertainment Bottom

 

Entertain_top.jpgEntertainment top

 

Server_bottom.jpgServer bottom

 

Rack_front.jpgFront of Racks

 

Wire_patch15.jpgCable trough

 

Door6.jpgEntrace to my lair

 

 

 

Cameras_status.jpgCamera StatusCameras_all.jpgUVC Cameras

 

My old setups

https://community.ubnt.com/t5/UniFi-Stories/UniFi-house-network-large-with-FTTH-Fioptics-USG-XG-US-8-and-UAP/cns-p/1801850

https://community.ubnt.com/t5/UniFi-Stories/Ubiquiti-Upgrades-for-my-Fioptics-Installation/cns-p/1705612

 

 

10G Home Data Center

by on ‎02-01-2019 06:35 AM

I couldn't figure out how to cross-post, so I duplicated (also in video).  If someone can do that, this can be deleted.

 

This is an update to a previous story.  I moved, and seriously upgraded my home network to a full on 10G dual fiber Data Center.  The full story is on my wiki, but I took a spare 17'x11' room in the basement, UniFi'ed the crap out of it, added a bunch of power and AC, and below is the result.

 

It's a biggish house, and I wanted to go all out, no compromises with this build since this is our "forever" home.  I'm also going big into home automation, so I wanted a very reliable network with as many things on Ethernet as I could to free up WiFi spectrum.  I'm very pleased with the outcome, my WiFi is providing 400Mbps download speeds.

 

* Cincinnati Bell 1G down/250M up FTTH primary, 200M/6M Spectrum cable fail-over WAN

* XG-8 10G Router with full IPS/IDS enabled

* 4 XG-16 core switches, dual 10G SFP+ LAG uplinks

* 3 US-48 (1 750W POE, 1 48-L2-POE, 1 US-48) + 1 US-16 POE Distribution switches

* 8 US-8 (some POE, some powered by POE) local switches

* 6 UAP-AC-HD WiFi access points

* 1 UAS-XG UniFi and UVC server

* 5 UVC-Pro

* 4 UVC-G3 Cameras

* 5 UVC-Micro cameras

* 5 Ring Floodlight cameras

* 2 Ring Doorbell cameras

* 7 Sonos Connect Amp for audio

* 3 Tivo Bolt for TV

* 2 HD HomeRun for local TV

* 3 Dell rack mount severs (Proxmox and FreeNAS)

* 1 Rack mount Mac Mini server

* 7 VLAN

* All private services (web, email, wiki, Plex, NextCloud)

* Sonnet 10G copper NIC for local PCs

* Dual 10G fiber to 4 PC locations

* Cat6a cabling throughout (80 drops)

* 1.5 ton mini-split AC

* 6 1500VA CyberPower UPS 

 

Network_top.jpgNetwokring Top

 

 

 

 

 

Network_bottom.jpgNetworking Bottom

 

 

Server_top.jpgServer Top

 

 

Entertain_bottom.jpgEntertainment Bottom

 

Entertain_top.jpgEntertainment top

 

Server_bottom.jpgServer bottom

 

Rack_front.jpgFront of Racks

 

Wire_patch15.jpgCable trough

 

Door6.jpgEntrace to my lair

 

 

 

Cameras_status.jpgCamera StatusCameras_all.jpgUVC Cameras

 

My old setups

https://community.ubnt.com/t5/UniFi-Stories/UniFi-house-network-large-with-FTTH-Fioptics-USG-XG-US-8...

https://community.ubnt.com/t5/UniFi-Stories/Ubiquiti-Upgrades-for-my-Fioptics-Installation/cns-p/170...

 

 

{"location":{"title":"Glendale, OH 45246, USA","placeId":"ChIJ5ZpHE-BNQIgROs400sOI628"},"addedProducts":[{"id":"unifi-video-camera-g3","count":1},{"id":"unifi-video-g3-micro","count":1},{"id":"unifi-video-g3-pro-camera","count":1}],"solved":"","numbers":"","description":"","mainImage":"192423i91AA3D85F80BE225"

 

 

{"location":{"title":"Glendale, OH 45246, USA","placeId":"ChIJ5ZpHE-BNQIgROs400sOI628"},"addedProducts":[{"id":"unifi-video-camer

{"location":{"title":"Glendale, OH 45246, USA","placeId":"ChIJ5ZpHE-BNQIgROs400sOI628"},"addedProducts":[{"id":"l2-managed-poe-gigabit-switch-with-sfp","count":1},{"id":"unifi-application-server","count":1},{"id":"unifi-switch-8-60w-b","count":1},{"id":"unifi-switch-8-150w","count":1},{"id":"unifi-security-gateway-xg","count":1},{"id":"unifi-switch-16-150w","count":1},{"id":"unifi-switch-16-xg-beta","count":1},{"id":"unifi-switch-48","count":1},{"id":"unifiswitch-48-750w","count":1},{"id":"unifi-ac-hd","count":1},{"id":"unifi-switch-8-beta","count":1}],"solved":"","numbers":"","description":"","mainImage":"192426i2ED0A8D0118FD447"}

Comments
by
on ‎02-03-2019 12:47 PM

So, looks like it doesn't want to let me add these attachments, so I put them on my Wiki page.  If you have thoughts on improving my naming scheme I'm open to input, I've tried to number the switches and things in a sensible manner.  The diagram got a little busy, but I plan to go back and put port numbers on the lines, but trying to get them to visually connect to the right port was just too much work.

 

Network_Diagrams

by
‎02-03-2019 12:58 PM - edited ‎02-03-2019 01:04 PM

@bferrell Amazing setup!  Very clean.

 

I have been looking for a 1U brush cable management panel that matches the Ubiquiti Unifi gear.

 

Would you mind telling me where you purchased yours?  Or did you customize a stock panel?

 

I am also looking for blank panels that match the Unifi coloring as well.

by
on ‎02-03-2019 01:16 PM

Yea, I did my own, after seeing someone else who had done it, so I just got regular blanks and brush plates and some "Aluminum" colored spray paint at the hardware store.

by
on ‎02-03-2019 01:35 PM

@bferrell 

 

Thank you!  Your panels are the closest color-matched panels I’ve seen. 

 

Do you recall which brand or which “aluminum” color you used?

by
on ‎02-03-2019 04:07 PM

@bferrell Visio looks top notch as expected.  Any chance you can tell me where you got those UBNT icons/svg/whatever they are?  I grabbed some posted on the forum but they aren't that clean.  Seeing yours gives me something to work off of for mine.

by
on ‎02-03-2019 07:59 PM

Its a fair question.  I don't honestly remember.  I got a starter set by googling UBNT visio or somesuch, and then for the newer things I think I just grabbed a image from Google and resized it to be honest.  Again, if folks have feedback I'm all ears, I struggle with the right way to show 10G versus 1G and fiber versus copper, and keep it all readable.  As you know, if it's not documented it's not maintainable, and it's not easy to read, well, maintenance is misery...

by
on ‎02-04-2019 02:21 AM

 

sub-buzz-7145-1509409754-1

 

Now that's one hell of a crazy setup, Love it!!

 

by
on ‎02-05-2019 05:46 AM

I believe the set in this posting are the Visio symbols I ended up starting with.

 

https://community.ubnt.com/t5/EdgeRouter/VISIO-shapes-stencils/td-p/834416

 

I'm pretty sure this is the Krylon Aluminum colored paint I used.

https://www.lowes.com/pd/Krylon-Fusion-All-in-one-General-Purpose-Gloss-Aluminum-Metallic-Spray-Pain...

 

Crazy like a Fox!

by
on ‎02-05-2019 08:09 AM

Is each dell server running their own instance of Proxmox or are they workign in a cluster mode?

by
on ‎02-05-2019 08:10 AM

They're both running proxmox, but in cluster mode, which is a little awkward as it wants 3 for a quorum.  I plan to get redundant hardware for both so I have a cluster of 4 and can setup high availability.