02/01/2019
10G Home Data Center
Used Products
×1
×1
×1
×1
×1
×1
×1
×1
×1
×1
×1
Location
Glendale, OH 45246, USA
Description

I couldn't figure out how to cross-post, so I duplicated (also in video).  If someone can do that, this can be deleted.

 

This is an update to a previous story.  I moved, and seriously upgraded my home network to a full on 10G dual fiber Data Center.  The full story is on my wiki, but I took a spare 17'x11' room in the basement, UniFi'ed the crap out of it, added a bunch of power and AC, and below is the result.

 

It's a biggish house, and I wanted to go all out, no compromises with this build since this is our "forever" home.  I'm also going big into home automation, so I wanted a very reliable network with as many things on Ethernet as I could to free up WiFi spectrum.  I'm very pleased with the outcome, my WiFi is providing 400Mbps download speeds.

 

* Cincinnati Bell 1G down/250M up FTTH primary, 200M/6M Spectrum cable fail-over WAN

* XG-8 10G Router with full IPS/IDS enabled

* 4 XG-16 core switches, dual 10G SFP+ LAG uplinks

* 3 US-48 (1 750W POE, 1 48-L2-POE, 1 US-48) + 1 US-16 POE Distribution switches

* 8 US-8 (some POE, some powered by POE) local switches

* 6 UAP-AC-HD WiFi access points

* 1 UAS-XG UniFi and UVC server

* 5 UVC-Pro

* 4 UVC-G3 Cameras

* 5 UVC-Micro cameras

* 5 Ring Floodlight cameras

* 2 Ring Doorbell cameras

* 7 Sonos Connect Amp for audio

* 3 Tivo Bolt for TV

* 2 HD HomeRun for local TV

* 3 Dell rack mount severs (Proxmox and FreeNAS)

* 1 Rack mount Mac Mini server

* 7 VLAN

* All private services (web, email, wiki, Plex, NextCloud)

* Sonnet 10G copper NIC for local PCs

* Dual 10G fiber to 4 PC locations

* Cat6a cabling throughout (80 drops)

* 1.5 ton mini-split AC

* 6 1500VA CyberPower UPS 

 

Network_top.jpgNetwokring Top

 

 

 

 

 

Network_bottom.jpgNetworking Bottom

 

 

Server_top.jpgServer Top

 

 

Entertain_bottom.jpgEntertainment Bottom

 

Entertain_top.jpgEntertainment top

 

Server_bottom.jpgServer bottom

 

Rack_front.jpgFront of Racks

 

Wire_patch15.jpgCable trough

 

Door6.jpgEntrace to my lair

 

 

 

Cameras_status.jpgCamera StatusCameras_all.jpgUVC Cameras

 

My old setups

https://community.ubnt.com/t5/UniFi-Stories/UniFi-house-network-large-with-FTTH-Fioptics-USG-XG-US-8-and-UAP/cns-p/1801850

https://community.ubnt.com/t5/UniFi-Stories/Ubiquiti-Upgrades-for-my-Fioptics-Installation/cns-p/1705612

 

 

10G Home Data Center

by on ‎02-01-2019 06:35 AM

I couldn't figure out how to cross-post, so I duplicated (also in video).  If someone can do that, this can be deleted.

 

This is an update to a previous story.  I moved, and seriously upgraded my home network to a full on 10G dual fiber Data Center.  The full story is on my wiki, but I took a spare 17'x11' room in the basement, UniFi'ed the crap out of it, added a bunch of power and AC, and below is the result.

 

It's a biggish house, and I wanted to go all out, no compromises with this build since this is our "forever" home.  I'm also going big into home automation, so I wanted a very reliable network with as many things on Ethernet as I could to free up WiFi spectrum.  I'm very pleased with the outcome, my WiFi is providing 400Mbps download speeds.

 

* Cincinnati Bell 1G down/250M up FTTH primary, 200M/6M Spectrum cable fail-over WAN

* XG-8 10G Router with full IPS/IDS enabled

* 4 XG-16 core switches, dual 10G SFP+ LAG uplinks

* 3 US-48 (1 750W POE, 1 48-L2-POE, 1 US-48) + 1 US-16 POE Distribution switches

* 8 US-8 (some POE, some powered by POE) local switches

* 6 UAP-AC-HD WiFi access points

* 1 UAS-XG UniFi and UVC server

* 5 UVC-Pro

* 4 UVC-G3 Cameras

* 5 UVC-Micro cameras

* 5 Ring Floodlight cameras

* 2 Ring Doorbell cameras

* 7 Sonos Connect Amp for audio

* 3 Tivo Bolt for TV

* 2 HD HomeRun for local TV

* 3 Dell rack mount severs (Proxmox and FreeNAS)

* 1 Rack mount Mac Mini server

* 7 VLAN

* All private services (web, email, wiki, Plex, NextCloud)

* Sonnet 10G copper NIC for local PCs

* Dual 10G fiber to 4 PC locations

* Cat6a cabling throughout (80 drops)

* 1.5 ton mini-split AC

* 6 1500VA CyberPower UPS 

 

Network_top.jpgNetwokring Top

 

 

 

 

 

Network_bottom.jpgNetworking Bottom

 

 

Server_top.jpgServer Top

 

 

Entertain_bottom.jpgEntertainment Bottom

 

Entertain_top.jpgEntertainment top

 

Server_bottom.jpgServer bottom

 

Rack_front.jpgFront of Racks

 

Wire_patch15.jpgCable trough

 

Door6.jpgEntrace to my lair

 

 

 

Cameras_status.jpgCamera StatusCameras_all.jpgUVC Cameras

 

My old setups

https://community.ubnt.com/t5/UniFi-Stories/UniFi-house-network-large-with-FTTH-Fioptics-USG-XG-US-8...

https://community.ubnt.com/t5/UniFi-Stories/Ubiquiti-Upgrades-for-my-Fioptics-Installation/cns-p/170...

 

 

{"location":{"title":"Glendale, OH 45246, USA","placeId":"ChIJ5ZpHE-BNQIgROs400sOI628"},"addedProducts":[{"id":"unifi-video-camera-g3","count":1},{"id":"unifi-video-g3-micro","count":1},{"id":"unifi-video-g3-pro-camera","count":1}],"solved":"","numbers":"","description":"","mainImage":"192423i91AA3D85F80BE225"

 

 

{"location":{"title":"Glendale, OH 45246, USA","placeId":"ChIJ5ZpHE-BNQIgROs400sOI628"},"addedProducts":[{"id":"unifi-video-camer

{"location":{"title":"Glendale, OH 45246, USA","placeId":"ChIJ5ZpHE-BNQIgROs400sOI628"},"addedProducts":[{"id":"l2-managed-poe-gigabit-switch-with-sfp","count":1},{"id":"unifi-application-server","count":1},{"id":"unifi-switch-8-60w-b","count":1},{"id":"unifi-switch-8-150w","count":1},{"id":"unifi-security-gateway-xg","count":1},{"id":"unifi-switch-16-150w","count":1},{"id":"unifi-switch-16-xg-beta","count":1},{"id":"unifi-switch-48","count":1},{"id":"unifiswitch-48-750w","count":1},{"id":"unifi-ac-hd","count":1},{"id":"unifi-switch-8-beta","count":1}],"solved":"","numbers":"","description":"","mainImage":"192426i2ED0A8D0118FD447"}

Comments
by
on ‎02-14-2019 01:33 PM

@Scud I just made it up, so that it would trigger my memory, so Red is WAN, Yellow is network interconnect, orange are security cameras, green are UPS, blue standard users, purple are GUEST.

by
on ‎02-15-2019 04:59 AM

Question for you, Are you running a Cloud Key on this setup? and if not what are you running it off of and are you able to access it remotely off-site? 

by
‎02-15-2019 07:15 AM - edited ‎02-15-2019 07:16 AM

@PHNetworks1 If you check out the blog post I reference you can see it all, but I'm running the controller on the UAS-XG application server, and yes, I can access it offsite.  I ran the gen 1 cloud key for a long time (actually I had a pair that I kept up to date with one as a cold spare for a long time).  Mostly upgraded so I could have a reliable UVC (video server) for my 16 camera setup, otherwise I'd run the controller on a VM probably.

by
on ‎02-15-2019 01:55 PM

Where did you get white racks?  Did you paint them?  Very cool setup.  Must've taken a dog year to fully configure, run wire, and finish out the installation.  Your home data center is more on point than some enterprise data centers I know.

by
on ‎02-15-2019 02:06 PM

@CarbonFiber Yea, check the blog entry I posted above for more details, but they're Great Lakes racks, from their "sound and security" line, and came white.  I got it from my brother in law who is in the surplus business.  I think it was originally in a Walgreen's or CVS.  I would've rather that they be black, but mostly I wish they had space for PDUs and cable management, but I got a good deal on them.  :-)

by
‎02-15-2019 02:11 PM - edited ‎02-15-2019 02:14 PM

@bferrell  Do you have to pay the subscription from unifi for having the remote access? 

by
on ‎02-15-2019 02:19 PM

@PHNetworks1 No, there is no subscription for UniFi that I'm aware of, and I've used UBNT for years now on mulitple sites.  Are you talking about their "Elite" support/service?  I do not subscribe to that, no.

by
3 weeks ago

@bferrell Regarding the load-balancing, you cannot load-balance only the download side since the response packets will always come to the IP address that the query originated from. Unless of course you register/buy (?) your own Autonomous System and enter into a peering agreement with your ISPs Man Happy.

What I did when I was dual-homed was that I had some clients in a "load balance address group" and only those were load balanced across the two WAN links while all others used the main link with the second link as fail over. I used an EdgeRouter and don't know how this is done with Unifi routers, but since they're basically also running EdgeOS inside, it should be possible.

 

How do you use the FreeNAS vs. QNAP systems? If the QNAP has all the HDDs, what's inside the FreeNAS server and what is it used for?

 

Lastly, how did you find the time to set all this up? I usually do not have time for my home networking stuff before the family's at sleep, which leaves the time slot between 11pm and 2am ;o)

by
3 weeks ago

Impresssive what and how. However, why?

by
3 weeks ago

@kiwifruta Because I could.  Actually, if you read the wiki entry you'll see it's because I run all of my own services, and I wanted to learn many of the skills, and to encourage the market development of these products.

 

@Alestrix Good idea o the address group.  I may try that sometime, and dedicate the fiber to my severs and main computers.  The QNAP is really just a media store now, the FreeNAS box has all of the storage for the virtualziation - images, backups, ISOs, everything.  The NextCloud storage is local to the 720.  The Plex VM pulls medai from the Plex.  I found the QNAP was not up to the task to server Plex and NextCloud, which is why I went down the Proxmox VM hosting path, but I haven't bought a chasis big enough for all of the storage the QNAP has yet.  As to time, I'm in dual income no kids situation, and my wife works a lot, so I get lots of free time.  Kind of unicorn, I guess, but there it is.