chipsenkbeil / chipsenkbeil.com Goto Github PK

Rack Setup

1. Bought 25U startech.com 4 post rack
   • More space than need
   • Bigger eyesore than expected
2. Moved old desktop PC (atx) and old proxmox server (mini-itx) to rosewill 4u chassis that got off newegg last cyber monday
   • Poor cable management
   • Maxed out ram of old PC to 32 GB from 16 GB
   • Not enough slots to support all drive bays, but realized going to use freenas as main storage and just leverage SSDs for proxmox in other servers
   • Swapped out old GPU in mini-itx for zotac fanless to reduce noise
   • Swapped out nvidia titan because with power coming from top wouldn't fit in case to another zotac
3. Took plunge and bought2 R710s with X5675 CPUs, 2x 870W PSUs, and 96GB of ram each
   • Worked a lot better than expected, other than a little noisy
   • Decided that I'll just use the two R710s and that I don't need the two original PCs that I moved into rack mounts
4. Bought ubiquiti UDM-Pro (1U) and 24-port switch (1U)
   • Can eventually switch raspberry pis to PoE to reduce cables
   • Old model of 24-port switch (non-pro) is fanless, although it doesn't have 10GB SFP+ ports, only 1GB, but I have no need for those speeds
   • Could write another post about raspberry pi setup with retro pi
     • Swapped out rhino power supply to official supply due to undervoltage
     • Got SN30 Pro+ controllers, had to make sure to switch to Windows/XBox mode as default is Switch, which didn't work well
     • Have notes about experience; make sure to mention rom storage using freenas
5. Bought cyberpower sinewave UPS (2U rack) of 2200W
   • Make sure it is at bottom of rack due to weight
   • Got kill-a-watt to measure total power consumption of rack; curious if UPS can measure anything
6. Bought dell rapid rails to replace navepoint and istarusa rails
   • Will use the earlier rails for network stability, potentially
7. Need to buy cables to hook everything up cleanly, but waiting to get all devices so I can properly measure what cable lengths will work
8. Bought startech.com 12U enclosure to replace 25U
   • Enclosure will help dampen noise a little bit
   • Easier to tuck away in a room or transport than the 25U
   • Total gear size is 8U, leaving room for a 4U chassis if I can move my Freenas mini xl hardware to a rackmount case
     • udm-pro is 1U
     • switch is 1U
     • R710 is 2U (with two, being 4U)
     • Cyberpower UPS is 2U
   • Alternatively, could by synology gear, but I like my freenas setup, understand it enough to get by, and it's much more powerful hardware (8 core/thread, 32 GB of memory)
     • Will be evaluating if can migrate hardware (psu, motherboard, etc) to a new chassis

Network setup

1. Originally had everything available publicly using an nginx server running on a raspberry pi that would proxy to static IPs on my network including freenas and several jails such as subsonic, plex, and transmission. This was annoying to maintain and - for the most part - a security concern as many of these did not need to be public.
2. Decided to move to an internal network, but without needing to remember IP addresses for individual gear
   • Enter DHCP, which is what jails defaulted to, but accessing via IP and looking up what IP they were was a nightmare
   • Then discovered that my edgelite 3 router from ubiquiti could have dnsmasq enabled over the current dhcp/dns setup. This would enable routing by hostname in combination with some domain name. E.g. freenas.example.com could be an internal route even if the address was public
   • With me experimenting using my iPad as my primary device over a laptop, I discovered that IP addresses, self-signed certs, and non-HTTPS was a problem in different ways. Even if I used chrome over safari, accessing HTML5 shell (freenas) or console (proxmox) would fail. Additionally, charting/graphs for my edge router, freenas, and proxmox would not render.
     • Solution was to get real certs for freenas, proxmox, and my edge router. Using cloudflare to sit on top of a google domain of mine, I was able to use Let's Encrypt with a DNS API through cloudflare to get certs without needing a server to be public (compared to HTTP verification)
       • Freenas required me to log in, install acme.sh, provide CF_Token and CF_Account_ID when running acme, and then use a python script called deploy_freenas to connect the cert to freenas (see guide here)
         • Switched to DHCP over static address as no longer had need to statically allocate with hostname working
       • Proxmox was the easiest as it had built-in CF integration and acme registration for domains
         • Does not seem like DHCP for web interface is supported and attempts mentioned on reddit and elsewhere indicate that changing configs get overwritten on power cycling, so sticking with static address, which was easy to add to the edge router config following the dnsmasq guide
       • Edge router involved config changes, but nothing more and followed guide in this repo
3. Considering running nginx on proxmox itself to redirect 443 (SSL) to 8006 per this guide, but 8006 is usually fine on its own
   • Alternatively, have chip.network (no subdomain) serve a page that provides access to different services on my network (research Grafana)
     • Could run Grafana on a Debian container, but might be a good idea to have it exist separately from proxmox, so could repurpose my old Raspberry pi 3 B+ that was running nginx to run Grafana