Giter Club home page Giter Club logo

my-homelab's Introduction

My Homelab Kubernetes-based HTPC Setup

htk8s diagrams

This repository contains the configuration for a Kubernetes-based Home Theater PC (HTPC) setup running on k3s, a lightweight Kubernetes distribution. The system is designed to provide a comprehensive media management and streaming solution using various open-source applications.

System Components

The setup includes the following applications:

  1. Media Management:

    • Sonarr: For managing TV shows
    • Radarr: For managing movies
    • Bazarr: For managing subtitles
  2. Download Management:

    • Transmission: BitTorrent client for downloading media
    • Jackett: For searching and managing torrent trackers
  3. Media Servers:

    • Jellyfin: Open-source media server for streaming content
  4. Infrastructure:

    • Traefik: Reverse proxy and load balancer
    • cert-manager: For managing SSL/TLS certificates
    • MetalLB: Load balancer implementation for bare metal Kubernetes clusters

Architecture and System Design

The system is designed using a microservices architecture, with each component running as a separate service within the Kubernetes cluster. Here's a detailed breakdown of the architecture:

1. Kubernetes Infrastructure

  • The system uses k3s as the Kubernetes distribution, providing a lightweight and easy-to-install cluster.
  • Deployments are managed using Kustomize, allowing for easy customization and overlay-based configuration management.
  • All applications and services are deployed in the htpc namespace.

2. Networking and Ingress

  • Traefik serves as the ingress controller and reverse proxy:
    • Configured using a Helm chart (version 28.0.0) with custom values.
    • Uses LoadBalancer service type with a specific IP (192.168.1.245).
    • Configured to redirect HTTP to HTTPS.
    • Custom middleware is set up for security headers and authentication.
  • MetalLB provides a load balancer implementation:
    • Configured to use the IP range 192.168.1.240/28.
    • Set up using a Helm chart (version 0.14.5).
  • cert-manager automatically manages SSL/TLS certificates:
    • Installed via Helm chart (version 1.14.5).
    • Configured with ClusterIssuer resources for both staging and production environments.
    • Uses Cloudflare DNS for ACME DNS-01 challenge.
    • Issues wildcard certificates for *.my-homelab.party.
  • Ingress is managed through Traefik IngressRoute custom resources:
    • Each application (Jellyfin, Jackett, Sonarr, etc.) has its own route.
    • The Traefik dashboard is also exposed with basic authentication.

3. Storage

  • Persistent storage is provided using hostPath volumes, mapping to the /opt/htpc directory on the host system.
  • Each application has its own subdirectory within this storage, ensuring data persistence and separation:
    • Transmission: /opt/htpc/transmission
    • Sonarr: /opt/htpc/sonarr
    • Radarr: /opt/htpc/radarr
    • Jackett: /opt/htpc/jackett
    • Bazarr: /opt/htpc/bazarr
    • Jellyfin: /opt/htpc/jellyfin
  • Shared directories are used for downloads and media:
    • Downloads: /opt/htpc/downloads
    • TV Shows: /opt/htpc/media/tv
    • Movies: /opt/htpc/media/movies
  • Volume mounts are configured in each deployment to map these directories to the appropriate locations within containers.
  • A PersistentVolumeClaim named "local-path-pvc" is defined, using the "local-path" storage class, for potential use by applications requiring dynamic provisioning.

4. Media Management Workflow

  1. Content Discovery:

    • Sonarr (TV shows) and Radarr (movies) monitor for new content.
    • They use Jackett to search across multiple torrent trackers.
  2. Download Process:

    • When new content is found, Sonarr/Radarr send download requests to Transmission.
    • Transmission handles the torrent downloads.
  3. Post-Processing:

    • Once downloads are complete, Sonarr/Radarr move and rename the files to the appropriate media directories.
    • Bazarr automatically searches for and downloads subtitles for the new content.
  4. Media Serving:

    • Jellyfin scans the media directories and makes the content available for streaming.
    • Users can access the Jellyfin interface through a web browser or compatible apps.

5. Security and Access

  • All services are exposed externally through Traefik, which handles SSL termination.
  • cert-manager is configured to automatically obtain and renew Let's Encrypt certificates.
  • Certificates are stored and managed using Cloudflare DNS:
    • The system is configured to use Cloudflare as the DNS provider for DNS-01 challenge.
    • A Cloudflare API token is stored as a Kubernetes secret for secure access.
    • The domain "my-homelab.party" is used for all services, with subdomains for each application.
  • Basic authentication is set up for the Traefik dashboard for added security.
  • The system uses ClusterIssuer and Certificate custom resources to manage SSL/TLS certificates:
    • Staging and production environments have separate issuers.
    • Wildcard certificates are used to cover all subdomains.
  • All sensitive information, including API keys and tokens, are stored as Kubernetes secrets.

6. Configuration Management

  • The repository uses a base + overlays structure with Kustomize:
    • base/ contains the core configurations for all services.
    • overlays/ contains environment-specific configurations (staging and production).
  • Environment variables and sensitive data are managed through Kubernetes secrets and ConfigMaps.

7. Continuous Integration/Continuous Deployment (CI/CD)

  • GitHub Actions workflows are set up for automated testing and deployment:
    • Pull Request CI (push.yml):
      • Triggered on all pull requests.
      • Runs YAML linting using yamllint.
      • Validates Kubernetes manifests using kubeconform.
      • Tests the manifest update process.
    • Merge CI (merge.yml):
      • Triggered on pushes to the main branch.
      • Updates the install.yaml manifest file.
      • Commits and pushes changes if the manifest is modified.
  • Scripts in the scripts/ directory facilitate the CI/CD process:
    • update-manifests.sh: Generates the combined install.yaml file using kustomize.
    • validate.sh: Runs validation checks on YAML files and Kustomize overlays.
    • deploy.sh: Handles the deployment process, including separating CRDs and waiting for their establishment.
  • The repository uses different configurations for staging and production:
    • Overlays in overlays/staging/ and overlays/production/ allow for environment-specific settings.
    • CI processes use the production overlay for generating the final manifest.
  • Image tags are set to "latest" for all applications in both staging and production overlays, allowing for easy updates.

Deployment and Usage

  1. Ensure you have a k3s cluster set up.
  2. Clone this repository to your local machine.
  3. Customize the configurations in the base/ and overlays/ directories as needed.
  4. Use the provided scripts in the scripts/ directory to deploy and manage the setup:
    • update-manifests.sh: Generates the combined install.yaml file.
    • deploy.sh: Applies the configurations to your cluster.
    • validate.sh: Validates the Kubernetes manifests.

Credits

This project is inspired by and builds upon htk8s.

my-homelab's People

Contributors

chaitanya2692 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.