Skip to content

NVIDIA DGX Spark Integration - Summary

What's on this page

  • Documentation for this infrastructure topic.

What this enables

  • Operators can deploy and operate Inkorporated systems confidently.

Overview

This document provides a comprehensive summary of the NVIDIA DGX Spark integration plan for the Inkorporated homelab infrastructure. The integration will add two NVIDIA DGX Spark systems as dedicated GPU-accelerated worker nodes to the existing k3s Kubernetes cluster.

Current Project Status

Based on the analysis of all documentation, the Inkorporated project is currently in Phase 4 of its implementation lifecycle with the following status:

Implemented Services

  • ✅ Cloudflared (Zero-trust access)
  • ✅ Homepage (User dashboard)
  • ✅ Traefik (Ingress controller)
  • ✅ Authentik (Identity provider)
  • ✅ MongoDB (Database)
  • ✅ Rocket.Chat (Team chat)
  • ✅ WorkAdventure (Virtual office)
  • ✅ Jitsi (Video conferencing)
  • ✅ Coturn (TURN/STUN server)
  • ✅ Vaultwarden (Password manager)
  • ✅ HashiCorp Vault (Secrets management)
  • ✅ Ollama (Local LLM runner)
  • ✅ Open WebUI (Ollama web interface)

Services To Be Implemented

  • 🔄 Langflow (Visual LangChain builder)
  • 🔄 Kokoro TTS (Local TTS server)
  • 🔄 Docling (Document parsing server)
  • 🔄 AppFlowy (Collaborative knowledge base)
  • 🔄 SearXNG (Metasearch engine)
  • 🔄 SurfSense (AI research agent)
  • 🔄 Perplexica (AI search engine)
  • 🔄 LinkWarden (Bookmark manager)
  • 🔄 Homebox (Inventory tracker)
  • 🔄 Home Assistant (Smart home hub)
  • 🔄 Kasm Workspaces (Browser-based workspaces)
  • 🔄 Gitea (Git server)
  • 🔄 Coder (Cloud IDE workspaces)
  • 🔄 CloudNativePG (PostgreSQL operator)
  • 🔄 Longhorn (Distributed block storage)
  • 🔄 MinIO (S3 object storage)
  • 🔄 Velero (Backup/restore)
  • 🔄 Cert-manager (TLS automation)
  • 🔄 MetalLB (LoadBalancer provider)
  • 🔄 Prometheus/Grafana/Loki (Monitoring)
  • 🔄 DGX Spark Integration (GPU worker nodes)

Integration Plan

Phase 1: Prerequisite Infrastructure Deployment (Phase 4 Tasks)

Before integrating the DGX Spark systems, the following infrastructure components must be deployed:

  1. Task 4.01: Deploy NFS CSI Driver
  2. Purpose: Provide NFS-based persistent volumes for shared storage

  3. Status: Not Started

  4. Task 4.02: Deploy Longhorn Storage

  5. Purpose: Distributed block storage for GPU workloads

  6. Status: Not Started

  7. Task 4.03: Deploy MinIO Object Storage

  8. Purpose: S3-compatible object storage for backups and AI datasets

  9. Status: Not Started

  10. Task 4.04: Deploy cert-manager

  11. Purpose: TLS certificate automation for secure communication

  12. Status: Not Started

  13. Task 4.05: Deploy MetalLB

  14. Purpose: LoadBalancer service type for external access

  15. Status: Not Started

  16. Task 4.06: Deploy Observability Stack

  17. Purpose: Monitor GPU metrics and system performance
  18. Status: Not Started

Phase 2: DGX Spark Integration (Task 4.28)

Once prerequisite infrastructure is deployed, integrate the DGX Spark systems:

  1. Physical Setup
  2. Connect DGX systems to LAN switch via Ethernet
  3. Configure static IPs in management VLAN
  4. Connect DGX nodes directly with OSFP cables (400 Gbps total)
  5. Configure private subnet for direct communication

  6. Set up jumbo frames (MTU=9000) and bonding

  7. Software Installation

  8. Install k3s agent on DGX nodes
  9. Label nodes for GPU scheduling: node-role.kubernetes.io/ai-gpu=true

  10. Taint nodes for GPU workloads: dedicated=gpu:NoSchedule

  11. GPU Operator Deployment

  12. Deploy NVIDIA GPU Operator via Helm
  13. Configure driver, device plugin, and DCGM components
  14. Verify GPU functionality

Phase 3: AI Workload Configuration

Update existing AI services to utilize GPU nodes:

  1. Ollama Deployment
  2. Apply node selector for GPU nodes

  3. Configure GPU resource limits/requests

  4. Open WebUI Deployment

  5. Apply same node selector configuration

  6. Configure GPU resources

  7. Langflow Deployment

  8. Configure node affinity and GPU resources

Phase 4: Monitoring and Testing

  1. GPU Metrics Monitoring
  2. Configure DCGM exporter for GPU metrics
  3. Set up Prometheus scraping

  4. Create Grafana dashboards

  5. Performance Testing

  6. Deploy CUDA test workloads
  7. Run distributed training jobs
  8. Validate storage performance

Implementation Timeline

  • Week 1: Complete prerequisite infrastructure deployment (Longhorn, MetalLB, cert-manager, Observability)
  • Week 2: Physical setup and k3s agent installation on DGX systems
  • Week 3: GPU Operator deployment and AI service configuration
  • Week 4: Testing and performance validation

Resources Required

  • Hardware: 2x NVIDIA DGX Spark systems (Grace CPU + Blackwell GPU)
  • Network: 10/25/100 Gbps Ethernet connectivity, 400 Gbps direct OSFP links
  • Software: k3s cluster (existing), NVIDIA GPU Operator Helm chart, Longhorn storage operator

Success Criteria

  • DGX nodes successfully join k3s cluster as worker nodes
  • GPU Operator deployed and functional
  • AI services can be scheduled on GPU nodes
  • GPU metrics available through monitoring stack
  • Performance benchmarks meet expected throughput requirements

Documentation References

  • Complete integration plan: docs/DGX_INTEGRATION_PLAN.md
  • Implementation status tracking: apps/shared/implementation_status.md
  • Progress tracking: memory-bank/progress.md

Next Steps

  1. Begin deployment of prerequisite infrastructure components
  2. Prepare physical infrastructure for DGX systems
  3. Update Ansible playbooks for DGX node configuration
  4. Configure network settings per integration plan