logo1
GSD Skill Creator
Den Operations Guide

Documentation > Specialized Systems > Den Operations Guide

Guide IDSS-1
AudienceSysAdmins, Operators, Teachers
PrerequisitesGS-1: Installation, T-7: Knowledge Packs
Time30 minutes
DifficultyAdvanced

Den Operations Guide

The Den is the home lab infrastructure that serves as gsd-skill-creator's physical deployment platform. It transforms commodity hardware into an educational and operational environment where knowledge packs come alive through real server management, VM provisioning, and hands-on system administration. This guide covers everything you need to set up, configure, and operate the Den infrastructure using the scripts and configurations in the infra/ directory.

What is the Den?

The Den is a home lab designed as an educational platform. Rather than treating infrastructure as a black box, the Den exposes every layer -- from PXE boot and bare-metal provisioning through VM management to application-level monitoring -- as a learning surface. Students, operators, and teachers interact with real systems while skill-creator tracks patterns and captures operational knowledge as reusable skills.

The Den architecture spans three tiers: the physical host layer (hardware discovery and resource budgeting), the virtualization layer (hypervisor-agnostic VM provisioning), and the application layer (Minecraft servers, monitoring stacks, and educational workloads). Each tier is managed by bash scripts in infra/ that abstract away distribution-specific details while preserving full operational transparency.

Hardware Discovery and Resource Budgeting

The Den begins with hardware discovery. The infra/local/ directory contains machine-specific data generated by discovery scripts. The hardware-capabilities-local.yaml file captures a complete inventory of the host system: CPU model and core counts, memory capacity, storage devices with types and sizes, GPU capabilities, network interfaces, USB controllers, audio hardware, and distribution details including kernel version and security modules.

Resource budgeting is computed from hardware capabilities. The resource-budget.yaml file defines how physical resources are allocated across workloads. A typical budget reserves a portion of RAM and CPU cores for the host operating system, allocates resources for the Minecraft VM (including dedicated RAM, CPU cores, and storage), and reports unallocated capacity available for additional VMs or experiments. The budget calculation also assigns a tier classification -- such as "comfortable" when resources exceed minimum requirements with significant headroom.

Infrastructure Scripts

The infra/ directory organizes operational scripts by function. Each subdirectory targets a specific infrastructure concern:

  • Platform scripts -- Hardware discovery and distribution abstraction. These scripts detect CPU, memory, storage, GPU, network, USB, and audio capabilities, then generate the YAML inventory files that downstream provisioning scripts consume. Distribution abstraction handles differences between Tier 1 distributions (Ubuntu, Fedora, Arch) and their package managers, init systems, and security frameworks.
  • VM provisioning -- Hypervisor-agnostic VM creation supporting KVM, VMware, and VirtualBox. Template-based provisioning uses kickstart files and configuration templates from infra/templates/ to produce consistent VMs regardless of the underlying hypervisor. Network configuration is handled through bridge interfaces detected during hardware discovery.
  • PXE boot -- Network boot server setup for bare-metal provisioning. PXE scripts configure DHCP, TFTP, and HTTP services to serve installation media and kickstart templates over the local network, enabling automated OS deployment to physical machines.
  • Backup -- RCON-quiesced backup system with a 24/7/4 rotation scheme (24 hourly, 7 daily, 4 weekly backups). Before each backup, the system issues RCON commands to quiesce running game servers (save-all, save-off), performs the backup, then resumes (save-on). This ensures data consistency without downtime.
  • Monitoring -- Prometheus-based metrics collection with configurable alert rules. Monitors system health (CPU, memory, disk), application metrics (player count, TPS for game servers), and infrastructure state (VM status, network connectivity). Alert rules trigger notifications when thresholds are breached.

Knowledge Packs

Knowledge packs bridge the Den's infrastructure with educational content. The infra/packs/ directory contains structured study materials that reference real Den systems. The RFC knowledge pack, for example, stores cached RFC documents alongside generated analysis reports in JSON, Markdown, and BibTeX formats -- enabling students to study networking standards against the live network infrastructure they can inspect and modify.

Knowledge packs integrate with the Den at multiple levels. TRADE-101 packs use Den hardware inventory data to teach capacity planning with real numbers. ENGR-101 packs reference VM provisioning scripts to teach infrastructure-as-code principles. TECH-101 packs use the monitoring stack to teach observability and alerting. Each pack connects abstract concepts to tangible Den infrastructure that students can explore hands-on.

Output and Deployment Artifacts

The infra/output/ directory stores generated deployment artifacts. When provisioning scripts run, they produce configuration files, deployment manifests, and state records in this directory. The Minecraft subdirectory (infra/output/minecraft/) collects server-specific outputs such as generated configurations, backup manifests, and operational logs. These artifacts provide an audit trail for every infrastructure change and enable reproducible deployments.

Local Configuration

The infra/local/ directory is gitignored because it contains actual system data -- MAC addresses, IP addresses, serial numbers, and hostnames. The hardware-values.yaml file stores raw hardware detection values before they are processed into the capabilities format. This separation ensures that provisioning logic lives in version control while machine-specific secrets remain local.

When setting up a new Den installation, run the hardware discovery scripts first to populate infra/local/, then run the budget calculator to generate resource allocations. From there, VM provisioning, PXE boot, and monitoring can be configured using the discovered hardware profile as input.

Operational Runbooks

Den runbooks codify common operational procedures as step-by-step guides. These cover tasks such as initial Den setup on new hardware, adding a new VM workload to the resource budget, recovering from a failed backup rotation, and migrating a running VM between hypervisors. Runbooks reference the specific scripts, configuration files, and monitoring dashboards involved in each procedure, ensuring that operators can follow them without prior Den experience.

What's Next