Skip to content

Project Roadmap

Current state and planned improvements.

Current State

Area Status
Services 65 stacks across 8 categories
Servers 2 (Unraid: 41 stacks, Development: 13 stacks, deploy=false: 1)
Monitoring OTel Collector → Mimir/Loki/Tempo → Grafana + Uptime Kuma
Deployment Komodo GitOps fully operational
Quality Gates 7 pre-commit hooks
CI/CD Komodo Builds for custom Dockerfile builds
Networking Tailscale VPN, Caddy reverse proxy (*.home), AdGuard DNS
Cloudflare Tunnel, Hyperdrive (2 configs), Workers scaffold, WAI bridge via LiteLLM
Media Jellyfin (HW transcode), Navidrome
Orchestration Airflow 3.x (GitDagBundle), n8n (workflow automation)
Dashboard Homepage (service dashboard with Docker integration)
ADRs 3 (Lakekeeper, Hyperdrive+Workers, Workers AI)

Category Breakdown

Category Count Status
Applications 20 Active, Homepage dashboard
Database 11 Stable
Networking 9 Caddy + AdGuard HA deployed
Monitoring 6 Full observability stack
LLM 6 Active
Development 4 Stable (Komodo deploy=false)
Storage 3 Stable
Orchestration 2 Airflow 3.x + n8n operational

Near-Term Goals

Monitoring Improvements

  • Expand Prometheus scrape coverage to all services
  • Add custom Grafana dashboards for each service category
  • Configure alerting rules in Grafana (not just Ntfy notifications)
  • Add log-based alerting for critical services
  • Implement SLO/SLI dashboards for key services

Quality & Automation

  • Add compose.yaml validation tests (service naming, port uniqueness)
  • Automate TOML linting beyond taplo-format (validate required fields)
  • Add git hooks to prevent common pitfalls (relative run_directory)
  • Improve pre_deploy scripts with idempotency checks

Media Stack

  • Grafana dashboard for Jellyfin transcoding stats (HW vs SW)
  • Automate music library cleanup (duplicates, low-quality)
  • Evaluate Navidrome playlist backup/restore
  • Evaluate media download automation (successor to MeTube)

Infrastructure Improvements

Backup & Disaster Recovery

  • Automated database backups (PostgreSQL, ClickHouse)
  • Volume snapshot automation via Unraid API
  • Off-site backup for critical data (Immich photos, Gitea repos)
  • Document and test full server restore procedure
  • Git repo mirror (backup Gitea to external service)

Network & Security

  • Tailscale ACL audit — restrict service access per device
  • Certificate rotation for ts.net domains
  • Network segmentation between categories
  • Container vulnerability scanning in CI/CD

Service Additions Being Considered

Applications

  • Matrix/Synapse — self-hosted messaging
  • Paperless-ngx — document management
  • ~~Homepage — dashboard for service links~~ → Homepage deployed (ghcr.io/gethomepage/homepage)

Database

  • Valkey — Redis-compatible cache (open source)
  • Dragonfly — high-performance Redis alternative

Monitoring

  • Pyroscope — continuous profiling
  • Alloy — Grafana's OTel-native collector (replace OTel Collector)
  • ~~Uptime monitoring~~ → Uptime Kuma deployed (32 checks: 22 unraid + 10 dev)

Networking

  • ~~AdGuard High Availability~~ — Primary + replica + sync deployed (30min lag, automatic failover)
  • Headscale — self-hosted Tailscale control server
  • Stirling PDF — PDF manipulation service

Documentation Improvements

  • Service-specific runbooks (startup, shutdown, backup per service)
  • Onboarding guide for new developers
  • Architecture decision records (ADRs) for key choices → ADR-0001 (Gravitino→Lakekeeper), ADR-0002 (Hyperdrive+Workers), ADR-0003 (Workers AI via LiteLLM) completed
  • API documentation for Unraid GraphQL and Komodo CLI integrations
  • Port mapping reference table (service → port → URL)

Edge Compute Layer (NEW — 2026-07-18)

Cloudflare Hyperdrive + Workers integration. See ADR-0002 and ADR-0003 for design.

Completed

  • Workers VPC connectivity service + 2 Hyperdrive configs (OpenTofu managed)
  • Supabase PG TLS enabled (self-signed cert via conf.d/)
  • cloudflared attached to supabase-internal Docker network
  • Workers scaffold (workers/template/): wrangler 4, TS 5.7, vitest, pg 8.16.3
  • cloudflare-exporter deployed + Grafana cloudflare-overview dashboard
  • Supabase PG datasource in Grafana (uid=supabase)
  • LiteLLM cf-qwen3-embedding model (1024 dims, 119 langs)

Pending

  • Pilot worker deploy — Phase 1+2 of ADR-0002 currently DEFERRED. ~~synthnews-api~~ dropped 2026-07-18 (Docker stack decommissioned). Replacement TBD (gmail-webhook-classifier per ADR-0003).
  • Workers CI via GitHub Actions — wrangler deploy on push to workers/ dir
  • Custom GraphQL scraper for Hyperdrive metrics — hyperdriveQueriesAdaptiveGroups + hyperdrivePoolSizesAdaptiveGroups. No public Prometheus exporter supports them yet (as of Jul 2026).
  • LiteLLM chat models via Workers AI (Phase 2 of ADR-0003) — @cf/meta/llama-3.2-1b-instruct for summarize/classify tasks
  • CF Access per protected Worker endpoint (per ADR-0002 risk table — opt-in per worker)

Long-Term Vision

  1. Full observability — every service emits metrics, logs, and traces
  2. Zero-downtime deploys — rolling updates where possible
  3. Automated testing — compose validation, integration tests
  4. Multi-region — potential third server for geographic redundancy
  5. Self-healing — auto-restart policies and health check driven recovery
  6. Cost tracking — resource usage dashboards per service

Contributing to This Roadmap

This roadmap is maintained as a living document. To suggest changes:

  1. Open an issue using bd (beads issue tracker): bd ready
  2. Discuss the proposed addition or improvement
  3. Update this file when decisions are made