DGX SPARK · LOCAL INFERENCE

Qwen3.8
27B Uncensored

A reproducible, isolated OpenAI-compatible deployment on NVIDIA Grace Blackwell, privately reachable over Tailscale.

Implementation in progress GB10 · CUDA 13 · ARM64
121 GiBUnified memory
32KInitial context
45%Memory target
8100Local API port

Architecture

Deliberately isolated

MODELHF FP8 checkpoint
→
RUNTIMENVIDIA vLLM
→
LOCAL127.0.0.1:8100
→
PRIVATETailscale HTTPS

The service has its own container, cache, API key, logs, and port. LTX-2.5 processes and ports are observed for safety but never managed by this project.

Model

Qwen3.8 27B

  • FP8 checkpoint
  • Multimodal architecture
  • Reasoning and tool calls
  • OpenAI-compatible API

Network

Tailnet only

vLLM binds to localhost. Tailscale Serve terminates HTTPS and proxies the API privately. No Funnel or public ingress is enabled.

Validation matrix

What we test

System preflight Model integrity API health Deterministic chat Output coherence Streaming Tool-call JSON Mac/Tailscale route LTX isolation

Reproducibility

Everything except secrets and weights

The repository contains the Compose definition, guarded lifecycle scripts, smoke tests, benchmark tooling, Tailscale setup, and operational documentation. Model weights, API keys, Hugging Face tokens, and local artifacts stay outside Git.