Docker

The repo ships docker/Dockerfile for the API server. It installs vaak[all], ffmpeg and the CUDA runtime so every backend is available.

Build

git clone https://github.com/mirukullacode/vaak
cd vaak
docker build -f docker/Dockerfile -t vaak:latest .

The image is large because of torch and NeMo. Build once and reuse.

Run on CPU

docker run --rm -p 8000:8000 \
  -e VAAK_MODEL=faster-whisper:small \
  -e VAAK_COMPUTE_TYPE=int8 \
  -v vaak-cache:/root/.cache/huggingface \
  vaak:latest

Run on an NVIDIA GPU

Needs the NVIDIA Container Toolkit on the host.

docker run --rm --gpus all -p 8000:8000 \
  -e VAAK_MODEL=faster-whisper:large-v3-turbo \
  -e VAAK_DEVICE=cuda \
  -e VAAK_COMPUTE_TYPE=float16 \
  -v vaak-cache:/root/.cache/huggingface \
  vaak:latest

What the container does

The entrypoint runs:

vaak serve --host 0.0.0.0 --port 8000

VAAK_MODEL, VAAK_DEVICE, VAAK_HOST, VAAK_PORT and VAAK_COMPUTE_TYPE configure it. See configuration.

Weights download on first request into /root/.cache/huggingface. Mount a named volume there, as above, so restarts do not download again.

Health check

curl http://localhost:8000/health

Returns {"status":"ok","model":"faster-whisper:large-v3-turbo"} once the model is loaded. Before that, the route returns 503, which makes it a usable readiness probe.

docker compose

services:
  vaak:
    image: vaak:latest
    ports: ["8000:8000"]
    environment:
      VAAK_MODEL: faster-whisper:small
      VAAK_DEVICE: auto
    volumes:
      - vaak-cache:/root/.cache/huggingface
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
volumes:
  vaak-cache: {}

Remove the deploy block for CPU-only hosts.

Smaller images

If you only need Whisper, edit the Dockerfile to install vaak[whisper] instead of vaak[all]. That drops NeMo and roughly halves the image.