feat: add NVIDIA Jetson GPU support

[toolboc]

Summary

Add Dockerfile.jetson and docker-compose.jetson.yml for building and running the whisper-asr-webservice on NVIDIA Jetson devices with full GPU acceleration.

Closes #359
Relates to #54, #133

Target Platform

• NVIDIA Jetson Orin (Nano / NX / AGX) — JetPack 6.x, L4T R36.x, CUDA 12.6, aarch64
• Configurable via build args for other Jetson generations

What's Included

File	Purpose
Dockerfile.jetson	Multi-stage build with CUDA support
docker-compose.jetson.yml	Compose file for Jetson with GPU passthrough

Key Technical Decisions

1. CTranslate2 compiled from source — PyPI ships CPU-only aarch64 wheels. Built with -DWITH_CUDA=ON -DCUDA_ARCH_LIST="8.7" against JetPack's CUDA toolkit.
2. Jetson AI Lab pip index — PyTorch, torchaudio, and onnxruntime-gpu installed from https://pypi.jetson-ai-lab.io/jp6/cu126/+simple/ using --index-url (not --extra-index-url) because pip prefers manylinux_2_28 (CPU) over linux_aarch64 (CUDA) wheels when both are available.
3. Bypasses Poetry resolver — poetry-core's PEP 517 metadata generation merges [tool.poetry.dependencies] source mappings with [project.optional-dependencies], producing incorrect version constraints (e.g. torch==2.7.1+cu126 instead of the actual version). Dependencies are installed explicitly via pip with a constraints file protecting CUDA packages.
4. torchaudio compatibility shim — Jetson AI Lab torchaudio builds strip AudioMetaData, info(), and list_audio_backends(). A soundfile-based .pth monkey-patch restores them for pyannote.audio 3.x compatibility.
5. torch.load compatibility shim — PyTorch >=2.6 defaults weights_only=True, but pyannote VAD checkpoints contain omegaconf.ListConfig globals. The shim defaults weights_only=False when None is passed (as lightning_fabric does).
6. huggingface_hub use_auth_token→token shim — huggingface_hub 1.5.0 removed the deprecated use_auth_token parameter. pyannote.audio and whisperx still pass use_auth_token=. The shim translates it to token= for hf_hub_download, model_info, and hf_hub_url across all submodules.
7. Guard step — Force-reinstalls torch, torchaudio, and onnxruntime-gpu from the Jetson index after dependency resolution, ensuring CPU-only PyPI wheels haven't overwritten them.

Verified On

• Jetson Orin, JetPack 6.2.2, L4T R36.5.0, CUDA 12.6
• torch.cuda.is_available() = True, device = Orin (8, 7)
• CTranslate2 4.4.0 CUDA compute types: float16, bfloat16, int8, float32
• All three ASR engines tested and passing:
  • faster_whisper — 200 OK, GPU transcription ✓
  • openai_whisper — 200 OK, GPU transcription ✓
  • whisperx — 200 OK, word-level timestamps ✓
• HF_TOKEN support for gated model access (diarization) ✓

Build & Run

# Build
docker compose -f docker-compose.jetson.yml build

# Run
docker compose -f docker-compose.jetson.yml up

Docker Hub

Pre-built image available:

docker pull toolboc/whisper-asr-webservice-jetson:jp6.1-cu12.6-py3.10