MiniMax H3 on MLX: Local Video+Audio on M3 Ultra
MiniMax shipped open H3 weights: a joint video+stereo-audio diffusion system, not a chat LLM. We wanted a concrete answer for the Mac side of the fleet: does the Apple-native path load, and can it produce a real clip on the 512 GB M3 Ultra?
mlx-serve PR #122.Stock
v26.8.1 dies on load. Once on PR122, text-to-video and first/last-frame FL2VA both returned playable frames with native stereo audio.
What we ran
| Piece | Choice |
|---|---|
| Checkpoint | ddalcu/MiniMax-H3-FL2VA-MLX-Serve-8bit @ 32bfc37f |
| Base | MiniMaxAI/MiniMax-H3 FL2VA (text / first-last frame → video+audio) |
| Quant | MLX affine 8-bit, group size 64 (DiT + text encoder); VAEs dense |
| Server | mlx-serve 26.8.0-pr122.7620516 (Actions artifact from PR #122) |
| Bind | 127.0.0.1:11234 only — no LAN, no LaunchAgent |
| API | POST /v1/video/generations |
H3 is not a chat model. Pointing /v1/chat/completions at it returns a named 400. That is correct behavior.
Compatibility gate
Stock release binary v26.8.1 loaded the safetensors, then aborted with:
MISSING WEIGHT: model.norm.weightIt treated the pack like a text LLM. The H3 path lives on the open PR branch — DiT + dual VAEs + joint sampling — not mainline yet. PR notes also claim large speed work versus an earlier internal path; we did not re-run that baseline, so the only speed claims below are from this session.
Measured smokes
| Request | Result | Wall | Notes |
|---|---|---|---|
| 64×64, 5f, 2 steps | PASS | ~3 s | rgb non-uniform; stereo PCM @ 32 kHz |
| 256×256, 5f, 12 steps | PASS | ~6 s | DiT sample ~3.2 s; decode ~0.5 s |
| 960×768, 124f, 30 steps, FL2VA | PASS | ~19.3 min sample + ~64 s VAE | Chewy puppy→adult; stream=true |
| Large non-stream JSON response | GEN OK / HTTP WriteFailed | — | Prefer SSE stream for big canvases |
Memory shape during generation: DiT ~20 GB resident mlx-active; text encoder loads, encodes, and releases; VAEs load for decode then drop. Peak is staged components, not the full 69 GB sum at once.
Demo: Chewy, pup → adult
First/last-frame FL2VA with two real photos of Chewy (chocolate Lab). First frame: outdoor puppy. Last frame: adult Chewy cropped from a couch shot (Bennie removed so the morph stays single-subject). Prompt locked identity and asked for continuous growth, no cut/dissolve, no second dog, soft breathing soundtrack.
960×768 · trimmed to ~3.5 s · 24 fps · H.264 + AAC · on-device CG. Kept the strong outdoor growth beat and cut before the weak late morph.
# skeleton of the request (keyframes base64-encoded)
curl -N http://127.0.0.1:11234/v1/video/generations \
-H 'Content-Type: application/json' \
-d '{
"prompt": "…continuous growth of one chocolate Lab…",
"first_frame_image": "<png-b64>",
"last_frame_image": "<png-b64>",
"num_frames": 124,
"width": 960,
"height": 768,
"steps": 30,
"seed": 42,
"stream": true,
"fast": true
}'Operational notes
- Streaming matters. A successful large generation still failed once when returning the full base64 blob non-stream (
WriteFailed). SSEprogress→completeis the reliable path. - Canvas grid. Width/height must be multiples of 32; otherwise a named 400.
- No LoRA on this backend.
lora_path→ named 400 for H3. - Frame ladder. Frame counts snap to the model’s
17k+5grid (5 → 5, 40 → 56, 124 → 124). - Quality bar. This is an 8-bit local FL2VA path at ~768p short edge, not hosted Context-IR + 2K regenerate. Treat demos as capability proof, not production grade.
Verdict for the fleet
Apple Silicon can host H3 FL2VA today if you pin the PR122 mlx-serve binary and the 69 GB 8-bit pack. It is useful for offline creative smoke and identity morph experiments on the M3 Ultra. It is not yet a drop-in production media service: mainline server support is incomplete, generation is slow at real canvases, and large responses need streaming discipline.
Next if we keep it: longer 10–15 s clips, native ~1344×768 quality run with explicit wall-clock table, Ref2VA pack when an MLX conversion lands, and a LaunchAgent only after the PR stabilizes.