After Kokoro shipped as the fast, self-hosted TTS tier, the next gap was uncomfortable to admit: OpenAudio Suite’s “voice cloning” feature wasn’t cloning anything. Upload a reference clip, and the backend ran it through Whisper for a transcript, threw the audio away, and quietly assigned your “cloned” voice to whichever stock voice happened to default. It worked, in the sense that it returned audio. It just wasn’t true to what it claimed to do.
Chatterbox (Resemble AI, MIT license) was the fix — a real zero-shot voice cloning model that beat ElevenLabs 65.3% to 24.5% in blind testing. These are the field notes from getting it running on RunPod, across three sessions and five real bugs. For most of this it looked like it wasn’t going to work — the version of this post written mid-session ended on a platform bug with no resolution in sight, and that’s worth being honest about too.
Don’t install it locally
Before writing any deployment code, we checked what pip install chatterbox-tts would actually pull in — hard pins, not suggestions:
torch==2.6.0 transformers==5.2.0 diffusers==0.29.0 numpy<2.0.0
The dev machine’s shared Python environment — used across several unrelated local projects — already had torch 2.11.0, transformers 5.5.4, numpy 2.3.5. Installing Chatterbox locally would have forced a downgrade that could break every other project sharing that interpreter, just to get one feature’s dependencies satisfied.
The good news: Chatterbox’s own plan was always “runs on a single mid-range GPU via serverless,” so this was never meant to run locally anyway. Building it as a RunPod Flash worker — same shape as Kokoro — sidesteps the whole problem.
The wrong century of GPU
First deploy attempt, using flash dev for fast iteration — the same tool that worked fine for Kokoro — threw this on the very first request:
Classic torch/torchaudio ABI mismatch. Checking which GPU RunPod had actually assigned to the AMPERE_24 pool request explained it immediately: an RTX PRO 6000 Blackwell. Chatterbox pins torch==2.6.0, released in December 2024 — before Blackwell’s CUDA compute capability existed.
flash deploy (and flash build) auto-exclude torch, torchaudio, and torchvision from reinstallation — keeping whatever the base image already has, matched to the hardware it’s running on. flash dev has no equivalent behavior; it installs the literal dependencies=[] list every time. For any model with a hard-pinned torch version, flash dev isn’t a safe iteration tool — go straight to flash deploy.
A save function that wants a component that isn’t there
With the GPU mismatch fixed, generation actually completed — worker logs showed ✓ EOS token detected! Stopping generation at step 52, meaning the model produced real audio. It just couldn’t save it:
Newer torchaudio versions deprecated their old save backend in favor of TorchCodec, which wasn’t installed. Rather than add another dependency, the fix mirrored what Kokoro’s worker already does: write the output with soundfile directly instead of torchaudio.save().
The endpoint that wasn’t there (except when it was)
After both fixes, a redeploy reported success, the build artifact on disk had the correct code — and the worker kept running the old, pre-fix version anyway. Then the endpoint stopped showing up in RunPod’s REST v2 API at all, while direct HTTP calls to its run/status/health URLs kept working fine.
Two different sources of truth: the RunPod MCP tools (list-endpoints, REST-backed) and Flash’s own tracking (flash undeploy list, GraphQL-backed). They disagreed. flash undeploy list showed five tracked endpoints when the REST-backed tools showed zero or one — including two endpoints from a much earlier Kokoro session believed already deleted.
flash undeploy --all --force is the tool that actually cleared everything. Lesson for cost hygiene: after any Flash session, check flash undeploy list as well as the RunPod MCP’s list-endpoints — they can genuinely disagree, and only one of them might show you the thing that’s still costing money.
The volume that silently starved every job
Round two started with adding a NetworkVolume to cache Chatterbox’s multi-gigabyte HuggingFace weights across worker restarts. NetworkVolume(name="chatterbox-hf-cache", size=50) with no explicit datacenter defaults to EU-RO-1. Submitting a job against that volume plus an AMPERE_24 GPU request produced… nothing. Not an error — the job sat IN_QUEUE for minutes with zero workers of any status ever appearing.
Pinning the volume explicitly to US-WA-1 produced an actual, loud error this time:
Cross-referencing that list against the GPU pool’s eligible datacenters gave US-CA-2. Even then, AMPERE_24 stalled at zero workers there too — same silent starvation, different combination. Switching the GPU pool to ADA_24 (RTX 4090 — “High” stock per RunPod’s own capacity API, versus AMPERE_24’s scarcer cards) finally got workers running, confirmed within about 90 seconds.
RunPod’s own artifact delivery
With workers finally alive, a new failure showed up — one that had nothing to do with the model code:
This happened across multiple independent workers on the same deploy — not a one-off blip. Workers were crash-looping trying and failing to download their own application code from RunPod’s storage backend. Seven workers had spun up chasing this against a configured max of three — a sign the platform itself was retrying and churning. This looked like transient infrastructure flakiness, not a config mistake fixable from our end, and continuing to retry against it would have just meant paying for more crash-looping GPU workers. Everything was torn down and confirmed clean — the code carried every fix, but with no confirmed successful run through it yet.
Redeployed, first try, it worked
The next session, we redeployed the exact same configuration — no code changes, just flash deploy again against an identical chatterbox_main.py. Worker came up, model loaded, and a runsync request against a real reference clip came back COMPLETED with a valid, decodable WAV file. RunPod’s artifact-distribution backend had just been having a bad afternoon.
Five bugs were real, ours, and fixable; the sixth was the platform’s, and the fix was patience.
That’s a slightly unsatisfying resolution to a debugging story — “we didn’t fix it, we just tried again” — but it’s an honest one. Knowing which bug was which mattered: chasing a platform issue as if it were a config mistake would have burned a lot more GPU time for nothing.
Where it landed
With a confirmed working endpoint, the rest came together in one sitting. backend/tts_chatterbox.py mirrors Kokoro’s RunPod-calling pattern but deliberately has no local fallback — it raises a clear error if the endpoint isn’t configured, rather than silently degrading, since there’s nowhere sensible to degrade to. POST /api/tts/generate now checks whether a custom: voice has a reference clip on file and routes it through Chatterbox instead of resolving to a stock timbre; voices saved without one still fall back to the old adaptation behavior, so nothing broke for what already existed. The frontend base64-encodes the uploaded file client-side and sends it through on save, and the voice list now shows a Cloned or Adapted badge per voice — the product no longer implies more than it delivers.
Verification stayed to the same standard as Kokoro’s: a mocked unit-test suite for the RunPod call path, a live integration test skipped cleanly when no endpoint is configured, and a full browser pass — upload a reference clip, save, hit preview, hear a real cloned voice come back, zero console errors. The RunPod endpoint and its network volume were torn down again immediately afterward, same cost discipline as always.