feature/speech #17

Merged

marti merged 6 commits from feature/speech into main

2026-10-01 00:55:06 +02:00

marti commented

2026-10-01 00:54:26 +02:00

Owner

No description provided.

marti added 6 commits

2026-10-01 00:54:26 +02:00

build(cum-tools): add the audio.cpp flake input 40a02a0166

feat(cum-tools): catalog tts:kokoro and stt:qwen3-asr 4dade8158e

feat(cum-tools): serve TTS/STT from audio.cpp as always-resident llama-swap models 9e7a9b3ea9

Both speech models spawn their own audiocpp_server under llama-swap. They
are preloaded at boot, ttl 0, and live in a persistent group that swaps
false and exclusive false, so they neither evict each other nor get
evicted when a chat model claims the GPU. weights are fetched onto /mnt/ai
by audio-models.service mirroring sd-models, and Kokoro's G2P gets eSpeak
through per-model env vars (the package keeps no static eSpeak).

feat(cum-tools): advertise audio modes through LiteLLM without a token window ea31795aca

Speech entries get max_input_tokens only when the catalog entry has a ctx:
the window for audio is seconds, not tokens, so emitting one would have
Open WebUI reject TTS/STT models for lacking a token budget.

docs: count the two new speech models in the README host table 9c5a94aa06

sp 45acdd3fa2

marti merged commit eec5498bd5 into main

2026-10-01 00:55:06 +02:00

marti referenced this pull request from a commit

2026-10-01 00:55:07 +02:00

Merge pull request 'feature/speech' (#17) from feature/speech into main

marti deleted branch feature/speech

2026-10-01 00:55:08 +02:00