mirror of
https://github.com/QuentinFuxa/WhisperLiveKit.git
synced 2026-03-06 22:04:06 +00:00
docs: add benchmark collaboration call, voxtral in powered-by section
This commit is contained in:
15
BENCHMARK.md
15
BENCHMARK.md
@@ -188,3 +188,18 @@ python test_backend_offline.py --backend voxtral-mlx --audio your_file.wav --no-
|
||||
|
||||
The benchmark harness computes WER and timestamp accuracy automatically when ground truth
|
||||
`.transcript.json` files exist alongside the audio files. See `audio_tests/` for the format.
|
||||
|
||||
---
|
||||
|
||||
## Help Us Benchmark on More Hardware
|
||||
|
||||
These results are from a single Apple M4 machine. We'd love to see numbers from other setups: Linux with CUDA GPUs, older Macs, different CPU architectures, cloud instances, etc.
|
||||
|
||||
If you run the benchmark on your hardware, please open an issue or PR with your results and we will add them here. The more data points we have, the better the recommendations get.
|
||||
|
||||
What we are especially interested in:
|
||||
- **NVIDIA GPUs** (RTX 3090, 4090, A100, T4, etc.) with faster-whisper
|
||||
- **Older Apple Silicon** (M1, M2, M3) with mlx-whisper and voxtral-mlx
|
||||
- **Medium and large-v3 models** (we only tested base and small so far)
|
||||
- **Longer audio files** or domain-specific audio (medical, legal, call center)
|
||||
- **Other languages** beyond English and French
|
||||
|
||||
@@ -25,6 +25,7 @@
|
||||
- [WhisperStreaming](https://github.com/ufal/whisper_streaming) (SOTA 2023) - Low latency transcription using [LocalAgreement policy](https://www.isca-archive.org/interspeech_2020/liu20s_interspeech.pdf)
|
||||
- [Streaming Sortformer](https://arxiv.org/abs/2507.18446) (SOTA 2025) - Advanced real-time speaker diarization
|
||||
- [Diart](https://github.com/juanmc2005/diart) (SOTA 2021) - Real-time speaker diarization
|
||||
- [Voxtral Mini](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602) (2025) - 4B-parameter multilingual speech model by Mistral AI
|
||||
- [Silero VAD](https://github.com/snakers4/silero-vad) (2024) - Enterprise-grade Voice Activity Detection
|
||||
|
||||
|
||||
@@ -88,6 +89,7 @@ See **Parameters & Configuration** below on how to use them.
|
||||
</p>
|
||||
|
||||
See **[BENCHMARK.md](BENCHMARK.md)** for the full benchmark with tables, model size comparison, and more.
|
||||
We are actively looking for benchmark results on other hardware (NVIDIA GPUs, different Apple Silicon chips, cloud instances). If you run the benchmarks on your machine, please share your results via an issue or PR!
|
||||
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user