Quentin Fuxa
7108d2ddc5
fixes https://github.com/QuentinFuxa/WhisperLiveKit/issues/269
2025-11-09 20:08:18 +01:00
Quentin Fuxa
a732e0903e
Add a script to detect alignement heads, usefull for distilled whisper
2025-11-09 18:12:09 +01:00
Quentin Fuxa
0491681be4
Distilled model compatibility with HF config.json to ModelDimensions
2025-11-08 20:20:05 +01:00
Quentin Fuxa
ffe5284764
_processing_tasks_done checks task completion
2025-11-05 23:34:00 +01:00
Quentin Fuxa
ece02db6a3
Use optional new separate NLLW package for translation
2025-10-30 19:36:28 +01:00
Quentin Fuxa
65250db92c
tensor to list at the stream end
2025-10-26 16:40:12 +01:00
Quentin Fuxa
416dce7975
fixes #261
...
Co-authored-by: yosagi <11404771+yosagi@users.noreply.github.com >"
2025-10-25 14:20:08 +02:00
Quentin Fuxa
714fb3b14a
custom faster-whisper/mlx whisper encoder available
2025-10-23 20:33:17 +02:00
Quentin Fuxa
0af379c465
DOC: information about file format
2025-10-23 20:32:05 +02:00
Quentin Fuxa
a7db39d999
solves incorrect spacing in buffer diarization
2025-10-02 23:04:00 +02:00
Quentin Fuxa
d080d675a8
cutom alignment heads parameter for custom models
2025-09-27 11:04:00 +02:00
Quentin Fuxa
8cbaeecc75
cutom alignment heads parameter for custom models
2025-09-27 11:04:00 +02:00
google-labs-jules[bot]
70e854b346
feat: Allow loading fine-tuned models in simulstreaming
...
This change modifies the `simulstreaming` backend to support loading fine-tuned Whisper models via the `--model_dir` argument.
The `SimulStreamingASR` class has been updated to:
- Use the `model_dir` path directly to load the model, which is the correct procedure for fine-tuned `.pt` files.
- Automatically disable the `faster-whisper` and `mlx-whisper` fast encoders when `model_dir` is used, as they are not compatible with standard fine-tuned models.
The call site in `core.py` already passed the `model_dir` argument, so no changes were needed there. This change makes the `simulstreaming` backend more flexible and allows users to leverage their own custom models.
2025-09-27 07:29:30 +00:00
Quentin Fuxa
b22478c0b4
correct silences handling when language not auto
2025-09-25 23:20:00 +02:00
Quentin Fuxa
4dd5d8bf8a
translation compatible with auto and detected language
2025-09-22 11:20:00 +02:00
Quentin Fuxa
93f002cafb
language detection after few seconds working
2025-09-20 11:08:00 +02:00
Quentin Fuxa
674b20d3af
in buffer while language not detected »
2025-09-21 11:05:00 +02:00
Quentin Fuxa
a5503308c5
O(n) to O(1) for simulstreaming timestamp determination
2025-09-21 11:04:00 +02:00
Quentin Fuxa
426d70a790
simulstreaming infer does not return a dictionary anymore
2025-09-21 11:03:00 +02:00
Quentin Fuxa
add7ea07ee
translator takes all the tokens from the queue
2025-09-09 19:55:39 +02:00
Quentin Fuxa
3358877054
Fix StorageView conversion for CPU/GPU compatibility
2025-09-09 15:44:16 +02:00
Quentin Fuxa
1f7798c7c1
condition on encoder_feature_ctranslate type
2025-09-09 12:16:52 +02:00
Alexander Lindberg
c7b3bb5e58
Fix regression with faster-whisper encoder_feature
2025-09-09 11:18:55 +03:00
Quentin Fuxa
334b338ab0
use platform to determine system and recommand mlx whisper
2025-09-07 15:49:11 +02:00
notV3NOM
abd8f2c269
Fix exponentially growing simulstreaming silence timer
2025-09-04 21:49:07 +05:30
Quentin Fuxa
f3ad4e39e4
torch.Tensor to torch.as_tensor
2025-09-04 16:39:11 +02:00
Quentin Fuxa
e0a5cbf0e7
v0.1.0 chrome extension
2025-09-04 16:36:28 +02:00
Quentin Fuxa
953697cd86
torch.Tensor to torch.as_tensor
2025-09-04 15:25:39 +02:00
Quentin Fuxa
3bd2122eb4
0.2.8 : only the decoder of whisper is loaded in memory when a different encoder is used
2025-09-02 21:12:25 +02:00
Quentin Fuxa
d5008ed828
mlx/fasterWhisper encoders are loaded once and shared in simulstreaming
2025-09-01 12:33:19 +02:00
Quentin Fuxa
199e21b3ef
faster-whisper as an optional encoder alternative for simulstreaming
2025-08-30 23:50:16 +02:00
Quentin Fuxa
1d926f2e67
mlx-whisper used as simulstreaming encoder: improve speed for macos systems
2025-08-30 22:19:11 +02:00
Quentin Fuxa
4a5d5e1f3b
raise Exception when language == auto and task == translation
2025-08-29 17:44:46 +02:00
Quentin Fuxa
9895bc83bf
auto detection of language for warmup if not indicated
2025-08-27 20:37:48 +02:00
Quentin Fuxa
b32dd8bfc4
Align backend and frontend time handling
2025-08-21 10:33:15 +02:00
Quentin Fuxa
d0e9e37ef6
simulstreaming: cumulative_time_offset to keep timestamps correct when audio > 30s
2025-08-17 09:33:47 +02:00
Quentin Fuxa
820f92d8cb
audio_max_len to 30 -> 20, ffmpeg timeout 5 -> 20
2025-08-17 09:32:08 +02:00
Quentin Fuxa
e2184d5e06
better handle silences when VAC + correct offset issue with whisperstreaming backend
2025-08-17 01:27:07 +02:00
Quentin Fuxa
0f2eba507e
use with_offset to add no audio offset to tokens
2025-08-17 00:33:24 +02:00
Quentin Fuxa
55e08474f3
recycle backend in simulstreaming thanks to new remove hooks function
2025-08-16 23:06:16 +02:00
Quentin Fuxa
1652db9a2d
Use distinct backend models for simulstreaming and add --preloaded_model_count to preload them
2025-08-15 23:03:55 +02:00
Quentin Fuxa
15c3df1cba
warmup base whisper when using simulstreaming
2025-08-12 18:52:52 +02:00
Quentin Fuxa
728e1f1290
simulstreaming warmup is done for each instance of online, not for the backend
2025-08-12 18:35:04 +02:00
Quentin Fuxa
87b9ed6ecd
nonspeech_prob from 1 to 0.5
2025-08-12 18:34:37 +02:00
Quentin Fuxa
d098af3185
each SimulStreamingOnlineProcessor now contains PaddedAlignAttWhisper instance. SimulStreamingASR only contains loaded whisper model
2025-08-11 08:24:14 +02:00
Quentin Fuxa
b678a55f63
remove duplicate file
2025-08-09 23:10:34 +02:00
Quentin Fuxa
5491964e81
clean SimulStreamingOnlineProcessor initialization + audio processing
2025-08-09 20:16:27 +02:00
Quentin Fuxa
b05297a96d
clean simulwhisper backend and online
2025-08-09 18:02:15 +02:00
Quentin Fuxa
197293e25e
refactor(simulstreaming): extract backend + online module into separate files from whisper streaming
2025-08-08 18:07:51 +02:00
Quentin Fuxa
4cfed6e98e
in MultiHeadAttention and ResidualAttentionBlock include cache_id for compatibility with simulstreaming code
2025-08-02 13:16:58 +02:00
