7 Commits

Author	SHA1	Message	Date
Quentin Fuxa	cf6c49f502	Ruff lint cleanup	2026-01-03 10:23:00 +01:00
Quentin Fuxa	d1fe932241	Apply DRY method v0 - to try to catch and resolve infinite loops such as in #338	2026-03-03 22:52:00 +01:00
Quentin Fuxa	7f3a3df620	simulstreaming mlx & torch dedup of common base	2025-02-15 23:52:00 +01:00
Quentin Fuxa	8c799fa4d1	fix simulstreaming vram leak: cap cross-attn accumulation + token budget ... fixes #283, fixes #275 - accumulated_cross_attns was growing unboundedly during decoding loop, using up to ~5GB for repetition loops. now capped to rolling window of 16 - max_tokens_per_chunk was using TOKENS_PER_SECOND (mel frame rate = 50) instead of actual text token rate (~15/s), allowing 10-40x too many decoding steps - removed unused torch.cat on early return path - removed dead self.committed/last_result_tokens lists (never read) - same fixes applied to mlx variant	2026-02-11 22:10:00 +01:00
Quentin Fuxa	719e8b1a20	adapt online for mlx detection	2024-11-25 23:52:00 +01:00
Quentin Fuxa	6fc20b9562	new dec class	2024-11-21 23:52:00 +01:00
Quentin Fuxa	fac8659161	uses native mlx function for attention	2024-11-21 23:52:00 +01:00