Faster Text Generation with Self-Speculative Decoding

Hugging Face Blog · 2024-11-20

Chinese Original

Related items

AIHugging Face Blog2024-10-08

Faster Assisted Generation with Dynamic Speculation

AIHugging Face Blog2022-07-27

Faster Text Generation with TensorFlow and XLA

AIHugging Face Blog2024-10-29

Universal Assisted Generation: Faster Decoding with Any Assistant Model

AIHugging Face Blog2023-12-20

Speculative Decoding for 2x Faster Whisper Inference

AIHugging Face Blog2024-01-30

Accelerate StarCoder with 🤗 Optimum Intel on Xeon: Q8/Q4 and Speculative Decoding

AIHugging Face Blog2024-05-01

Powerful ASR + diarization + speculative decoding with Hugging Face Inference Endpoints

Feedback

TypeMessage