Unlocking Longer Generation with Key-Value Cache Quantization

Hugging Face Blog · 2024-05-16

Chinese Original

Related items

AIHugging Face Blog2024-10-29

Universal Assisted Generation: Faster Decoding with Any Assistant Model

AIHugging Face Blog2024-11-20

Faster Text Generation with Self-Speculative Decoding

AIHugging Face Blog2023-05-24

Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA

AIHugging Face Blog2022-07-27

Faster Text Generation with TensorFlow and XLA

AIHugging Face Blog2024-12-23

Controlling Language Model Generation with NVIDIA's LogitsProcessorZoo

AIHugging Face Blog2023-07-27

Stable Diffusion XL on Mac with Advanced Core ML Quantization

Feedback

TypeMessage