AIHugging Face Blog2022-05-02Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel
AIHugging Face Blog2020-02-14How to train a new language model from scratch using Transformers and Tokenizers
AIHugging Face Blog2024-03-20Cosmopedia: how to create large-scale synthetic data for pre-training Large Language Models