Ultra-FineWeb-en-v1.4 is released! 2.2T tokens fully open-sourced!
As the core training fuel for MiniCPM4 / 4.1, Ultra-FineWeb-en-v1.4 has been fully updated based on FineWeb v1.4.0, bringing several important improvements.
What's New:
Fresher Data: Added CommonCrawl snapshots from April 2024 to June 2025 to capture the latest world knowledge.
Easier Access: CC Dump Slices are here! No need to download the entire massive dataset anymore, fetch exactly what you need seamlessly.
Highlights & Performance:
Comprehensive quality filtering process ensures high data quality and relevance. Validated data filtering strategy significantly improves model training effectiveness. Supports large-scale pre-training, providing strong support for the MiniCPM4 series models. Fully open-sourced for easy use by researchers and developers.
Ultra-FineWeb-en-v1.4 represents our latest achievement in high-quality pre-training data, laying a solid foundation for building more powerful language models.