Ultra-FineWeb-L1 Is Here: 1T+ Tokens of High-Quality Web Data
🔥 Ultra-FineWeb-L1 is here —— 1T+ tokens of high-quality web data are now available!
High-quality data is the foundation of powerful LLMs. Ultra-FineWeb-L1 is now open-sourced as part of the UltraData ecosystem, providing a carefully processed English web corpus from Common Crawl.
Built for Better Data Quality
1️⃣ Fresh Web Data: Collected from recent Common Crawl snapshots, covering data up to CC-MAIN-2025-51, with 1T+ tokens and ~1.14B documents.
2️⃣ Refined Processing Pipeline: Combines text extraction, language filtering, heuristic filtering, sensitive-field replacement, deduplication, and customized cleaning to improve data quality.
⚡ Highlights
1T+ tokens of filtered English web data
Enhanced cleaning for noisy web content, encoding issues, and abnormal documents
Designed as the L1 filtered layer in the UltraData management framework
Aligned with the latest Ultra-FineWeb updates
🔗 Resources
🤗 Dataset: Hugging Face
📄 Paper: arXiv
🧩 Classifier: UltraData