MASSIVE UltraData Upgrades: Ultra-FineWeb-L3 & UltraData-SFT-2605 Released
MASSIVE upgrades for the UltraData data stack! The tiered data management (L0-L4) framework has now been fully battle-tested on MiniCPM5-1B and is ready for your models.
What's NEW:
Ultra-FineWeb-L3 — 600B tokens (200B+ Chinese, 400B+ English) of high-density synthetic pre-training data, expanded from Ultra-FineWeb via multi-style rewriting & QA generation, used in MiniCPM5-1B's decay stage.
Hugging Face: https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L3
UltraData-SFT-2605 — 15M+ post-training samples across math, code, knowledge & instruction following, with deep-thinking and non-thinking training styles, used in MiniCPM5-1B's SFT stage.
Hugging Face: https://huggingface.co/datasets/openbmb/UltraData-SFT-2605
Explore previews and samples on the UltraData platform. Source: OpenBMB on X.