FlagOS × MiniCPM inference throughput optimization challenge opens: two chips, big prizes

Hundred-billion-parameter models are moving rapidly into industry, and the high compute and memory costs bring high inference latency, low throughput and poor hardware utilization that hold back large-scale deployment. End-to-end inference optimization on a fixed hardware cluster, without sacrificing accuracy or service stability, has become a central technical problem for large-model systems.

Following the ongoing Track 1 SGLang framework kernel optimization challenge, Track 2 of the FlagOS Open Computing Global Challenge (Season 2) — MiniCPM inference throughput optimization — is now open for global registration.

Organized by the FlagOS community and IEEE, co-organized by ModelBest and the OpenBMB open-source community, and specially sponsored by MetaX and Iluvatar CoreX, the track targets the pain points that follow large-scale deployment: high latency, low throughput and poor utilization driven by compute and memory costs. Participants use the FlagOS inference framework to push the throughput limits of MiniCPM5-2B on two chips — Iluvatar CoreX BI-V150 and MetaX Xiyun C500 (64 GB VRAM) — and to turn hardware potential into meaningful gains in speed and utilization.

Whether you are a systems engineer, a university researcher working on large models, or a core open-source developer, you can form a team and sharpen your inference-system optimization plan.

Where is the real challenge? Extreme throughput optimization as a full-stack test

Unlike single-point kernel optimization, this track tests end-to-end inference system optimization. Under the hard constraint that model accuracy must not drop noticeably, contestants combine parallelism strategy, memory management, operator kernels and framework logic to raise MiniCPM5-2B's total tokens/s throughput.

Goal: raise total tokens/s.

Baseline requirement: maintain accuracy.

Permitted optimizations include but are not limited to: memory management (KV cache optimization, dynamic batching) and compute-logic optimization (operator fusion, kernel rewriting) — techniques that require real development work in the framework. Strategies described in the submitted technical report must appear in the submitted code and be used in the final version.

Development framework and operator library:

FlagOS inference framework:

🔗 https://github.com/flagos-ai/vllm-plugin-FL (branch: flagos-2026-s2)

High-performance operator library FlagGems:

🔗 https://github.com/flagos-ai/FlagGems (tag: v5.3.5)

Evaluation hardware: Iluvatar CoreX BI-V150, MetaX Xiyun C500 (64 GB VRAM)

Two chips, one contest: a full-stack optimization showdown

Your optimization is evaluated separately on the Iluvatar CoreX BI-V150 and the MetaX Xiyun C500. Teams with optimizations on both chips must merge the source code for both platforms into one submission.

Evaluation covers a 4k scenario ([4096,1024,64,256]) and a 16k scenario ([16384,1024,64,128]), testing performance across input lengths.

Scoring:

The track offers cash prizes for first, second and third place. Total score = performance score (Iluvatar card) + performance score (MetaX card) + innovation score.

Performance score (70%): the relative improvement. For example, a baseline of 200 improved to 210 is a 5% gain, so the performance score = 100 × 5% × 70%.

Innovation score (30%): expert rating of the technical report's novelty. Innovation score = expert rating × 30%.

*If teams tie on performance improvement, technical novelty is assessed from the submitted technical reports and decided by a vote of the organizers.

Prizes:

🥇 First prize: CNY 30,000

🥈 Second prize: CNY 20,000

🥉 Third prize: CNY 10,000

Official compute support for the technical work

This track uses compute provided centrally by the organizers. To avoid wasting compute, teams must submit application materials within the stated window. Materials must include at least a preliminary technical plan and a team introduction (member résumés and descriptions of relevant project and competition experience). The organizing committee allocates compute based on its overall review.

Iluvatar compute: after approval, book it on the online lab platform (https://flags.io/OnlineLab).

MetaX compute: claim a compute voucher, then redeem it on the Moli Fangzhou platform (https://ai.gitee.com/compute). (Note: vouchers are limited — shut down machines when not developing. To avoid data loss, restart every 36 hours.)

Technical guidance: experts from the Beijing Academy of Artificial Intelligence (BAAI), a member of the FlagOS community, provide guidance throughout the competition.

Key dates

1. Registration: July 19 – November 10, 2026 (UTC+8)

Registration portal: https://flagos.io/race-detail-season2?id=539vlt2p⟨=cn

*Registration must be completed on the official site to be eligible for review and prizes.

3. Official compute application: September 14 – November 10, 2026 (UTC+8)

Submit materials named [team-name.pdf] with a preliminary technical plan and team introduction. The committee reviews within 7 working days; compute is limited and allocated first come, first served.

Iluvatar CoreX BI-V150 compute: book it on OnlineLab after approval.

MetaX Xiyun C500 compute: claim a voucher and redeem it on the Moli Fangzhou platform; restart every 36 hours to avoid data loss.

4. Compute review and allocation: September 21 – November 15, 2026 (UTC+8)

Development and submission: September 21 – November 20, 2026 (UTC+8). Complete development in this window and submit the packaged project on the platform.

6. Judging: early December 2026

7. Results: mid-December 2026

*Any schedule changes will be announced in advance on the FlagOS platform; official notices prevail.

Submission and judging essentials

Teams: up to 3 members per team; solo entries are allowed. The track welcomes participants from companies, universities, research institutes, innovation teams and independent researchers, in China and abroad.

2. Development environment:

Model: /workspace/MiniCPM5-2B

evalscope evaluation dataset: /workspace/evalscope-datasets/math_500

vllm-plugin-FL: /workspace/vllm-plugin-FL

FlagGems: /workspace/FlagGems

Updating vllm-plugin-FL and FlagGems source: pip install --no-build-isolation

3. Submission requirements

Contents: submit a .zip named after the team, containing source code (merge both platforms if you optimized both chips), a technical report and reproduction instructions.

Technical report (report.pdf): explain the design rationale, algorithmic principles, implementation architecture, optimization strategies, experimental environment, test results and analysis; use figures with text, keep the logic clear, and support the solution so judges can follow the overall technical approach and novelty.

Reproduction instructions (readme.md): list the hardware and software environment, dependency versions, deployment steps, build and run flow, dataset preparation, run commands and result verification, complete enough for judges to reproduce the results.

4. Evaluation rules

Accuracy check: re-measured accuracy on the math_500 Level3 dataset must be ≥ 0.95; falling short invalidates the score.

Performance: in the 4k and 16k scenarios, ranking uses the total tokens/s throughput reproduced by the committee.

Prohibited: enabling quantization, speculative sampling, tuning benchmark scripts, or merging upstream framework branches to gain performance, among others; any violation invalidates the score.

Winning PRs: winning teams must submit a GitHub PR within 3 working days after judging (https://github.com/flagos-ai/vllm-plugin-FL/tree/flagos-2026-s2); late submissions forfeit the prize.

*The FlagOS Open Computing Global Challenge organizing committee holds final interpretation rights.

As large-model inference reaches production, teams face high memory costs, limited concurrent throughput, inefficient long-text inference and complex heterogeneous hardware adaptation. Unlocking business value requires both low-level kernel work and coordinated progress in scheduling, memory management and framework logic — a full-stack engineering problem spanning software and hardware.

Season 2 of the FlagOS Open Computing Global Challenge asks developers to take the whole inference system as their unit of analysis, confront real deployment problems and find optimization paths that balance accuracy, latency and throughput. Whether you are an engineer deep in inference systems or a student working on a system-optimization project, form a team, put your optimization plan into practice, validate it and open-source it to help the inference ecosystem move forward.

💡 Where to get help

Competition inquiries: contact@flagos.io

Challenge details: https://flagos.io/race-detail-season2?id=539vlt2p⟨=cn

GitHub repositories:

FlagOS inference framework:

🔗 https://github.com/flagos-ai/vllm-plugin-FL/tree/flagos-2026-s2

FlagGems operator library:

🔗 https://github.com/flagos-ai/FlagGems

MiniCPM5-2B downloads:

Hugging Face:

🔗 https://huggingface.co/openbmb/MiniCPM5-2B

Github:

🔗 https://github.com/OpenBMB/MiniCPM

ModelScope:

🔗 https://modelscope.cn/models/OpenBMB/MiniCPM5-2B

👇 Scan the QR code to join the group for Q&A, technical discussion and official notices