SOAR Weekly Champion Notes 01 | Cao Yi: first look at the task, environment setup, and configuration exploration
Editor's note · Opening the column
Every weekly champion is a tribute to the limits of performance.
With the arrival of SOAR 2026's first weekly leaderboard champion, we are officially launching the "SOAR Weekly Champion Notes" technical column. The column aims to build a platform for in-depth technical exchange:
For contestants: We invite each week's front-runner to share their core optimization ideas and real-world pitfall-avoidance guides, hoping to spark inspiration for developers tackling hard problems and raise the community's overall technical level.
For weekly champions: This is a stage to showcase top technical strength. We record every thought process from environment setup to architectural breakthroughs, so excellent solutions and pure geek spirit can reach more people.
Here there is no absolute finish line, only benchmarks that keep being refreshed. In this first installment, let's follow Week 1 champion Cao Yi and break down how he used a weight quantization strategy to break through in the first week's showdown.
About the author: Cao Yi (SlightwindSec) is an LLM inference framework developer and model performance optimization engineer at a domestic computing platform. With more than two years of frontline engineering experience, he works on inference acceleration, low-bit quantization, and other key technical areas.
➤ GitHub link
🔗 https://github.com/SlightwindSec
Below is the author's original share.
SOAR 2026: Week 1
SOAR 2026 is a competition aimed at optimizing the inference performance of the MiniCPM-SALA model, with rankings settled once a week. This article records my experience in the first week (3.1 - 3.5).
01 Competition rules at a glance
The core of the competition is to minimize inference time while maintaining model accuracy. The scoring formula is:
Final score = (S₁×40% + S₈×30% + S_∞×30%) × correctness coefficient C
Here S₁/S₈/S_∞ correspond to the relative latency scores under three stress-test scenarios: single concurrency, 8-way concurrency, and unlimited concurrency. The correctness coefficient C is a stepwise penalty: the model's relative score must exceed 97%, otherwise the total score drops to zero; 97%–99% is multiplied by 0.96, and only above 99% does it receive the full coefficient of 1.0.
There are only 3 submission opportunities per day, and only after each submission do you get real feedback from the evaluation server. Every opportunity is precious.
02 Groundwork: getting the environment up
The first day was mostly infrastructure: I wrote a service script that spins up temporary containers from a Docker image, a local performance test script, and got several attention backends working.
To make local test results more useful, I designed three test datasets covering different usage scenarios:
0.1k-4k: short input, long output, simulating generation-intensive scenarios
0.1k-0.5k: short input, short output
60k-0.5k: long input, short output, simulating long-document Q&A scenarios
This local evaluation setup quickly filters out clearly regressive solutions before submission, avoiding wasted attempts.
03 Early attempts: the parameter-tuning phase
The first few submissions were mostly about tuning SGLang startup parameters, exploring directions such as KV Cache compression and attention backend switching. The baseline submission used the default configuration directly and scored about 94, with full accuracy but no performance optimization.
After enabling KV Cache FP8 quantization, all three performance tiers improved somewhat, but only modestly — the total score landed around 77–78, and since other contestants kept optimizing, the relative score was actually falling.
I also tested an online 8-bit weight quantization scheme, but it performed poorly in the single-concurrency scenario. Possibly because kernel compute time is basically the same at single concurrency, quantization only introduced extra overhead.
04 The turning point: weight quantization
One option remained untried: weight quantization, so I decided to try 4-bit quantization. Sharply compressing weight precision not only reduces VRAM usage but also lowers memory bandwidth demand — for decode-heavy inference tasks, memory bandwidth is often the real bottleneck.
During quantization I ran into some engineering issues with model adaptation and had to make some adjustments to MiniCPM-SALA's code structure.
After quantization, model size dropped from about 18GB to about 5.4GB, a compression ratio close to 3:1. In local tests, single-concurrency latency fell by about 40% versus the previous best configuration — a very significant gain.
After submitting to the evaluation server, the final result:
{
"C": 0.92, "S1": 100, "S8": 99.99, "Smax": 96.39,
"final_score": 91, "performance_score": 98.91
}
The performance score was near perfect (98.91), but the correctness coefficient was only 0.92, indicating some accuracy loss after quantization; the relative score fell in the 97%–98% band.
05 Leaderboard analysis
At the end of the weekly contest, the top of the leaderboard looked roughly like this:
The accuracy coefficients of ranks 2–5 were all 1.0 (that is, almost no accuracy loss), while my C was only 0.92. But my S1 reached 100 (the fastest of the week), and the performance advantage offset the accuracy penalty.
4-bit weight quantization did deliver significant inference speedup, but it also exposed an accuracy bottleneck. By comparison, other teams had already brought their end-to-end latency close to mine while maintaining higher accuracy. That suggests they used optimization strategies I haven't found yet, and I need to review them further after the contest.
Overall, the Week 1 approach was: trade aggressive weight compression for speed, and use the performance score advantage to make up for the accuracy penalty. I won the weekly championship with 91 points. But over the long run, cutting latency while preserving accuracy is the healthier path — after all, a correctness coefficient of 0.92 is not far from the elimination line, and that's a problem to solve.
Acknowledgements: Thanks to the SOAR 2026 organizing committee for the competition platform and to the SGLang team for their open-source contributions. The contest continues, and I'll keep working to achieve better results in the remaining weeks!
🌟 Fellow members of the OpenBMB community:
The SOAR 2026 Sparse Operator Acceleration Grand Prix is in full swing, with new challenges every week and a live leaderboard waiting for you. If you want to show your technical strength in model inference acceleration, operator-level optimization, and quantization, we sincerely invite you to enter the contest, compete alongside Cao Yi and many other outstanding participants, and together push the inference limits of MiniCPM-SALA! Click the image below or visit the contest website https://soar.openbmb.cn/ to learn more.