Verified run· 1 of 2 tasks regraded from the event log; coding tasks not regraded
gemma-3-27b-it Q4_K_M
suite-standard@1.0 · run by Li Wei · Nov 1, 2026
- Pass rate
- 67%
- 4 of 6 checks
- Speed
- 19.7 tok/s
- decode, short prompt
- At 8k context
- 15.4 tok/s
- decode
- First token
- 79.2 ms
- short prompt
Tasks
Each task runs 3 times and is graded by a program, not by another model.
Codingcode-intervals
Writes a Python function. Unit tests decide the pass.
2/3Not regradedTool usetool-flight
Calls a flight-search tool. The arguments have to be right.
—Not supportedLong contextneedle-8k
Finds one fact hidden in about 8K tokens of text.
2/3Regraded from log
A community run on the creator’s own machine. It counts toward their profile, not toward headline numbers.
PassedFailedNot supported
Against BenchClub cloud
The same model and suite on the pinned NVIDIA L40S · 48 GB, across 6 cloud runs.
Pass rate
This run67%Cloud86%
Speed
This run19.7 tok/sCloud36.2 tok/s
0.54× the cloud’s speed