Priya Raman
@priya-infersRegular
Inference engineer. Writes about what quantization costs you.
- Quantization
- Inference
- Long context
Bengaluru, India · English, Hindi, Tamil · Asia/Kolkata
Pinned
Q4 vs Q8 on needle-in-a-haystack: a 60-run studyOct 21, 2026ArticleIndependently reproduced
- Establishes
- Another member re-ran it and matched the reported results.
- Doesn’t establish
- Does not establish results on other hardware.
- Disclosure
- Independent work
Runs
18 verified · 3 on BenchClub cloud
ModelWhereSpeed
gemma-3-12b-it Q4_K_MNov 5, 2026RTX 4070 Ti SUPER · 16 GB65.5 tok/s
Qwen3.5-2B Q4_K_MOct 31, 2026Apple M2 Pro · 32 GB89.6 tok/s
Phi-3.5-mini-instruct Q4_K_MOct 27, 2026RTX 4070 Ti SUPER · 16 GB190.7 tok/s- Qwen3-4B Q4_K_MOct 22, 2026Apple M2 Pro · 32 GB51.8 tok/s
- Qwen3-4B Q4_K_MOct 18, 2026L40S · 48 GBCloud247.9 tok/s
- Qwen3-4B Q4_K_MOct 16, 2026RTX 4070 Ti SUPER · 16 GBNot counted
- gemma-3-12b-it Q4_K_MOct 15, 2026L40S · 48 GBCloud79.9 tok/s
gpt-oss-20b MXFP4Oct 12, 2026Apple M2 Pro · 32 GB35.1 tok/s
PassedFailedNot supported
Articles
Your 8k context is not 8kSep 30, 2026ArticleSource linked
- Establishes
- The link points to this creator's published work.
- Doesn’t establish
- Does not establish the numbers in it.
- Disclosure
- Independent work
Accounts, metrics and works carry separate evidence. Updated Nov 5, 2026. Public JSON