Cerebras kernels are now 5.10× faster at decode attention across the wafer.
220,273 → 43,165 cycles · Head 64 · context 128 · 32×1 PE · FP32 · simulator cycles, fewer is better · from open PR #1, not merged yet
Decode the way the wafer runs it: grouped-query attention for one new token spread over many PEs, with the KV cache appended in place, a distributed softmax and partial results reduced across the fabric. This is where on-wafer SRAM and the mesh earn their speed.
- Record
Bank DSR vector max newton
43,165 cycles
- Tried
16 attempts
7 approaches
- Next target
5.36× or better
1.05× the record, memory within +10%
Each dot is a validated attempt: its speed against the baseline, by the date it was recorded. Higher is faster.
- Confirmed
- Screened or unconfirmed
- Best confirmed so far
- Baseline (1×)
Why a wafer is fast
A Cerebras wafer is a grid of hundreds of thousands of small processing elements (PEs), each with its own SRAM, wired by an on-chip fabric. Weights live next to the arithmetic instead of in distant DRAM, which is why token-by-token decoding, usually limited by memory bandwidth, runs so fast.
Why the decoder layer leads
Every generated token is one pass through each layer: RMSNorm, QKV projections, RoPE, causal attention over a growing KV cache, then a SwiGLU MLP. The decoder-layer profile is that pass, over four consecutive tokens with every output and cache store timed and checked. Its longer-context version makes attention over the cache count.
How it is scored
Kernels are written in CSL and run in the Cerebras SDK fabric simulator. The score is the most cycles any PE spends (max per-PE local elapsed cycles). Every result must match an independent reference on six cases, and program memory may grow by at most 10%.
What counts as a win
A candidate is screened, fully validated, then confirmed against a fresh baseline run in a paired comparison. Only a confirmed run of at least 1.05× is retained. Screened and unconfirmed numbers are shown, but are not records.
Caveats: these are simulator cycles on small, scaled workloads (the decoder layer runs on one PE), not wall-clock on hardware, and not a full model. Hardware tokens per second remain unmeasured.
- 1
Check the setup (Cerebras SDK in a Linux VM)
tools/sdk-run python3 research.py doctor
- 2
Measure a baseline
tools/sdk-run python3 research.py run decoder.h16-f32-ctx12
- 3
Start an editable CSL candidate
python3 research.py candidate-init decoder.h16-f32-ctx12 my-idea
- 4
Screen, validate and confirm it
tools/sdk-run python3 research.py challenge decoder.h16-f32-ctx12 --variant candidate --candidate my-idea
- Repository
- teddyjfpender/cerebras-autoresearch
- Snapshot
- 386128a756
- Ledger
- 193 journal rows; failures and cancelled runs stay in the repository.