I Vibe-Coded a Triton Kernel

The author says GPT-OSS is hard to fine-tune because the recommended setup de-quantizes MXFP4 weights to BF16 and the attention design rules out Flash Attention and PyTorch SDPA, leaving eager attention as the fallback. He then tries to turn the release’s forward-only Triton attention kernel into one that works for training.

Reading notes

#

• He says his kernel experience is limited, but he still managed to produce a correct-looking Triton kernel for GPT-OSS attention.
• He treats correctness as the main goal and builds tests around the kernel and a PyTorch reference.
• Cursor CLI generated a backward kernel and some tests, which he then used as a starting point.
• A notebook-based test setup failed because running hundreds of tests produced too much console output.
• He switched to a Modal app that loads the kernel and tests, runs them on an H100, and saves results as JSON.
• The tests failed immediately, including some forward tests, and the first issue was invalid memory accesses.
• He used a loop of pasting failing tests into ChatGPT, pasting rewritten kernels back into the IDE, and rerunning tests.
• That approach fixed the simple memory-access problems, but the backward pass still disagreed with the PyTorch reference.
• He then let Claude Code work autonomously on the repo with a prompt that pointed it to the Modal test command and the failing test case.
• Claude improved the implementation but still failed on query-padding cases.
• He handed the same task to GPT-5 through Codex CLI after Claude summarized the remaining issues in BUG.md.
• GPT-5 eventually got the tests to pass after some back-and-forth about running Modal tests.
• He ends by suggesting the episode makes recursive self-improvement feel more plausible, while saying he has doubts about that broader narrative.