#Llm-Inference
3 notas · última:
The efficient frontier of LLM inference
The article distinguishes techniques that trade latency for throughput or quality for speed from those that expand the whole serving frontier, then situates quantization, parallelism, kernels, speculative decoding, and disaggregation on that map.
Celeris-1's Diffusion Bet on Speed
Celeris-1 swaps autoregressive generation for diffusion, claiming 15x faster responses than GPT-5-mini at a slightly lower MMLU-Pro score.
tokens are getting more expensive
The post argues that cheaper models do not fix AI margins because demand shifts to the newest frontier model and agents now consume far more tokens, making flat-rate subscriptions unsustainable.