TokenFlow: Responsive LLM Text Streaming Serving under Request Burst via Preemptive Scheduling

Hybrid Models as First-Class Citizens in vLLM

Meet ‘kvcached’: A Machine Learning Library to Enable Virtualized, Elastic KV Cache for LLM Serving on Shared GPUs

Sigmoidal Scaling Curves Make Reinforcement Learning RL Post-Training Predictable for LLMs

2:4 Sparsity + Quantisierung: Der Schlüssel zur effizienten LLM‑Kompression

vLLM: Schnellere und einfachere Bereitstellung großer Sprachmodelle

MoonshotAI Released Checkpoint-Engine: A Simple Middleware to Update Model Weights in LLM Inference Engines, Effective for Reinforcement Learning