Benchmark vLLM speculative decoding on Polyaxon
Compare ordinary decoding with prompt-lookup proposals under the same model, workload, and GPU allocation, including latency, failures, and output checks.
LLMOpsvLLMInference
Latest articles
The latest guides, tutorials, and product updates.
Run batch inference with vLLM on Polyaxon
Generate predictions from a JSONL prompt file with vLLM, schedule the work on a Polyaxon GPU agent, and save the results as run artifacts.
LLMOpsvLLMInference
Serve multiple LoRA adapters with vLLM on Polyaxon
Deploy two task-specific LoRA adapters over one Qwen base model, select them by name, and measure quality and shared GPU capacity with Polyaxon.
LLMOpsvLLMInference
Make vLLM prefix caching work for repeated prompts
Understand exact prefix reuse in vLLM, structure repeated context, account for replica routing, and compare cold and warm workloads with Polyaxon.
LLMOpsInferencevLLM