Polyaxon v3 is coming →
DeepSeek V4

Serve DeepSeek V4 on Polyaxon

A model name alone is not a deployment. The DeepSeek V4 recipe binds a specific checkpoint revision to SGLang, precision, parallelism, parser settings, memory policy, and a supported GPU topology; Polyaxon makes that complete recipe operable.

Outcome

A versioned DeepSeek V4 service recipe prepared for an eight-GPU Blackwell node and an evaluation gate before production traffic.

Start the guideRead the SGLang guide

Service responsibilities

Choose a recipe

Select a model revision, serving runtime, precision, context length, and supported accelerator topology.

Polyaxon operates

The component makes the recipe schedulable, repeatable, observable, and governed on Kubernetes.

Evaluate before promotion

Quality, latency, throughput, memory, and failure behavior are checked against the intended workload.

Prerequisites

  • Access to the selected DeepSeek V4 checkpoint and acceptance of its model terms.
  • A validated SGLang image that supports the model and the target Blackwell GPUs.
  • An eight-GPU node, sufficient cache disk and shared memory, and a representative evaluation set.

Deploy and validate

  1. 1

    Define the model and runtime together

    Keep the model revision, SGLang parsers, tensor and data parallelism, and GPU request in the same component revision.

    version: 1.1kind: componentname: deepseek-v4-servicerun:  kind: service  ports: [30000]  rewritePath: true  connections: [hf-token, model-cache]  environment:    nodeSelector:      workload.polyaxon.com/accelerator: b200  container:    image: lmsysorg/sglang:deepseek-v4-validated    command: ["sglang", "serve"]    args: ["--model-path", "deepseek-ai/DeepSeek-V4-Pro", "--tp", "8", "--dp", "8", "--tool-call-parser", "deepseekv4", "--reasoning-parser", "deepseek-v4", "--host", "0.0.0.0", "--port", "30000"]    resources:      limits:        nvidia.com/gpu: "8"

    The node label and image tag are deployment-specific placeholders. Confirm the current model/runtime recipe and pin all revisions before running it.

  2. 2

    Submit the operation

    Run the component through the target Polyaxon project, queue, preset, and approval path.

    polyaxon run -f deepseek-v4.yaml
  3. 3

    Validate reasoning and tool calls

    Use a fixed request set to verify ordinary chat, reasoning fields, tool-call structure, latency, memory headroom, and failure recovery.

    SERVICE_URL=$(polyaxon ops service --external --url)curl "$SERVICE_URL/v1/models"

Production checklist

Confirm the model license, gated-access requirements, revision, tokenizer, chat template, and parser settings.
Size the selected precision and context length on the exact accelerator topology before setting replicas.
Keep model cache, evaluation datasets, and promotion evidence versioned and reproducible.
Treat runtime flags as part of the model release and compare quality and performance before rollout.
Treat reasoning and tool parsers as API behavior: regression-test them alongside model quality and performance.

Troubleshooting

The model fails to load

Confirm access, revision, tokenizer, runtime support, precision, disk capacity, and aggregate accelerator memory.

Responses have the wrong structure

Check chat template, reasoning parser, tool parser, tokenizer version, and client request format.

Results differ after deployment

Compare model revision, runtime image, quantization, sampling parameters, context truncation, and evaluation inputs.

Sources

Official platform, library, model, and Polyaxon references used by this guide.

Continue

Compare AI platformsExplore LLMOps