Polyaxon v3 is coming →
Qwen 3.6

Serve Qwen 3.6 on Polyaxon

Qwen 3.6 can be served by multiple runtimes and accelerator families, but the model revision, multimodal mode, context length, parser flags, and memory envelope must travel together. Polyaxon captures that deployment contract as a reusable service.

Outcome

A four-accelerator Qwen 3.6 service recipe that can be specialized for an NVIDIA or AMD queue without forking application logic.

Start the guideRead the SGLang guide

Service responsibilities

Choose a recipe

Select a model revision, serving runtime, precision, context length, and supported accelerator topology.

Polyaxon operates

The component makes the recipe schedulable, repeatable, observable, and governed on Kubernetes.

Evaluate before promotion

Quality, latency, throughput, memory, and failure behavior are checked against the intended workload.

Prerequisites

  • A validated SGLang image for either CUDA or ROCm and the selected Qwen 3.6 revision.
  • Four compatible accelerators with enough aggregate memory for the requested context and modality.
  • A model cache, optional Hugging Face connection, and evaluation prompts covering reasoning and tool use.

Deploy and validate

  1. 1

    Define the Qwen service

    Put hardware selection in the queue or preset and keep the portable model/runtime arguments in the component.

    version: 1.1kind: componentname: qwen-3-6-servicerun:  kind: service  ports: [30000]  rewritePath: true  connections: [model-cache]  container:    image: lmsysorg/sglang:v0.5.18    command: ["sglang", "serve"]    args: ["--model-path", "Qwen/Qwen3.6-27B", "--host", "0.0.0.0", "--port", "30000", "--tp", "4", "--reasoning-parser", "qwen3", "--context-length", "262144"]    resources:      limits:        nvidia.com/gpu: "4"

    For AMD, use the tested ROCm image and request amd.com/gpu through an AMD-specific preset. Do not change only the resource name while keeping a CUDA image.

  2. 2

    Submit the operation

    Run the component through the target Polyaxon project, queue, preset, and approval path.

    polyaxon run -f qwen-3-6.yaml
  3. 3

    Test the selected Qwen mode

    Verify text or multimodal input, reasoning and tool fields, context behavior, memory headroom, and latency with fixed requests.

    SERVICE_URL=$(polyaxon ops service --external --url)curl "$SERVICE_URL/v1/models"

Production checklist

Confirm the model license, gated-access requirements, revision, tokenizer, chat template, and parser settings.
Size the selected precision and context length on the exact accelerator topology before setting replicas.
Keep model cache, evaluation datasets, and promotion evidence versioned and reproducible.
Treat runtime flags as part of the model release and compare quality and performance before rollout.
If the workload is text-only, validate any language-only runtime mode for memory savings before adopting it.

Troubleshooting

The model fails to load

Confirm access, revision, tokenizer, runtime support, precision, disk capacity, and aggregate accelerator memory.

Responses have the wrong structure

Check chat template, reasoning parser, tool parser, tokenizer version, and client request format.

Results differ after deployment

Compare model revision, runtime image, quantization, sampling parameters, context truncation, and evaluation inputs.

Sources

Official platform, library, model, and Polyaxon references used by this guide.

Continue

Compare AI platformsExplore LLMOps