Choose kubectl apply or create

kubectl create and kubectl apply can both submit YAML to the Kubernetes API, but they express different ownership expectations. create asks the API to add an object that does not exist. apply declares fields a manager intends to own and creates or updates the object toward that configuration.

The command is only part of the decision. Teams also need one source of truth, a review workflow, and clear field ownership between users, automation, controllers, and policy.

Use create for one-time creation

kubectl create -f sends a create request. If an object with the same type, namespace, and name already exists, the request fails rather than merging the manifest into it:

kubectl create \
  --context acme-staging \
  --namespace ml-team \
  --filename ./trainer-serviceaccount.yaml

That behavior is useful when existence is unexpected and should stop the workflow. It is also appropriate for certain generated, one-time resources where replacement and update semantics are handled elsewhere.

Imperative generators such as kubectl create secret or kubectl create configmap can help produce initial manifests, but sending sensitive values on a command line can expose them through history or process inspection. Use an approved secret workflow instead of pasting credentials into a terminal.

Use apply for declarative management

kubectl apply creates an object when it is absent and updates managed fields when it exists:

kubectl apply \
  --context acme-staging \
  --namespace ml-team \
  --filename ./trainer-deployment.yaml

This is a natural fit for version-controlled configuration that is reviewed and reconciled repeatedly. The Kubernetes declarative-management guide recommends configuration files as the source for create and update operations.

Declarative does not mean “safe regardless of input.” An incorrect desired state can be applied consistently. Review diffs, policy, ownership conflicts, and rollout effects.

Preview the server's interpretation

Client-side parsing cannot show admission defaults, mutations, or all schema behavior. Use server-side dry run or kubectl diff against the intended cluster:

kubectl apply \
  --context acme-staging \
  --namespace ml-team \
  --filename ./trainer-deployment.yaml \
  --dry-run=server \
  --output yaml

kubectl diff \
  --context acme-staging \
  --namespace ml-team \
  --filename ./trainer-deployment.yaml

These commands require API permissions because the server evaluates the request. Treat the rendered output as potentially sensitive: admission may add references and configuration that should not be copied into public logs.

Understand client-side apply

Traditional client-side apply records the last applied configuration in an annotation and calculates a merge using the local file, that annotation, and the live object. It works well for established workflows but can become difficult when several tools edit the same fields or objects were not originally created for apply.

Avoid mixing create, ad hoc edit, patches, Helm, operators, GitOps reconcilers, and client-side apply on the same fields without a documented ownership model. A successful command does not prove another controller will keep the change.

Use server-side apply for explicit field ownership

Server-Side Apply lets the API server track which manager owns individual fields. A manager declares intent with an apply patch; conflicting changes can be surfaced instead of silently merged.

kubectl apply \
  --context acme-staging \
  --namespace ml-team \
  --filename ./trainer-deployment.yaml \
  --server-side \
  --field-manager=ml-platform-config

Choose a stable field-manager name for each automation path. Read conflicts before using --force-conflicts; forcing transfers field ownership and can override another manager's declared intent. The Kubernetes API concepts explain managed fields and conflicts.

Keep generated resources under their owner

A Pod generated by a Deployment, Job, operator, or Polyaxon operation is not the right layer for durable changes. Its owner will recreate it from a template. Apply a change to the declaring resource or the higher-level platform definition.

Likewise, do not apply exported live YAML as a new source file without removing status, server-populated metadata, and fields owned by other controllers. Live output is evidence, not automatically a maintainable declaration.

Make automation idempotent and reviewable

A production workflow should:

1. render the intended resources deterministically;
2. select the cluster and namespace explicitly;
3. validate policy and server-side schema;
4. display a reviewable diff;
5. apply with a known field manager;
6. observe controller rollout and workload health;
7. preserve the source revision and audit result.

Avoid apply --prune until object-set membership and deletion scope are carefully designed. Explicit deletion from reviewed configuration is easier to reason about than a broad selector or directory assumption.

Let Polyaxon own ML operations

Polyaxon compiles ML components and operations into Kubernetes resources while preserving parameters, connections, scheduling policy, lineage, and execution status. Change the Polyaxon component, operation, or platform preset rather than applying edits directly to generated Pods.

Use create when duplicate existence should be an error. Use apply when a declared configuration is the source of truth. In both cases, the decisive question is who owns the fields after the request succeeds.