Run Polyaxon AI workloads from GitHub Actions

GitHub Actions can validate a workload definition and submit it to Polyaxon whenever code, configuration, or an approved release changes. Polyaxon then schedules and tracks the AI, ML, or agent workload on the appropriate Kubernetes environment.

This separation keeps CI responsive. A GitHub runner verifies source and initiates work; it does not need to hold a GPU session open for hours or become the system of record for checkpoints, metrics, evaluations, and artifacts.

Separate CI from workload execution

Use GitHub Actions for:

• repository events, review gates, and environment approvals;
• static checks on components and operations;
• building, scanning, signing, and publishing immutable images;
• selecting an approved Polyaxon operation and parameters;
• recording the source revision and workflow identity;
• submitting or promoting work after policy passes.

Use Polyaxon for:

• queueing and scheduling CPU or GPU workloads;
• applying presets, connections, and execution policy;
• running distributed, mapped, scheduled, or long-lived operations;
• tracking parameters, logs, metrics, lineage, and artifacts;
• comparing runs and promoting accepted outputs.

The CNCF article on extending GitOps separates artifact creation from declarative deployment reconciliation. Apply the same discipline to ML: build immutable inputs, review desired configuration, and let the owning control plane reconcile execution.

Choose events and gates deliberately

Do not launch expensive training on every branch push by default. Match the event to the cost and risk:

Use GitHub environments and required reviewers for production changes. Limit concurrency where a newer commit makes an older pending run obsolete, but do not cancel workloads whose external effects cannot be safely interrupted.

Submit a Polyaxon operation

The Polyaxon GitHub Action integration uses the Polyaxon CLI to check definitions and submit operations. A compact workflow can look like this:

name: Submit Polyaxon operation

on:
  workflow_dispatch:
  push:
    branches: [main]
    paths:
      - "operations/**"
      - "src/**"

permissions:
  contents: read

jobs:
  submit:
    runs-on: ubuntu-latest
    container: polyaxon/polyaxon-cli:1.x.x
    steps:
      - uses: actions/checkout@v4

      - name: Validate operation
        run: polyaxon check -f operations/train.yaml --lint

      - name: Configure and submit
        env:
          POLYAXON_HOST: ${{ secrets.POLYAXON_HOST }}
          POLYAXON_TOKEN: ${{ secrets.POLYAXON_TOKEN }}
          POLYAXON_PROJECT: acme/foundation-model
        run: |
          polyaxon config --host="$POLYAXON_HOST"
          polyaxon login -t "$POLYAXON_TOKEN"
          polyaxon run \
            --project "$POLYAXON_PROJECT" \
            --file operations/train.yaml \
            --description "GitHub Actions run $GITHUB_RUN_ID" \
            --tags "github-actions,sha-$GITHUB_SHA"

Pin the CLI image to an approved version or digest in a production workflow. Apply the organization's policy for pinning actions, and update them through reviewed automation.

The workflow submits a versioned operation. The operation file should define or reference the component, inputs, environment, resources, connections, termination behavior, and output contract rather than hiding those decisions in shell commands.

Protect credentials and permissions

Use a dedicated Polyaxon service account or token with the minimum organization and project permissions needed to validate or submit the intended operation.

Store credentials in GitHub environment secrets, restrict which branches and reviewers can access them, and avoid exposing them to workflows from untrusted forks. Keep job-level GitHub permissions narrow; contents: read is sufficient for checkout unless another step has a justified need.

GitHub's OIDC documentation describes short-lived identity for supported cloud providers. Use it for registry, storage, or cloud access where available instead of static cloud keys. Polyaxon authentication still follows the mechanism supported by the deployment.

Do not pass production dataset or model credentials through GitHub if Polyaxon can attach an approved connection at execution time.

Preserve immutable workload inputs

Build the container once and submit its digest, not a mutable latest tag. Record:

• Git commit SHA and repository;
• workflow name, run ID, attempt, and triggering actor;
• component and operation versions;
• image digest and build provenance;
• dataset, model, prompt, or evaluation-set versions;
• parameters, queue, preset, and target environment.

GitHub context explains why automation initiated the work. Polyaxon run metadata explains what actually executed and what it produced.

Keep high-cardinality identifiers in run metadata and logs rather than infrastructure metric labels.

Make retries and concurrency safe

Rerunning a GitHub job can submit a second Polyaxon operation. Decide whether this is a new experiment, a retry of the same logical request, or an accidental duplicate.

Carry a stable correlation key based on repository, workflow, source revision, and intended workload. Before resubmission, inspect whether a matching run is pending, running, succeeded, or failed. Do not blindly deduplicate work where repeated trials are intentional.

Agent workloads require particular care. Retrying a tool-using agent can repeat messages, writes, purchases, or deployment changes. Persist task state and use idempotency keys at the application boundary; infrastructure retry policy cannot infer whether a side effect is safe.

Keep GitOps and run orchestration distinct

GitOps is a strong fit for long-lived platform and service state: Polyaxon installation, agents, policies, observability components, and model-serving deployments.

Experiment and evaluation runs are often event-driven and finite. Represent their reusable definition in version control, then let Polyaxon create a distinct tracked operation for each execution.

Do not commit generated run status, logs, or metrics back into the desired-state repository. Link the Git revision to the Polyaxon run instead.

Close the loop with Polyaxon

The operations quick start shows how components and operations become executable runs. Polyaxon queues and scheduling presets separate user intent from cluster-specific capacity and placement policy.

Report the Polyaxon run URL or identifier in the GitHub job summary so reviewers can follow execution without keeping the CI job open. Promotion should depend on explicit acceptance criteria such as evaluation thresholds, security policy, and reviewer approval—not only a successful container exit.

GitHub Actions and Polyaxon complement each other when each owns the right lifecycle: GitHub governs source-driven automation, while Polyaxon governs reproducible AI execution and its operational evidence.