Give coding agents controlled access to Polyaxon operations

A coding agent can help an ML team investigate failed runs, prepare evaluation components, and turn a working fix into a repeatable operation. Polyaxon gives that assistant concrete objects to work with: projects, run UUIDs, component versions, parameters, logs, metrics, and artifacts.

The integration becomes useful when the assistant follows the same workflow as an engineer. Find the relevant run, inspect its evidence, reproduce the problem in an appropriate environment, propose a change, and launch an approved evaluation. Each stage can return a reference another team member can inspect in Polyaxon.

Teach the agent your Polyaxon project structure

Create a versioned instruction file in the repository that explains which Polyaxon project owns the work, where its Polyaxonfiles live, and which components perform training, evaluation, and reporting. Include links to the operations guide, your approved component versions, and the metrics used for release decisions.

For example, an evaluation assistant needs to know whether accuracy is a classification metric or an evaluator score, which dataset revision produced it, and whether higher values are better. Repository context should explain those meanings. The tool implementation should supply the actual run identifiers and recorded values.

Keep the organization, project, allowed component versions, and sandbox run selection in trusted application configuration. The model can propose a task and arguments within that context. Changing its prompt should not silently expand which workloads it can access.

Start with run discovery and evidence

Polyaxon's CLI query interface can select runs by metadata and metrics. The following example assumes a configured client and a project named agent-evaluations whose jobs report accuracy:

polyaxon ops ls -p agent-evaluations \
  -q "kind: job" \
  -s "-metrics.accuracy" \
  -l 5 -io \
  -c "uuid,out.accuracy"

Use returned UUIDs to open the corresponding runs and inspect their configuration, logs, and artifacts. For an application integration, wrap the run client or CLI behind tools with structured responses. Return explicit fields such as run_uuid, status, metric_name, and artifact_reference, so a model can cite evidence without inferring identifiers from a table.

A first useful assignment is: compare two evaluation runs, identify cases that regressed, and draft a hypothesis. It gives the assistant access to the evidence needed for a decision while leaving the proposed change reviewable.

Reproduce the issue in a sandbox-enabled service

Use the sandbox quick start to prepare an environment with the relevant image, repository, and connections. Wait for the service to be running and check its health before offering execution tools to the agent.

For unattended investigation, expose a tool that performs a specific operation. This example returns the working-tree status of an already prepared repository; the caller cannot replace the command or target project:

import os

from polyaxon.client import SandboxClient


def inspect_workspace():
    with SandboxClient(
        project="agent-evaluations",
        run_uuid=os.environ["RUN_UUID"],
    ) as sandbox:
        sandbox.ping()
        result = sandbox.process.exec(
            command=["git", "-C", "/workspace/repo", "status", "--short"],
            timeout_ms=30_000,
        )
        return {
            "exit_code": result.exit_code,
            "timed_out": result.timed_out,
            "stdout": result.stdout,
            "stderr": result.stderr,
        }

This requires Git and /workspace/repo in the service. Extend the same pattern with an approved diagnostic script or a known evaluation command. Review tool output for secrets and bound its size before returning it to the model.

The Connect LLMs guide covers provider integration and a general command tool with host approval. The sandbox plugin gives access inside the existing service container, so choose its user, mounts, and network access deliberately. Keep the model provider's credentials in the host application when they are only needed there.

Turn proposed changes into explicit operations

After an investigation, have the assistant prepare a Polyaxon operation that references an approved evaluation component. Review its component version, inputs, image, resource configuration, and data connections before submission. Scheduling presets let the platform team package recurring queue, resource, and environment choices.

For a local operation file named operations/evaluate-change.yaml, a reviewer can submit an unapproved run with:

polyaxon run -p agent-evaluations \
  -f operations/evaluate-change.yaml \
  --approved=false

The run command documents this human-in-the-loop setting. Keep the ability to approve and start consequential work with the authorized reviewer or trusted automation. An application that lets the model choose approved=true has made a different authorization decision, regardless of what its prompt says.

Preserve the path from suggestion to result

For each attempted change, retain the source run UUID, repository commit, proposed patch, instruction revision, selected component version, and evaluation run UUID. Store useful reports through Polyaxon artifacts and compare the new run against its baseline in the runs dashboard.

This makes a statement such as “the change fixed the failure” inspectable. A reviewer can see the failing case, the code revision, and the measured result. A generated summary alone does not establish that connection.

When the workflow proves useful, register its stable evaluation or reporting logic in the component hub. The assistant can then select an established platform operation, while engineers maintain its implementation and execution policy in the same place as the rest of their ML workflows.