Separate agent planning, execution, and evaluation

An agent that plans, executes, and grades its own work inside one unrestricted process is difficult to operate. When the answer is wrong, it is unclear whether the plan was unsuitable, the tool failed, or the evaluator accepted weak evidence.

Separate those responsibilities. Polyaxon can run and track each stage while your application defines how information and authority move between them.

Make the plan an inspectable proposal

For an assistant investigating model regressions, a plan might propose comparing two dataset revisions, inspecting failed examples, and running a candidate preprocessing change.

Represent that proposal as structured application data. Include the permitted tool, input references, expected output, and estimated execution budget for each step. Do not let the plan supply credentials or unrestricted cluster configuration.

Store the plan revision with the controller's run. If the model revises its approach after a failed experiment, preserve the earlier proposal rather than overwriting it.

Give execution a narrower contract

The execution adapter checks the proposal against policy and selects a reviewed Polyaxon component. Fixed analyses can run as jobs. Interactive coding can use a sandbox-enabled service.

The executor receives only the inputs necessary for that step. It returns an execution receipt and artifact references, not an instruction to expand its own privileges.

This separation matters when tool output contains untrusted text. A traceback, retrieved document, or generated README may suggest another command, but the controller must authorize the next action independently.

Evaluate against an external expectation

An evaluator should compare the result with a criterion defined before the candidate output is known. For the preprocessing example, check required schema, deterministic fixture outputs, and the quality change on an approved evaluation dataset.

Use tracking metrics for numerical results and artifacts for detailed findings. A successful job status indicates that its process completed; it does not automatically mean the candidate meets a release threshold.

The evaluator's implementation belongs in reviewed code. If it uses a model judge, preserve the judge configuration and include deterministic checks where possible. Avoid letting the candidate modify the fixtures or evaluation logic.

Choose fixed and dynamic orchestration intentionally

A known sequence—prepare data, run candidate, evaluate, produce report—fits a predefined workflow. An agent that decides which experiment to try next needs an application-owned control loop.

Both can use Polyaxon workloads. The dynamic controller should still impose maximum steps, cumulative runtime, and allowed transitions. A new observation can change the plan without changing the security policy.

Dependencies also need explicit data transfer. Ordering two jobs does not automatically copy the first job's workspace into the second. Persist approved outputs and configure the downstream input path deliberately.

Keep promotion outside exploration

A useful candidate becomes a release only after the team's approval process. Keep deployment credentials away from exploratory executors and record the reviewed artifact revision in the promotion decision.

For a failed attempt, preserve enough evidence to explain the failure without retaining unnecessary private data. For an accepted attempt, capture the component, image, dataset, and evaluator versions needed to reproduce the result.

This architecture makes autonomy easier to reason about. The planner explores options, the executor performs authorized work, and the evaluator measures the outcome. Polyaxon supplies the common execution and tracking context that connects them.