Isolate reinforcement-learning agent rollouts

Reinforcement-learning agents explore. That exploration can include repeated failures, unexpected action sequences, and attempts to exploit weaknesses in the reward function.

When a rollout can execute code or call tools, treat its environment and evaluator as separate responsibilities. Polyaxon can organize the rollout jobs, resource budgets, and trajectory evidence without granting the policy control over its own reward.

Define the episode boundary

An episode should have an explicit environment version, initial state, seed, maximum steps, and termination conditions. Include the action vocabulary and the systems those actions may reach.

For a coding task, provide a fixed repository revision and synthetic fixtures in a fresh workspace. Do not let separate candidates share a mutable working directory, evaluator files, or unrestricted credentials.

Use a job for a finite rollout. If the rollout needs interactive commands, a controller can manage a sandbox-enabled service for the episode and stop it afterward.

Keep reward computation outside the candidate

The policy can produce a patch or trajectory, but a trusted evaluator should calculate the reward. Store its fixtures and implementation separately from the writable candidate workspace.

For example, a code-repair rollout may return a patch, execution receipt, and selected outputs. The evaluator applies the patch to a clean source tree and checks behavior using approved cases.

A reward improvement is not automatically a product improvement. Inspect whether the candidate learned a shortcut, omitted difficult cases, or changed an output format to satisfy a weak checker. Preserve failure examples alongside aggregate scores.

Track the experiment at episode granularity

This example belongs in a trusted rollout harness running with Polyaxon tracking configured. The measured values are illustrative placeholders for the harness's actual results.

from polyaxon import tracking

tracking.init()
tracking.log_inputs(
    environment_revision="code-repair-v3",
    policy_revision="candidate-17",
    evaluator_revision="checks-v5",
    seed=42,
)
tracking.log_metrics(
    episode_reward=0.75,
    episode_steps=12,
    invalid_actions=1,
)
tracking.log_outputs(
    termination_reason="step_budget",
)

Persist the trajectory and detailed evaluator report through artifact tracking. Record the actual termination reason even when a partial reward is available.

A seed alone does not guarantee reproducibility when tools, model providers, or external services change. Preserve those versions and relevant responses under an appropriate data policy.

Bound parallel exploration

Limit per-episode runtime, memory, output size, and action count. Set workflow concurrency and, where available, queue quotas so an exploratory sweep cannot consume all shared capacity.

Keep platform retries distinct from a new episode. A retried rollout may repeat side effects or contaminate measurements if it resumes from an altered environment. Prefer explicit episode identifiers and a clean reconstruction policy.

Promote policies using held-out evidence

Evaluate promising policies against held-out tasks and failure scenarios. Compare reward, task success, invalid actions, runtime, and resource consumption rather than optimizing one score in isolation.

Use run comparisons to inspect policy candidates under the same environment and evaluator revisions.

This makes RL experimentation a controlled Polyaxon workflow: bounded episodes, independent evaluation, and enough trajectory evidence to distinguish real progress from reward exploitation.