Build guardrails for AI-generated code

AI-generated code should enter the software lifecycle as an untrusted contribution. A plausible diff may still expose a secret, weaken authorization, select a compromised dependency, modify unrelated files, or pass narrow tests while violating the task.

Useful guardrails do more than filter model output. They control the context the coding system can read, the actions it can take, the changes it can propose, and the evidence required before software is merged or deployed.

Separate code guardrails from model guardrails

A model guardrail evaluates prompts or responses. A code guardrail evaluates a change and the process around it. Both may be needed, but they protect different boundaries.

Do not ask the coding model to enforce its own permissions. Its explanation of why a command is safe is input to a decision, not the decision itself.

Guard the context before generation

Authorize repository and file access using the current user and task. Exclude credential files, production exports, private keys, and unrelated repositories by default. Scan prompts and attachments for secrets before sending them to a model, but assume pattern detection will miss some sensitive data; source-level access control remains the primary boundary.

Treat issue bodies, documentation, comments, test fixtures, and retrieved web pages as untrusted. They can contain text that attempts to redirect an agent. Preserve the source of each context item and keep it distinct from trusted policy.

For every run, record the requested task, allowed path scope, base revision, permitted commands, network policy, time budget, and approver requirements. These fields make later review more reliable than reconstructing intent from a chat transcript.

Restrict actions independently of the model

Expose a small tool surface. Prefer structured file edits, declared build commands, and read-only repository inspection over an unrestricted shell. If a shell is required, run it in an ephemeral sandbox with a constrained filesystem, non-root identity, resource limits, and no ambient cloud or production credentials.

Default-deny network access and allow only required package registries or services. Separate dependency download from arbitrary outbound requests where practical. Require explicit approval before the agent sends a message, opens a pull request in another repository, changes infrastructure, merges, or deploys.

The authorization layer should check action, arguments, path, target repository, environment, and current principal. Recheck after a pause or handoff. A delegated coding task must not gain the receiving agent's broader authority.

Evaluate the patch in layers

Start with deterministic checks that are easy to explain:

1. Reject changes outside the declared file or directory scope.
2. Detect secrets and high-risk generated files.
3. Run formatting, linting, and type checks defined by the repository.
4. Scan source, dependencies, containers, and infrastructure definitions.
5. Run task-specific tests and negative security cases.
6. Require a reviewer appropriate to the affected component and risk.

The repository's own instructions determine the permitted checks; an agent should not install new tools or silently weaken configuration to make a gate pass. A generated test is useful evidence, but it is not independent evidence if the same agent wrote the behavior and the assertion.

NIST's Secure Software Development Framework groups practices around preparing the organization, protecting software, producing well-secured software, and responding to vulnerabilities. Apply those outcomes to AI-assisted changes rather than inventing a lower standard for generated code.

Preserve provenance through the build

Pin dependencies and build from a reviewed revision in a controlled environment. Produce an immutable artifact and record its source, builder, inputs, and digest. Verify the artifact at promotion instead of rebuilding it differently for each environment.

The SLSA specification defines practices for artifact provenance and verification. Provenance cannot prove that generated code is correct, but it can establish which reviewed source and build process produced the deployed artifact.

Keep the AI contribution trace as supplementary evidence: model and tool versions, authorized context, commands, proposed patch, policy decisions, human edits, and approvals. Avoid storing raw sensitive prompts when a redacted record or protected reference is sufficient.

Scale gates to the consequence

Use change classification to decide which approvals apply. Documentation and isolated test fixtures may use normal review. Authentication, authorization, cryptography, data migration, dependency policy, infrastructure, and production workflow changes should receive specialist review and stricter deployment gates.

Check both the intended diff and its semantic effect. A one-line permission change can be riskier than a large generated test file. Conversely, a blanket rule requiring manual security approval for every generated line encourages bypasses without improving judgment.

Define emergency and exception paths with owners, expiry, compensating controls, and an audit record. Never let the agent create or approve its own exception.

Monitor the deployed outcome

After release, connect errors, security events, latency, resource use, and rollback signals to the deployed artifact. Preserve the difference between a guardrail denial, a tool failure, a test failure, and a production regression.

Measure secret exposures prevented, out-of-scope edits, vulnerable dependencies, security findings, reviewer overrides, escaped defects, rollback rate, and legitimate task completion. Also track latency and developer intervention. A guardrail that is routinely disabled is not operating as designed.

Reproduce the workflow with Polyaxon

Use Polyaxon pipelines to separate generation, scanning, testing, evaluation, packaging, and promotion. Record revisions, configurations, metrics, and outcomes with tracking, and preserve reports and provenance as artifacts.

Evaluate the coding agent itself with legitimate tasks and adversarial fixtures, including injected repository content, restricted-file requests, tool misuse, and attempts to bypass approval. The companion guide on evaluating LLM guardrails explains how to report protection alongside false refusals and task success.

The central rule is simple: let the agent propose quickly, but make permissions, verification, approval, and deployment deterministic and independently enforced.