Package the evidence behind a model release

Consider a visual inspection model approved for staging. Before putting it on production line B, the team adds a requirement: performance must be checked separately on that line. The existing report contains only aggregate metrics, but the pipeline retained each image's model score, label, and production-line identifier.

The model can stay fixed. Its saved predictions can supply the new slice report. The release still needs a decision against the new requirement.

This case exposes three separate dependencies: what produced the predictions, what produced the report, and what authorized the release. Keeping those records separate lets a team reuse expensive computation while revisiting the conclusions affected by a change.

Give the release an exact subject

Call the candidate m42. Its package includes the weights, image preprocessing, class mapping, decision threshold, loader configuration, and expected input schema. The release also identifies the serving image by digest. A change to any of those can change the system that reaches users.

Create a manifest of the required files, their relative paths, sizes, and SHA-256 digests. Define how to handle symlinks and external downloads; required content should be resolved into an immutable snapshot or separately identified dependency. Hash the saved manifest and retain the producing run, code revision, training-data references, and environment.

A digest identifies bytes. Attribution also needs a trustworthy record of who produced them and how. SLSA build provenance formalizes that distinction for software builds through artifact subjects, resolved dependencies, and builder identity. A model release record can link such provenance alongside its evaluation evidence.

Resolve references before review. Names such as best-model, latest, or production are useful for navigation; the release record needs the exact package and runtime they resolved to.

Package the evidence a reviewer will need

For the proposed release inspection-r15, retain these records together. Large artifacts can remain in their existing stores if the release manifest identifies their immutable locations and digests.

State which baseline the comparison uses. If labels or scoring rules changed, evaluate the baseline under the same definitions before interpreting a difference. Keep missing cases and failed candidates visible; the candidate-completeness guide provides a concrete reconciliation method.

In this example, inspection-r15 begins as pending because the required line-B report is absent. Successful training and inference runs do not satisfy that missing requirement. The existing approval for inspection-r14 remains a historical record of its own evidence and scope.

Recompute from the first affected dependency

The saved predictions p42 contain a score and input identity for every expected case. Their validity depends on m42, the image snapshot, preprocessing, inference implementation, numerical settings, and any randomness relevant to the measurement.

The report has additional dependencies: labels, slice membership, metric definitions, and the scoring implementation. The release decision adds policy, destination, current authorization, and freshness requirements.

For line B, first verify that the saved cases cover the newly required population and retain trustworthy line identifiers. Then calculate its metrics and sample counts from those predictions. Apply the policy's coverage and uncertainty rules. If coverage is insufficient, collect the missing evidence and keep the release pending. Filtering an old test set cannot establish performance on a population it never represented.

Fix the required slices and acceptance rules before computing the new report. If you tune thresholds or choose favorable slices after inspecting these predictions, confirm the resulting policy on an untouched evaluation set.

Use the same dependency reasoning for other changes:

The label-correction walkthrough covers rescoring in detail. The same boundary matters for training: correcting training labels may require retraining, even when a held-out-label correction would only require rescoring.

Retained predictions are one observation of the model. Reusing them does not create an independent repetition or a new latency sample. When the experiment requires either, run the measurement again.

Make reuse explicit in Polyaxon

Polyaxon's operation cache can reuse workflow results. Supply content identities as declared component inputs so the cache can distinguish the actual dependencies. A storage path or lineage entry alone does not make changes to its contents part of cache matching.

This operation fragment assumes a registered prediction component with three string inputs. The component must load the identified artifacts, verify them, and publish validated predictions:

name: predict-fixed-inspection-images
hubRef: YOUR_ORG/predict-inspection:v4

params:
  package_manifest_sha256: REPLACE_WITH_PACKAGE_MANIFEST_DIGEST
  image_manifest_sha256: REPLACE_WITH_IMAGE_MANIFEST_DIGEST
  inference_spec_sha256: REPLACE_WITH_INFERENCE_SPEC_DIGEST

cache:
  disable: false
  ttl: 86400

The inference specification should resolve the execution image, preprocessing, inference parameters, and relevant runtime assumptions. Keep the registered component revision fixed and pass real digests. The reader supplies the component and artifact-resolution code; this fragment does not implement inference or verify content.

The cache settings control reuse, including its lifetime. A TTL bounds age; it cannot detect a changed dependency omitted from the inputs. Keep label, slice, and scoring identities on the scoring operation. Keep policy and scope on the release operation.

Set cache.disable: true on the release-review operation so each request checks current evidence availability, authorization, expiry, and revocation. Also disable caching for steps measuring current serving performance or scanning against changing external intelligence. Application code performs those checks; fresh execution alone does not establish that they passed.

Retain the bundle with its run

Write an application-owned release.json that contains the references above, including the original producing runs for reused outputs. Record whether each step executed or reused evidence. A cached prediction should remain attributable to the run that produced it.

This helper saves that record in the current Polyaxon run's outputs and logs its digest and input references. Supply input_refs from the validated manifest: each named input has a retained uri and verified sha256, such as the package manifest, evaluation report, and policy. Call the helper between tracking.init() and tracking.end() in an existing packaging job:

import hashlib
import json
from pathlib import Path

from polyaxon import tracking


def record_release_evidence(evidence: dict, input_refs: dict) -> str:
    payload = json.dumps(
        evidence, sort_keys=True, indent=2, allow_nan=False,
    ).encode("utf-8")
    digest = hashlib.sha256(payload).hexdigest()
    path = Path(tracking.get_outputs_path("release/release.json"))
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(payload)
    for name, ref in input_refs.items():
        tracking.log_file_ref(
            name=name, path=ref["uri"], hash=ref["sha256"], is_input=True,
        )
    tracking.log_file_ref(
        name="release-evidence", path=str(path), hash=digest,
    )
    tracking.log_outputs(release_evidence_sha256=digest)
    return digest

The digest covers the exact saved JSON bytes. The helper records evidence supplied by your application; it does not validate reports, sign an approval, or perform registration.

Use the lineage view to connect the packaging run with its inputs, and run comparisons to inspect the old and new scoring runs. Keep input artifact references as well as the manifest so reviewers can follow the dependency chain.

Reference logging records where evidence lives. Retain the underlying bytes through the configured artifact store and verify that the release consumer can retrieve them. Coordinate retention across the package, predictions, reports, policy, and decision. Missing evidence is a release blocker or an explicitly reviewed exception; it should never disappear from the report as an empty successful result.

Register the reviewed package and verify the handoff

After the required checks and scoped approval, register a model version linked to the producing run and its named model artifacts. Attach the release-evidence digest and packaging-run reference in the version metadata. Keep registration separate from prediction workers, which have no reason to hold release credentials.

Use a unique version name and preserve the reviewed package snapshot. A registry stage is useful lifecycle metadata; your release process must enforce the authorization and storage rules behind it.

At deployment, resolve and verify the approved package and serving-image digests, destination, and applicable decision. Record what the service actually loaded. Preserve a rollback reference and recheck its current eligibility before using it. Model trust and artifact scanning cover the surrounding controls.

For the next release, give a reviewer the bundle and ask them to retrieve the package, reproduce one reported slice, and identify the policy and scope behind the decision. Any missing link is a specific handoff defect to repair before promotion.