Prevent data leakage in ML and LLM evaluation datasets

A support-routing model performs well on your evaluation set. Then you inspect its mistakes and discover that training and evaluation contain different messages from the same tickets. Some evaluation questions are also copies of training examples with new identifiers. The score may describe how well the model recognizes familiar cases, while your application needs it to handle new ones.

Leakage starts before the training command. It can enter through a split, a feature built from future information, a preprocessing step, or an evaluation set that gradually becomes part of prompt development. Research on leakage in ML-based science shows why the boundary between development and evaluation deserves explicit review.

This walkthrough builds a small support-case split audit. It preserves related examples as groups, compares a deliberately leaky control with a grouped assignment, and records the manifest behind the result. The same reasoning applies to fine-tuning datasets and application-level LLM evaluations, with additional boundaries around prompts and retrieval.

Decide what an unseen example means

Write the intended evaluation claim before choosing a splitting function. “New support cases from existing customers” and “cases from customers we have never served” are different claims.

Grouped and time-ordered evaluation answer different questions; scikit-learn's cross-validation guide describes both. A grouped split does not automatically make an experiment valid for future traffic. A chronological split does not automatically separate customers or document families.

For this example, the claim is narrower: route a new support case from the same general population. We keep each case and its known copies together. We do not claim to evaluate unseen customers or future operating conditions.

Preserve relationships before splitting

Imagine this small fragment of the dataset:

Unique row IDs do not make these four records independent. Grouping only by case ID also misses the reimport.

Build a relationship record before assigning partitions. Known parent identifiers connect messages, summaries, augmentations, and synthetic questions to their source. Content checks can expose copies whose identifiers changed. In our fixture, rows connect when they share a case ID or an identical normalized-text fingerprint. All records connected through either rule form one group, including indirect connections.

The normalization rule applies Unicode NFKC normalization, case folding, and whitespace collapsing before computing SHA-256. This catches the fixture's formatting changes. It does not detect arbitrary paraphrases, translations, or partial copies. The research on deduplicating language-model training data provides a broader motivation for auditing duplicate content and train–evaluation overlap.

Treat broader similarity matches as candidates for review. A shared greeting or template does not necessarily mean two cases are the same. Inspect unexpectedly large connected groups: a permissive rule can join many unrelated records. Record which relationships were accepted, which copies were removed, and how any conflicting labels were resolved.

For new augmentation work, assign the parent first and let its derivatives inherit that assignment. Do not generate several views of every case and then independently shuffle them. If a final benchmark already exists, preserve its membership and apply a documented exclusion policy to development data instead of silently rebuilding the benchmark around the latest results.

Run the split-audit example

The companion example uses synthetic text, Python 3.11 or newer, and scikit-learn. It needs no GPU or LLM provider. Clone the examples repository and enter the companion directory:

git clone https://github.com/polyaxon/polyaxon-examples.git
cd polyaxon-examples/blog/evaluation-data-leakage
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python prepare.py --output outputs
python -m pip freeze > outputs/resolved-requirements.txt

With an existing checkout, use its blog/evaluation-data-leakage directory. On Windows, use your environment's activation command. The dependency range is not an environment lock; retain the resolved versions for the experiment. Use a new output directory for each rerun so earlier evidence remains intact.

The fixture contains 15 synthetic cases with three views each, plus one reimport. The script creates two assignments over those same records:

• Row-wise control: sends successive views to training, validation, and test, deliberately placing related examples across boundaries.
• Grouped assignment: keeps each connected case family in one partition, then audits the result.

The control demonstrates a failure mechanism. It is not a production benchmark or an estimate of how often random splitting leaks. No model is trained, and this article reports no measured accuracy change.

In prepare.py, two GroupShuffleSplit calls first reserve test groups, then split the remaining groups into training and validation. Its core operation is:

from sklearn.model_selection import GroupShuffleSplit


def split_groups(rows, group_ids, seed=23):
    outer = GroupShuffleSplit(
        n_splits=1, test_size=0.2, random_state=seed
    )
    development, test = next(outer.split(rows, groups=group_ids))

    inner = GroupShuffleSplit(
        n_splits=1, test_size=0.25, random_state=seed + 1
    )
    train_local, validation_local = next(
        inner.split(
            development,
            groups=[group_ids[i] for i in development],
        )
    )
    return {
        "train": development[train_local],
        "validation": development[validation_local],
        "test": test,
    }

The fractions apply to groups, not rows. Reserving 20% of groups, then 25% of the remaining 80%, aims for a 60/20/20 allocation, subject to rounding. Unequal group sizes can produce different row proportions. GroupShuffleSplit documents this behavior.

Inspect class coverage and group sizes before training. The splitter does not guarantee balanced labels. Define those acceptance rules in advance; do not search seeds for whichever partition gives the most flattering score. Once approved, reuse the saved assignment across model and prompt candidates.

Read the audit before reading a score

Open outputs/audit.json. For each pair of partitions, the report counts distinct shared row IDs, case IDs, connected group IDs, and normalized-text fingerprints. The leaky control is designed to expose overlap; the grouped assignment must have zero overlap on each defined key. A failed grouped audit stops the script after saving its evidence.

That result has a precise scope. Zero shared fingerprints means the specified normalization rule found no exact matches across partitions. It does not prove semantic independence, representative coverage, or absence of information from the future.

The example also writes:

A seed alone is insufficient provenance: changing the input membership, ordering, grouping logic, or library behavior can change a split. Keep the assignment and its input identity. For private datasets, protect the manifests and labels according to their contents; a content hash is an identifier, not anonymization. The public synthetic fixture intentionally leaves all files inspectable for teaching.

Fit preprocessing inside the training boundary

An appropriate split can still leak if the feature pipeline learns from all partitions. Vocabulary, inverse-document-frequency weights, normalization statistics, imputers, and feature selection belong to the training procedure. scikit-learn's leakage guidance explains why fitting these transformations before splitting can inflate evaluation estimates.

For a small text-classification baseline, a Pipeline keeps vectorizer fitting with classifier fitting. From the companion directory, after preparing the data:

import csv
from pathlib import Path

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from sklearn.pipeline import make_pipeline


def read_split(name):
    path = Path("outputs/grouped") / f"{name}.csv"
    with path.open(encoding="utf-8", newline="") as handle:
        return list(csv.DictReader(handle))


train = read_split("train")
validation = read_split("validation")
model = make_pipeline(
    TfidfVectorizer(ngram_range=(1, 2)),
    LogisticRegression(max_iter=1000, random_state=23),
)
model.fit([r["text"] for r in train], [r["label"] for r in train])
predictions = model.predict([r["text"] for r in validation])
print(classification_report(
    [r["label"] for r in validation],
    predictions,
    labels=["access", "billing", "incident"],
    zero_division=0,
))

This small fixture demonstrates wiring, not model selection. Inspect each label's support; a missing class is not evidence of good performance. Notice that the example never reads test.csv. Use validation for development and keep final evaluation separate until the recipe and selection rule are frozen.

Respect the prediction's time boundary

Suppose a router predicts a team when a ticket opens. The final resolution, a later escalation, and an article written after the incident cannot be input features for that prediction, even if they are present in today's export.

Conceptual example: a later outcome may supply the target label, while the model's inputs must reflect what was available when it made the decision.

For future-traffic evaluation, choose a cutoff and document how cases spanning it are handled. Training labels must also have become available by the training cutoff. Keep boundary-spanning families out of conflicting partitions under the declared policy, and record exclusions rather than moving future information into training.

Reconstruct features and retrieval state as of each decision. A chronological split of rows cannot repair a feature table that was populated using later outcomes. Where observations or label windows overlap, decide whether a gap is needed. TimeSeriesSplit supports ordered folds and a gap, but it does not implement application-specific availability or group rules for you.

Apply the same discipline to LLM development

The development boundary includes more than weight updates. A case used to write few-shot examples, select a prompt, adjust a routing rule, or revise a judge rubric has influenced the system. Keep a development set for that work and a separately controlled final evaluation set.

Track how synthetic examples were produced. A question derived from a held-out document should inherit the document family's assignment when the claim concerns unseen documents. Changing its wording or identifier does not create an independent source.

For RAG, access to a relevant document is often part of the task. Do not automatically classify every retrieved answer as leakage. Define which documents and permissions the application would have at the decision time. Accidentally indexing an evaluation answer key or a future resolution breaks that contract. Our RAG evaluation guide covers retrieval and answer-quality assessment beyond the split itself.

An audit of your application data cannot establish what a foundation model saw during pretraining. Record that uncertainty, use appropriately sourced private or fresh cases when suitable, and avoid describing the model as uncontaminated merely because your local split is disjoint.

Keep the split evidence with Polyaxon runs

The companion's optional Polyaxon workflow runs the same audit using a standard Python image. Upload the local companion folder with -u/--upload; the job installs dependencies before starting the audit:

version: 1.1
kind: component
name: evaluation-split-audit
run:
  kind: job
  container:
    image: python:3.12-slim
    workingDir: "{{ globals.run_artifacts_path }}/uploads"
    command: [sh, -c]
    args:
      - |
        set -eu
        python -m pip install --no-cache-dir -r requirements.txt
        exec python tracked.py
    resources:
      requests:
        cpu: "1"
        memory: 1Gi

Before uploading, save a .polyaxonignore file in the companion directory to exclude the local environment, caches, and previous outputs:

.git/
.venv/
__pycache__/
*.pyc
outputs*/
resolved-requirements.txt

From that directory, submit the supplied component to your configured project:

polyaxon run -f polyaxonfile.yaml -u

The flag packages the current folder and uploads it under the run's uploads directory, which is also the container's working directory. The CLI applies the ignore patterns from the file above when preparing the upload.

Use a configured CLI/project and a deployment with artifact storage. The cluster needs access to the Python image and PyPI or your package mirror. requirements.txt includes both scikit-learn and Polyaxon. Dependencies install at the start of each run; installation failure stops the job before the audit begins. If your deployment needs a specific Polyaxon SDK version, pin its entry in requirements.txt.

The tracked runner saves resolved-requirements.txt with its reports. Use reviewed exact dependency versions and a pinned base image digest for repeatable comparisons; recording the versions alone does not freeze future installations.

The tracked runner writes the reports and manifests beneath the run's outputs directory. It then logs dataset and split identities, audit status, and partition sizes through the tracking API. Artifact-reference calls describe files that the program has already saved; they do not create those files or perform the leakage audit.

Carry the approved manifest identity into subsequent training and evaluation runs. In the comparison dashboard, compare candidates evaluated against the same data and scoring policy, then inspect per-case results. Python defines the grouping, preprocessing, and audit rules. Polyaxon retains their execution records and evidence.

Make the evaluation claim reproducible

Start with one existing experiment. Write down what “unseen” should mean, identify the source relationships that matter, and inspect the split before interpreting its score. Preserve the approved manifest, the preprocessing recipe, and the evaluation policy together.

Once that boundary is reliable, a bounded hyperparameter study becomes easier to interpret: each candidate is answering the same question. More trials cannot compensate for an evaluation set that already helped shape the answer.