Key takeaway: Code versioning is universal and data versioning rarely is, which means the input that most affects model behaviour is the one nobody can reconstruct.
The Investigation That Cannot Happen
A model in production starts performing worse on a segment of traffic. The team wants to compare the current model’s training data against the previous version’s.
The dataset lives at a path in object storage that gets overwritten by each pipeline run. There is one version — the current one. The data that produced the previous model no longer exists anywhere.
The investigation stops. The team can retrain, guess, or accept the regression, and none of those is diagnosis.
What Needs Versioning
Datasets are large, so copying them wholesale for each version is impractical. The workable approaches store references rather than copies.
| Approach | Storage cost | Fidelity |
|---|---|---|
| Overwrite in place | Minimal | None |
| Dated directory copies | Very high | Complete but unmanageable |
| Content-addressed with a pointer file | Low | Complete |
| Immutable append-only with snapshot IDs | Low | Complete |
| Query plus timestamp against a warehouse | Minimal | Depends on retention |
Content-addressed storage — the model DVC and similar tools use — keeps a small pointer file in Git alongside the code. The pointer records hashes; the actual data sits in object storage deduplicated by content. A commit therefore captures both code and data version together, and checking out an old commit retrieves the matching data.
For warehouse-derived datasets, recording the query and a timestamp works provided the underlying tables support time travel. Many modern table formats do, which makes this the cheapest option where available. Without time travel it is an illusion, because the query will return different rows next time.
Lineage Beyond the Snapshot
Knowing which bytes trained a model answers one question. Knowing where those bytes came from answers others.
Record the upstream sources, the transformations applied, the filters that excluded rows, and the version of the transformation code. When a data quality problem is discovered in a source system, lineage answers which models are affected — a question that is otherwise unanswerable and that determines the scope of the response.
The filtering step deserves particular attention because it is where silent bias enters. A pipeline that drops rows with missing values may systematically exclude a population, and that decision is invisible in the resulting dataset. Recording it makes the exclusion reviewable.
Connecting to Models
The link that makes all of this useful is a registry entry per model recording the data version, code commit, configuration and environment together.
With that link, a production regression becomes a comparison: what differed between the two versions. Without it, every investigation starts from nothing.
Include evaluation data versioning too. A model that appears to improve may simply have been evaluated on an easier held-out set, and that possibility cannot be excluded if the evaluation set is not versioned.
The Bottom Line
Adopt content-addressed data versioning with pointer files committed alongside code, record transformations and filters as lineage, version evaluation sets as well as training sets, and link every registered model to the exact data version that produced it.
