A Practical Model Versioning Strategy for Production ML Teams

Daniel Park MLOps
Model versioning strategy diagram with registry and rollback flow

Ask ten ML teams how they version their models and you'll hear ten different answers — and most of them involve ad hoc naming conventions that break down the moment a second engineer joins the project. A folder named model_v3_final_REAL_use_this_one is not a versioning strategy. It's a warning sign that the team is managing model lifecycle through tribal knowledge rather than systematic process.

Model versioning matters for the same reasons software versioning matters: it enables reproducibility, supports safe rollback when a new version degrades production metrics, and provides an audit trail that compliance and security teams require. But ML models have properties that make versioning more complex than versioning application code. A model artifact is large, binary, and inseparable from the data it was trained on, the code that produced it, and the preprocessing pipeline required to serve it. A complete versioning strategy must capture all of these dependencies, not just the model weights file.

Adapting Semantic Versioning for ML Models

Software teams use semantic versioning (MAJOR.MINOR.PATCH) to communicate compatibility guarantees: a major version increment signals breaking changes, a minor increment signals backward-compatible new features, and a patch increment signals backward-compatible bug fixes. This framework translates well to ML models with one important addition: the concept of a behavioral change.

For ML models, a MAJOR version increment should signal that the model's input/output interface has changed in a breaking way — different input feature schema, different output format, or a change in the model's task definition that makes old evaluation results incompatible. A MINOR version increment should signal a meaningful behavioral change that is backward-compatible at the interface level: a retrained model on new data, a fine-tuned variant, or a quantized version that trades off some quality for speed. A PATCH version increment should signal technical fixes that don't affect the model's behavior: updated metadata, corrected documentation, fixed preprocessing bug that was causing silent mishandling of edge cases.

This three-tier scheme gives consuming teams clear signals about what to expect when upgrading. A team consuming a model at version 2.4.1 knows that upgrading to 2.4.2 is low-risk; upgrading to 2.5.0 requires re-running evaluation on their use case; upgrading to 3.0.0 requires a migration effort. Without semantic versioning, every model update requires the consuming team to re-read release notes and manually assess risk — an expensive and error-prone process at scale.

Model Registry Structure

A model registry is the central store for all versioned model artifacts, metadata, and lifecycle state. At minimum, a model registry must store: the model artifact itself (weights, tokenizer, config), the training provenance (dataset version, training code commit, hyperparameters, evaluation results), the serving requirements (hardware, runtime, preprocessing dependencies), and the deployment state (which environments are running which version). Most teams use MLflow, Weights & Biases, or a cloud provider's managed registry (SageMaker, Vertex AI, Azure ML), but the structural requirements are the same regardless of the tool.

Organize your registry around named model families rather than individual artifacts. A model family like customer-intent-classifier has a version history, a current production version, a current staging version, and potentially a shadow deployment version. This family-centric view makes it easy to answer operational questions: "What's currently in production for intent classification? When was it last updated? What would happen if we rolled back to the previous version?" Without family-level organization, registries become flat lists of artifacts with no clear relationship between versions.

Enforce immutability: once a model version is registered, its artifact and training provenance should never change. If you need to add metadata or correct documentation, create a new version or add annotations to the existing one. Mutable model artifacts are a silent correctness hazard — a version number that doesn't correspond to a stable, reproducible artifact undermines the entire purpose of versioning.

Tagging Strategies for Lifecycle Management

Version numbers communicate interface compatibility; tags communicate deployment state. Use a standardized tag vocabulary to mark model versions with their lifecycle status: candidate for versions that have passed automated evaluation and are eligible for deployment, staging for versions deployed to the staging environment, production for the currently live version, shadow for versions running in shadow mode alongside production, and deprecated for versions that have been superseded and should be migrated away from.

Tags should be mutable — the same version transitions from candidate to staging to production as it moves through the deployment pipeline. Only one version per model family should carry the production tag at any given time (or two during a canary deployment). Your CI/CD system and serving infrastructure should look up the production model by tag rather than by explicit version number, enabling rollbacks without code changes.

Rollback Procedures

Rollback is not an edge case — it's a core operational capability that every production ML system must be able to execute reliably and quickly. Before deploying any new model version to production, verify that you can roll back to the previous version in under five minutes. This sounds obvious, but many teams discover they can't achieve fast rollback because their serving infrastructure caches model artifacts aggressively, their preprocessing pipeline changed between versions, or their monitoring system can't distinguish the new version's behavior from noise quickly enough to trigger the rollback decision.

Document the rollback procedure explicitly, not just the happy-path deployment procedure. The rollback procedure should include: which team member has authority to initiate a rollback, what signals trigger a rollback (automated vs. manual), the exact commands or UI actions required to execute it, how to verify that rollback completed successfully, and how to communicate the rollback to stakeholders. Rollback procedures that exist only in engineers' heads fail at 3 AM when the engineer on call is someone who joined last month.

Run rollback drills as part of your deployment process. After every major version deployment, schedule a rollback drill in the following week during a low-traffic period. Execute the rollback to the previous version, verify the system behaves as expected, then redeploy the new version. Teams that practice rollback regularly are orders of magnitude more confident in their ability to execute it under pressure than teams that treat rollback as a theoretical option.

Versioning Preprocessing and Postprocessing

Model weights are only part of the story. A model version is only reproducible if the preprocessing pipeline that transforms raw inputs into model inputs — and the postprocessing pipeline that transforms raw model outputs into application responses — are also versioned and pinned to the model version. A model fine-tuned with a specific tokenizer produces garbage when served with a different tokenizer version. A classification model trained with label encoding scheme A produces wrong answers when its outputs are decoded with label encoding scheme B.

Codify preprocessing and postprocessing as versioned pipeline components stored alongside model artifacts in the registry. Every model version record should contain a reference to the exact preprocessing pipeline version required to serve it. Serving infrastructure should retrieve and load the pinned preprocessing pipeline alongside the model artifact, not rely on a shared "current" preprocessing pipeline that may have been updated since the model was registered.

Key Takeaways

  • Adapt semantic versioning for ML: MAJOR for breaking interface changes, MINOR for behavioral changes, PATCH for technical fixes — giving consuming teams clear upgrade risk signals.
  • Organize your model registry around named model families with version history, not flat artifact lists; enforce artifact immutability to ensure version numbers correspond to stable, reproducible models.
  • Use a standardized tag vocabulary (candidate, staging, production, shadow, deprecated) to track lifecycle state; serving infrastructure should resolve models by tag, enabling rollback without code changes.
  • Document rollback procedures explicitly including authority, trigger signals, execution steps, and verification; practice rollback drills after every major deployment.
  • Version preprocessing and postprocessing pipelines alongside model weights; pin serving infrastructure to the specific pipeline version required by each model version.

Conclusion

Model versioning is an operational discipline, not a one-time setup task. The teams that do it well invest in clear conventions, enforce them through tooling and code review, and treat the model registry as first-class production infrastructure — not an afterthought. The payoff is confidence: confidence that you know exactly what's running in production, confidence that you can roll back in minutes if something goes wrong, and confidence that your compliance audit will go smoothly when it arrives.