MLOps Platforms
Isometric purple cube with a glowing lavender top sits on a dark platform ringed by glowing magenta cylinder nodes, evoking a central model ops hub
Production Operations

Model Ops Platform: What It Does and the Metric to Track

A model ops platform keeps production models inventoried, approved, monitored and reversible. What it covers and the metric to track.

By MLOps Platforms Editorial · · 7 min read

A model ops platform is the operational layer that keeps every production model inventoried, approved, monitored and reversible once training ends. For most ML teams it is not a separate purchase. It is a model registry (MLflow, SageMaker or Vertex AI), a serving layer and a drift monitor, wired so that every live endpoint traces to an approved, evidenced version.

What usually sends a team looking for one is an incident. A p99 latency alert fires on a fraud-scoring endpoint at 3am. The registry says champion is version 14. The pod is actually serving an artifact someone pushed with kubectl apply during a hotfix three weeks ago, built from a notebook run nobody registered. Rollback becomes archaeology. Everything a model ops platform promises, governance included, depends on answering “what is running right now?” quickly and correctly.

Vendor definitions run wider. EY calls ModelOps “the framework at an organization for governing AI models throughout their entire lifecycle” (EY). How that differs from MLOps is covered in ModelOps platform vs MLOps platform. The questions here are operational: what the platform does, and which number shows it is doing it.

What does a model ops platform actually do?

A model ops platform does five jobs once a model leaves training. It inventories what is live, gates promotion on evidence, controls rollout and rollback, watches inputs and outputs for drift, and retires models on purpose. Training, feature pipelines and experiment tracking happen earlier, in the wider MLOps stack.

JobQuestion it answersTypical tooling
InventoryWhat is serving traffic now, and who owns it?MLflow, SageMaker, Vertex AI or W&B registries
Promotion gatesWhat evidence backed this version going live?Registry tags plus CI checks on eval artifacts
Rollout and rollbackHow does a version reach traffic, and how fast can it leave?KServe, Seldon Core, Ray Serve, BentoML, managed endpoints
MonitoringHave inputs or prediction quality moved?Evidently, Arize, Fiddler
RetirementWhat no longer serves anything?Registry archival plus a deprecation policy

The registry is the spine. In MLflow, each registered version links to the run or notebook that produced it, and an alias such as models:/fraud-scorer@champion is a mutable pointer that deployment resolves (MLflow docs). Model stages are deprecated (MLflow workflow docs), so build promotion on aliases and tags. The conventions that keep a registry from rotting are in model registry patterns that hold in production.

Rollout borrows from SRE practice, with one gap. The Site Reliability Workbook defines canarying as “a partial and time-limited deployment of a change in a service and its evaluation” (Google SRE Workbook). The gap is in the evaluation. A model canary judged only on 5xx rate and p99 latency will pass a model that returns fast, well-formed, wrong scores. Compare prediction distributions between canary and control as well.

Do you need a separate model ops platform?

Usually not. If every production model comes out of an ML pipeline and nobody outside engineering audits the portfolio, a registry with enforced conventions plus the serving and monitoring you already run is the model ops platform. A dedicated governance suite is worth paying for when the inventory has to include rules engines, scorecards and vendor models, or when a regulator expects evidence on demand.

Banking is the clearest case. On April 17, 2026 the Federal Reserve issued SR 26-2, “Revised Guidance on Model Risk Management”, superseding SR 11-7. The Fed says it is most relevant to banking organizations with over $30 billion in total assets (Federal Reserve). Teams in that position should read best MLOps platform for regulated industries, which ranks platforms by the evidence they emit. Everyone else gets more from making the registry authoritative and measuring whether it is.

The metric that matters: registry coverage

Registry coverage is the share of live inference endpoints whose served artifact resolves to a registered version that is approved, carries owner and eval evidence, and holds a deployable alias: registry_coverage = traceable_endpoints / all_live_endpoints.

The obvious alternative, a count of registered versions, is what registry dashboards show by default. It measures what was written to the registry, not what is running. The count only goes up, it looked healthy during the 3am incident, and it cannot drop when someone deploys around the process. Coverage works from the serving side inward, so a side-door deploy shows up on the next reconcile.

Split the non-traceable endpoints into four buckets, because each has a different fix:

  • unregistered: no registry pointer. Someone deployed around the pipeline.
  • missing_evidence: registered, but no owner or eval run attached.
  • unapproved: evidence attached, validation status not approved.
  • alias_drift: approved, but not the version any deployable alias points to. The registry and the cluster disagree about what production is.

Wiring it up

The reconciler below runs as a nightly Kubernetes CronJob. It lists KServe InferenceServices, reads two annotations the deploy pipeline stamps on each one, checks the matching MLflow version, and pushes per-status counts to a Prometheus Pushgateway. The annotation keys are a local convention. KServe’s spec carries the artifact location in storageUri under spec.predictor.model, not a registry version (KServe docs).

from kubernetes import client, config
from mlflow import MlflowClient
from mlflow.exceptions import MlflowException
from prometheus_client import CollectorRegistry, Gauge, push_to_gateway

MODEL_KEY = "models.platform.internal/registered-model"
VERSION_KEY = "models.platform.internal/version"
REQUIRED_TAGS = {"owner", "eval_run_id", "validation_status"}
DEPLOYABLE_ALIASES = ("champion", "challenger")  # challenger = active canary
STATUSES = ("traceable", "alias_drift", "unapproved", "missing_evidence", "unregistered")


def live_endpoints():
    config.load_incluster_config()
    isvcs = client.CustomObjectsApi().list_cluster_custom_object(
        group="serving.kserve.io", version="v1beta1", plural="inferenceservices"
    )
    for item in isvcs["items"]:
        meta = item["metadata"]
        ann = meta.get("annotations") or {}
        yield f"{meta['namespace']}/{meta['name']}", ann.get(MODEL_KEY), ann.get(VERSION_KEY)


def alias_versions(mlflow_client, model_name):
    versions = set()
    for alias in DEPLOYABLE_ALIASES:
        try:
            versions.add(str(mlflow_client.get_model_version_by_alias(model_name, alias).version))
        except MlflowException:
            pass  # alias not set on this model
    return versions


def classify(mlflow_client, model_name, version):
    if not model_name or not version:
        return "unregistered"
    try:
        mv = mlflow_client.get_model_version(model_name, version)
    except MlflowException:
        return "unregistered"
    if not REQUIRED_TAGS.issubset(mv.tags):
        return "missing_evidence"
    if mv.tags["validation_status"] != "approved":
        return "unapproved"
    if str(mv.version) not in alias_versions(mlflow_client, model_name):
        return "alias_drift"
    return "traceable"


def main():
    mlflow_client = MlflowClient()  # reads MLFLOW_TRACKING_URI
    counts = dict.fromkeys(STATUSES, 0)
    for ref, model_name, version in live_endpoints():
        status = classify(mlflow_client, model_name, version)
        counts[status] += 1
        if status != "traceable":
            print(f"{status}\t{ref}\t{model_name}:{version}")  # detail goes to logs, not labels

    registry = CollectorRegistry()
    gauge = Gauge("modelops_live_endpoints", "Live endpoints by registry traceability status",
                  ["status"], registry=registry)
    for status, n in counts.items():
        gauge.labels(status=status).set(n)
    push_to_gateway("pushgateway.monitoring:9091", job="modelops_reconciler", registry=registry)


if __name__ == "__main__":
    main()

The Prometheus side is one recording rule and two alerts. Scrape the Pushgateway with honor_labels: true so the job label survives.

groups:
  - name: modelops
    rules:
      - record: modelops:registry_coverage:ratio
        expr: |
          sum(modelops_live_endpoints{status="traceable"})
            / sum(modelops_live_endpoints)
      - alert: RegistryCoverageBelowTarget
        expr: modelops:registry_coverage:ratio < 1
        for: 26h
        labels:
          severity: ticket
      - alert: ModelOpsReconcilerStale
        expr: time() - push_time_seconds{job="modelops_reconciler"} > 2 * 86400
        labels:
          severity: ticket

With for: 26h, two consecutive nightly runs must both report a gap, which filters out a promotion caught mid-rollout. Both alerts are tickets, not pages. A coverage gap is rollback debt, and you fix it in daylight.

What you’ll see

Healthy coverage is a flat line at 1.0 with the occasional dip when a reconcile lands mid-promotion, cleared by the next run. Total endpoints step up and down with launches and retirements while the ratio holds.

Unhealthy coverage has four recognizable shapes:

  • Slow staircase down. The unregistered count grows by one every few weeks. Teams are shipping around a sanctioned path that is slower than kubectl apply. Make the sanctioned path faster.
  • Cliff. Coverage drops sharply in one run while the total is unchanged. A tag key was renamed, a registry migration lost tags, or the alias vocabulary changed. Check missing_evidence first.
  • Persistent alias_drift. The alias moved and the deployment did not, usually because the deploy job reads a config file instead of resolving the alias. Your rollback target in the registry is not what the cluster runs.
  • Flat at 1.0 forever. Suspicious, not reassuring. The Pushgateway implements no TTL for pushed metrics (Pushgateway README), so a dead reconciler keeps reporting its last value. That is what the push_time_seconds alert catches.

Caveats

Canary windows. During a canary the endpoint serves the challenger by design. Without the challenger alias in the allowed set, every rollout raises an alias_drift ticket and the team learns to ignore the alert.

Cardinality. Keep status as the only label. Prometheus warns that “every unique combination of key-value label pairs represents a new time series” (Prometheus docs). Model version and endpoint name are unbounded over time, which is why per-endpoint detail goes to logs.

Coverage is not correctness. A fully traceable endpoint can still serve a model whose inputs drifted. Coverage says evidence exists and rollback is possible. PSI or KS checks in Evidently or Arize say whether the model still deserves the traffic, and the wider practice of machine learning observability covers what else to watch once a model is live. What the eval evidence should contain is in evaluation pipeline design.

Tags are self-reported. validation_status=approved is a string anyone with write access can set. Restrict who can write it, as covered in MLflow model registry security.

Blind spots. Batch scoring jobs, SageMaker endpoints and models embedded in application services are invisible to a KServe-only lister. Add one lister per serving surface, or coverage flatters you.

FAQ

Is MLflow a model ops platform?

MLflow covers the inventory and lineage half of a model ops platform: its registry tracks versions, aliases, tags and the run that produced each version. Open-source MLflow does not watch live traffic for drift or block unapproved promotions by itself, so teams usually pair it with KServe, Ray Serve or BentoML for serving and Evidently or Arize for monitoring.

What are examples of model ops platforms?

Common examples are managed suites (SageMaker, Vertex AI, Azure Machine Learning, Databricks), which bundle a registry, serving and monitoring; governance suites such as ModelOp Center, which focus on inventory, approvals and audit evidence across model types; and open-source assemblies of MLflow, KServe and Evidently, where you own the integration work.

Does a model ops platform work for LLMs?

Partly. Inventory, promotion gates, canary rollout and rollback carry over to LLM applications unchanged. The unit of versioning does not: an LLM release is weights or a pinned API model plus prompt template, retrieval configuration and decoding settings, so registering only the model identifier leaves much of the behavior untracked. Monitoring shifts from distribution drift toward graded evaluations on traces.

Sources

  1. MLflow Model Registry Documentation
  2. MLflow Model Registry Workflows
  3. KServe: Deploy Your First Predictive Model with InferenceService
  4. Google SRE Workbook: Canarying Releases
  5. Prometheus: Metric and Label Naming
  6. Prometheus Pushgateway README
  7. Federal Reserve SR 26-2: Revised Guidance on Model Risk Management
  8. EY: ModelOps Frameworks Bridge AI Governance and Value
#modelops #model-registry #mlflow #kserve#model-monitoring#prometheus
Subscribe

MLOps Platforms — in your inbox

Honest reviews and comparisons of MLOps platforms. Sent only when there is something worth sending.

No spam. Unsubscribe anytime.

Related