MLOps Platforms
A glowing purple node graph rises from one isometric platform to a tall pillar topped with a padlock icon on a second platform, set on a dotted grid.
ML Security

MLflow vs Kubeflow: Security and Isolation Compared

MLflow ships no authentication and an experimental basic-auth module; Kubeflow inherits Kubernetes RBAC and Istio policy. Which is safer, and when.

By MLOps Platforms Editorial · · 5 min read

Any honest mlflow vs kubeflow security comparison has to start with an inconvenient fact: these are not two implementations of the same thing. MLflow is a Flask tracking server with an artifact store bolted on. Kubeflow is an entire Kubernetes platform with an Istio service mesh underneath it. Comparing their security is really comparing two questions: how much protection do you get out of the box, and how much surface area do you have to operate to keep that protection working? MLflow gives you almost nothing by default and expects your infrastructure to compensate. Kubeflow gives you namespace isolation, RBAC, and mesh-level authorization, then makes you responsible for a distributed system large enough to misconfigure in interesting ways.

MLflow: a tracking server that trusts everyone by default

A stock mlflow server listens on a port and answers every request. No login, no tokens, no roles. Anyone who can reach the port can read every experiment, download every model artifact, and delete whatever they like. The project does ship an auth module, but the official MLflow authentication docs still carry the warning that “this feature is still experimental and may be enhanced in a future release without warning.” It is HTTP basic auth backed by a SQLite database, with four permission levels (READ, EDIT, MANAGE, NO_PERMISSIONS) and a built-in admin account whose default credentials (admin / password1234 in current releases) the docs tell you to rotate immediately after first login. There is no built-in OIDC, no SSO, no MFA; if you want those, you put the server behind a reverse proxy or an identity-aware gateway and enforce them there.

The CVE history reinforces the point that the server was not designed to face untrusted clients. CVE-2023-1177 is a path traversal in MLflow prior to 2.2.1 that NVD scores 9.8 critical: unauthenticated, network-reachable, and capable of reading arbitrary files off the tracking server host, which in practice means cloud credentials and database connection strings. If your MLflow version predates 2.2.1 and the port is reachable from anywhere you do not fully control, treat it as compromised and rotate what lived next to it. Trackers that follow ML-specific disclosures, such as ai-alert.org, are worth watching here, because MLflow patches land frequently and the tracking server rarely gets the same patch discipline as production API services.

The operational posture that works: pin a current MLflow release, never expose the port beyond a private network, front it with an authenticating proxy (OAuth2 Proxy, an ALB with OIDC, or your mesh’s ingress), and scope the artifact store credentials so the server cannot read anything outside its own bucket prefix.

Kubeflow: real isolation, bought with Kubernetes complexity

Kubeflow’s model is fundamentally different because it delegates security to the platform underneath it. Kubeflow Profiles wrap a Kubernetes namespace per user or team. Owners get a RoleBinding to kubeflow-admin, contributors to kubeflow-edit or kubeflow-view, and every profile gets an Istio AuthorizationPolicy that validates the kubeflow-userid header and only admits traffic that entered through the authorized ingress gateway. That is genuine multi-tenancy: one team’s notebooks, pipelines, and artifacts are invisible to another team’s, enforced by Kubernetes RBAC and mesh policy rather than by an application-level password table. Nothing in MLflow’s experimental auth comes close.

The cost is that every layer is now yours to configure correctly, and the failure mode is public. In June 2020 Microsoft documented a cryptomining campaign that hit tens of Kubernetes clusters through Kubeflow dashboards users had exposed to the internet, typically by switching the Istio ingress service to a public load balancer for convenience. ML nodes carry GPUs, which made them attractive miners. The dashboard is internal-only by default; the breach vector was operators loosening it. That is the Kubeflow security story in miniature: strong defaults, large configuration surface, and the gap between the two is where incidents live.

Wiring it up

MLflow with the basic-auth app and admin defaults rotated:

export MLFLOW_FLASK_SERVER_SECRET_KEY="$(openssl rand -hex 32)"
mlflow server \
  --app-name basic-auth \
  --backend-store-uri postgresql://mlflow@db:5432/mlflow \
  --artifacts-destination s3://ml-artifacts/mlflow \
  --host 127.0.0.1 --port 5000

Binding to 127.0.0.1 and letting an authenticating reverse proxy own the external interface is the single highest-value line in that block.

On the Kubeflow side, the per-profile Istio policy is the control worth auditing. A minimal AuthorizationPolicy that only admits traffic from the ingress gateway carrying a verified user identity looks like this:

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: ns-owner-access-istio
  namespace: team-fraud-models
spec:
  rules:
    - from:
        - source:
            principals:
              - cluster.local/ns/istio-system/sa/istio-ingressgateway-service-account
      when:
        - key: request.headers[kubeflow-userid]
          values: ["analyst@example.internal"]

If a namespace is missing its policy, or the ingress Service type quietly became LoadBalancer with a public IP, the isolation story is fiction regardless of what the dashboard shows.

Which one is more secure?

Wrong question. Kubeflow has stronger security architecture; MLflow has a smaller attack surface to reason about. A three-person team running MLflow inside a private VPC behind an OIDC proxy is in better shape than the same team running a Kubeflow deployment they do not have the Kubernetes depth to patch and audit. A platform organization with real multi-tenancy requirements, dozens of teams, and existing Istio expertise should not try to stretch MLflow’s experimental password table into an isolation boundary; that is what Profiles and namespace RBAC are for. Many stacks run both, with Kubeflow orchestrating and MLflow tracking, and the pipeline trade-offs of that pairing are covered in our Kubeflow vs Metaflow vs Flyte comparison.

Two caveats before you commit either answer to an architecture doc. First, tracking servers and dashboards are where credentials concentrate, so monitor them like production services; access anomalies on an MLflow endpoint are as much an observability problem as a security one, and the monitoring stack you already run for model health, along the lines of what sentryml.com covers, should include them. The MLflow-specific registry hardening that closes the default-open gap — default-deny permissions, isolated artifact storage and digest checks on promotion — is detailed in MLflow model registry security. Second, version drift is the silent killer in both ecosystems: MLflow because patches ship fast and often, Kubeflow because upgrading means upgrading Istio, cert-manager, and Kubernetes itself in the right order. Whichever you pick, put its upgrade cadence on a calendar, not a wishlist. For broader deployment hardening beyond the platform choice, see our guide to secure ML model deployment.

Sources

  1. MLflow Authentication (official docs)
  2. Kubeflow Profiles and multi-tenancy (official docs)
  3. NVD: CVE-2023-1177 (MLflow path traversal)
  4. Microsoft Security Blog: Misconfigured Kubeflow workloads are a security risk
Subscribe

MLOps Platforms — in your inbox

Honest reviews and comparisons of MLOps platforms. Sent only when there is something worth sending.

No spam. Unsubscribe anytime.

Related