Kubeflow

Kubeflow is an open-source platform for Machine Learning and MLOps on Kubernetes introduced by Google. The different stages in a typical machine learning lifecycle are represented with different software components in Kubeflow, including model development, model training, model serving, and automated machine learning.

Tool	Description
Arena	A CLI for Kubeflow
Romario	RESTful API for kick-starting a KF-Pipeline
Training-operator	Training operators on Kubernetes
Katib	Repository for hyperparameter tuning
Fairing	Python SDK for building, training, and deploying ML models
KServe	Standardized Serverless ML Inference Platform on Kubernetes
Kale	Kubeflow’s superfood for Data Scientists
BentoML	Unified Model Serving Framework
Yatai	Model Deployment at Scale on Kubernetes
Bootstrap	🪐 1-click Kubeflow using ArgoCD

Articles/Talks

• Building a Complete AI Based Search Engine with Elasticsearch, Kubeflow and Katib
• Search system on top of Elasticsearch, Kubeflow and Katib
• An end-to-end ML pipeline on-prem - Notebooks & Kubeflow Pipelines on the new MiniKF
• Distributed TensorFlow training using Kubeflow on Amazon EKS
• Building a ML Pipeline from Scratch with Kubeflow
• Train and Serve TensorFlow Models at Scale with Kubernetes and Kubeflow on Azure

Main components

Authentication

• Using Keycloak for Kubeflow (instead of dex)
• Using Keycloak as external OIDC provider on kubeflow

Kubeflow use ambasador-oidc call oidc-authservice as an authentication service with an HTTP Server in combination with Dex as OIDC provider.

Centraldashboard

The Kubeflow Central Dashboard provides an authenticated web interface for Kubeflow and ecosystem components. It acts as a hub for your machine learning platform and tools by exposing the UIs of components running in the cluster. Access through the Istio Gateway that provides access to the Kubeflow.

Notebook

Kubeflow Notebooks provides a way to run web-based development environments inside your Kubernetes cluster by running them inside Pods. Support Jupyter and VSCode.

Kubeflow use these services for deploy and control notebooks:

• jupyter-web-app-deployment
• volumes-web-app-deployment
• notebook-controller-deployment

Deactivate secure cookies if TLS is not configured.

# change in kubernetes templates
APP_SECURE_COOKIES = false

Metadata

Kubeflow Pipelines backend stores runtime information of a pipeline run in Metadata store. Runtime information includes the status of a task, availability of artifacts, custom properties associated with Execution or Artifact, etc. Learn more at ML Metadatafor using google/ml-metadata.

Kubeflow use these services:

• MySQL - store metadata
• metadata-envoy-deployment - envoy proxy
• metadata-grpc-deployment - allows other components to interact with the metadata service
• metadata-writer - comprehensive record of the entire machine learning lifecycle

Pipelines

Cache

The cache server is a key-value storage system that store frequently accessed ML artifacts, such as trained models and intermediate results.

• cache-server

Admission webhook

We need a way to inject common data (env vars, volumes) to pods (e.g. notebooks). PodPreset implementation, customize it for Kubeflow and rename it to PodDefault to avoid confusion. Use admision webhook and CRD to implement the functionality.

• admission-webhook-deployment

PVCViewer

Use filebrowser for provide a file managing interface within a specified directory.

Metacontroller

Use metacontroller as an add-on for Kubernetes that makes it easy to write and deploy custom resources

Istio

KServe

• KServe - The State and Future of Cloud-Native Model Serving
• Open Inference Protocol (V2 Inference Protocol)

Add SSL

• Create certificate

kubectl create -n knative-serving secret tls custom-certs --key registry.key --cert registry.crt

• Edit deployment knative-serving

apiVersion: apps/v1  
kind: Deployment  
metadata:  
  name: controller  
  namespace: knative-serving  
spec:  
  template:  
    spec:  
      containers:  
        - name: controller  
          volumeMounts:  
            - name: custom-certs  
              mountPath: /path-to-docker-registry-cert  # replace here.  
          env:  
            - name: SSL_CERT_FILE  
              value: /path-to-docker-registry-cert # replace here.  
            - name: GODEBUG  
              value: x509ignoreCN=0  
      volumes:  
        - name: custom-certs  
          secret:  
            secretName: custom-certs

Other components

CLI

Use kfpfor interact with Kubeflow as and API for Python, the CLI version only works with GCP (Google Cloud Platform).

• You can perform all operations with the API