Kubeflow in Minikube

This tutorial is part of the Get started with Kubeflow learning path.

Introduction

Kubeflow is known as a machine learning toolkit for Kubernetes. It is an open source project dedicated to making deployments of machine learning workflows on Kubernetes simple, portable, and scalable. Minikube provides a single node Kubernetes cluster that is ideal for development and testing purpose. In this tutorial, you learn:

  • Installing Docker Community Edition, kubectl, and Minikube in a single node Ubuntu system
  • Starting a Kubernetes dashboard
  • Installing Kubeflow
  • Starting Kubeflow Dashboard

Prerequistes

To run this tutorial, you need an Ubuntu 18 machine with a minimum 8 cores, 16 GB RAM, and 250 GB storage. You also need root privileges to run the tutorial steps.

Estimated time

It should take you approximately 30 minutes to complete the tutorial.

Installing Docker Community Edition, kubectl, and Minikube

This tutorial covers the steps to install Docker Community Edition, kubectl, and Minikube in a single node Ubuntu system. If you are interested in more details, read Install Minikube without hypervisor on Ubuntu 18.04.

  1. Install Docker Community Edition.

     apt-get update
     apt-get install -y apt-transport-https ca-certificates curl software-properties-common
     curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
     add-apt-repository \
        "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
        $(lsb_release -cs) \
        Stable"
     apt-get update
     apt-get install -y docker-ce
    
  2. Install kubectl, a Kubernetes command-line tool that lets you run commands against Kubernetes clusters.

     apt-get update
     curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
     echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list
     apt-get update
     apt-get install -y kubectl
    
  3. Install Minikube, which provides a single node Kubernetes cluster that is perfect for development and testing purpose. Alternatively, you can use IBM Cloud Private-CE, Google Cloud Platform, Amazon Cloud Service, or other cloud providers.

     curl -Lo minikube https://storage.googleapis.com/minikube/releases/v1.0.0/minikube-linux-amd64
     # move to /usr/local/bin directory
     chmod +x minikube
     cp minikube /usr/local/bin/
     rm minikube
    
  4. Start Minikube.

     minikube start --vm-driver=none --cpus 6 --memory 12288 --disk-size=120g --extra-config=apiserver.authorization-mode=RBAC --extra-config=kubelet.resolv-conf=/run/systemd/resolve/resolv.conf --extra-config kubeadm.ignore-preflight-errors=SystemVerification
    

Launching Kubernetes Dashboard

Kubernetes Dashboard provides a graphical interface to show Kubernetes resources (for example, pods and deployment) and events. It’s good for those who are not familiar with the Kubernetes command-line tool. To see Kubernetes Dashboard from a remote system using port 9090, run the following code.

minikube dashboard
kubectl proxy --address="0.0.0.0" --port=9090 --accept-hosts '.*'&

Now you can access Kubernetes Dashboard using a web browser.

http://<hostname>:9090/api/v1/namespaces/kube-system/services/http:kubernetes-dashboard:/proxy/

Kubernetes Dashboard

Installing Kubeflow

I have created a directory called kubeflow under /root, but you can use whatever directory you like. The following steps are for installing Kubeflow 0.6.2. Currently, you must use the --config option to bypass an issue in the default installation (without using –config option).

mkdir /root/kubeflow
cd /root/kubeflow
wget https://github.com/kubeflow/kubeflow/releases/download/v0.6.2/kfctl_v0.6.2_linux.tar.gz

tar -xvf kfctl_v0.6.2_linux.tar.gz            
export PATH=$PATH:/root/kubeflow
export KFAPP=kfapp
export CONFIG="https://raw.githubusercontent.com/kubeflow/kubeflow/v0.6-branch/bootstrap/config/kfctl_k8s_istio.0.6.2.yaml"
kfctl init ${KFAPP} --config=${CONFIG} -V

cd ${KFAPP}
kfctl generate all -V        
kfctl apply all -V

When the installation completes, run the following command to see whether all of the pods are in a running status.

kubectl get pod -n kubeflow

Display showing status

Note that you must wait until all of the pods are running before moving on to next step.

If installation fails, you can delete Kubeflow.

cd ${KFAPP}
kfctl delete all --delete_storage

Note: Deleting Kubeflow does not uninstall istio, so reinstalling Kubeflow will fail when the installation program detects a port conflict in istio. Use the following code to bypass this problem.

wget https://raw.githubusercontent.com/kubeflow/kubeflow/v0.6-branch/bootstrap/config/kfctl_k8s_istio.0.6.2.yaml

cp kfctl_k8s_istio.0.6.2.yaml noistio.yaml

Edit the noistio.yaml file, and comment out istio-crds and istio-install, as shown in the following code.

…
    # Istio install. If not needed, comment out istio-crds and istio-install.
    #- kustomizeConfig:
    #    parameters:
    #      - name: namespace
    #        value: istio-system
    #    repoRef:
    #      name: manifests
    #      path: istio/istio-crds
    #  name: istio-crds
    #- kustomizeConfig:
    #    parameters:
    #      - name: namespace
    #        value: istio-system
    #    repoRef:
    #      name: manifests
    #      path: istio/istio-install
    #  name: istio-install
    # This component is the istio resources for Kubeflow (e.g. gateway), not about installing istio.
…

Save your changes and start the installation again.

export CONFIG=/root/kubeflow/noistio.yaml
kfctl init ${KFAPP} --config=${CONFIG} -V        
cd ${KFAPP}
kfctl generate all -V        
kfctl apply all -V

Launch Kubeflow Dashboard

Kubeflow Dashboard can be accessed through the istio-ingressgateway service. To see your setting of istio-ingressgateway service, run the following code.

export INGRESS_HOST=$(minikube ip)
export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}’)

Then, you can access Kubeflow Dashboard in a web browser.

http://$INGRESS_HOST:$INGRESS_PORT

Summary

In this tutorial, I covered the installation of Kubeflow in Minikube as well as how to launch Kubernetes Dashboard and Kubeflow Dashboard.

This tutorial is part of the Get started with Kubeflow learning path. To continue with the learning path, look at the next tutorial in the series, Set up the development environment.

Acknowledgement

Many thanks to Weiqiang Zhuang for his instructions on installing Minikue without hypervisor on Ubuntu 18.04.