What is KubernetesKubernetes is a set of applications written in go. These applications work together to form an interactive platform for managing containers. Kubernetes is also open-source and is available on GitHub. See: https://github.com/kubernetes/kubernetes Kubernetes was donated by Google to the Cloud Native Computing Foundation. Kubernetes runs on many operating systems. However, for this guide we’ll be showing you how to setup for developing and testing Kubernetes on Ubuntu.
Development Environment RequirementsThe basic Kubernetes development environment requires the following:
Setting up Your Host Operating System on a Virtual MachineFirst thing you’ll need to do is install Ubuntu somewhere. Ubuntu 16.04 LTS Server or Desktop will suffice. You could also use 14.04, 15.10 etc., it’s up to you. I’m running Ubuntu 16.04 LTS in a Virtual Machine (VM) on my Mac® via the latest Virtual Box. To set up Ubuntu on Virtual Box:
- Download & Install Virtual Box: https://www.virtualbox.org/wiki/Downloads
- Download an Ubuntu 16.04 ISO file (Server/Desktop): http://www.ubuntu.com/
- Create a new vbox using the Ubuntu ISO you just downloaded.
Virtual Box Settings (Suggestions)
- Base memory 4g
- 80g vdi harddrive
- Shared Clipboard – Bidirectional Only (if desktop)
- Video Memory 128mb (if desktop)
- Networking: NAT
- Guest Additions installed (this is a must)
- VirtualBox VM -> Devices -> Insert Guest Additions CD Image… (if desktop)
- Port Forwarding
Setting up UbuntuIf you are using Ubuntu Desktop go ahead and create a terminal icon:
Start Using sshUse your pc/mac client to ssh over to your Ubuntu server/desktop. If you installed Ubuntu desktop and you want to run ssh:
Login to your ssh server from your client machine (with a password):
$ sudo apt-get install openssh-server
$ sudo service ssh start or sudo /etc/init.d/ssh start
$ ssh -v localhost
In this example the above specified route handler is forwarding 127.0.0.1:2222 TCP/IP packets to the guest ssh server that is running in the vm. The guest ssh server is listening on port 22 for packets sent to IP address 10.0.2.15. In other setups you might have a different IP address and might not have port forwarding enabled. For example, if the IP address of your guest is 192.168.120.129 (as returned from your terminal via the command
$ hostname -I) and there are no port forwarding rules setup, you can login via:
$ ssh email@example.com
Configure sshRunning the e2e Kubernetes tests requires configuring password-less ssh (using keys not passwords). First generate Ubuntu pubic and private keys from an Ubuntu terminal:
Then on your mac in a mac terminal from the ~/.ssh directory:
$ ssh-keygen -t rsa
use the default location and enter a password or just hit enter
$ ssh-copy-id firstname.lastname@example.org
Next configure your Ubuntu guest for pass-wordless ssh back on your Ubuntu terminal:
$ ssh-keygen -b 1024 -t rsa -f id_rsa -P ""
Copy & paste your mac public key (ex:
cat ~/.ssh/id_rsa.pub) into your ubuntu guest’s
/home/mike/.ssh/authorized_keys file. One way way to do this is by ssh’ing into your host and editing the authorized_keys file via nano. Do this step before setting up pass-wordless below. Not much fun trying to type a public key into a terminal editor that doesn’t have cut and paste support 🙂
$ sudo nano /etc/ssh/sshd_config
$ sudo service ssh restart or
sudo /etc/init.d/ssh restart
$ ssh mike@ubuntu (ubuntu = hostname) or
ssh 127.0.0.1 or
ssh 10.0.2.15 …
Make sure it works then exitSince we’ll be logging in as root via password-less, let’s take a shortcut and copy your user public and private keys, as well as authorized_keys over to
/root/.ssh, for example:
You should now be able to ssh, root or user, to your guest ubuntu, without a password, from your user account on your mac terminal:
$ sudo cp ~/.ssh/authorized_keys /root/.ssh/authorized_keys (you may have to
$ sudo cp ~/.ssh/id_rsa /root/.ssh/id_rsa
$ sudo cp ~/.ssh/id_rsa.pub /root/.ssh/id_rsa.pub
ssh -p 2222 email@example.com
ssh -p 2222 firstname.lastname@example.org
$ ssh email@example.com
$ sudo ssh firstname.lastname@example.org
Install goDownload: https://golang.org/dl/
$ sudo curl -O https://storage.googleapis.com/golang/go1.6.linux-amd64.tar.gz
Configure environment variables:
$ sudo tar -xvf go1.6.linux-amd64.tar.gz
$ sudo mv go /usr/local
$ sudo nano ~/.profile
Add these lines:
To run as root with your user profile edits:
mike@mike-VirtualBox:~/go$ go version
go version go1.6.1 linux/amd64
Test your configuration:
$ sudo su –
# source /home/mike/.profile
root@mike-VirtualBox:~# go version
go version go1.6 linux/amd64
Install DockerSee: https://docs.docker.com/engine/installation/linux/ubuntulinux/
To install Docker:
$ sudo apt-get update
$ sudo apt-get install apt-transport-https ca-certificates
$ sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
$ sudo nano /etc/apt/sources.list.d/docker.list
File should include the following (for 14.04 lts aka trusty):
deb https://apt.dockerproject.org/repo ubuntu-trusty main
File should include the following (for 16.04 lts aka xenial):
deb https://apt.dockerproject.org/repo ubuntu-xenial main
To install beta versions of Docker use
testing instead of
Test your Docker install:
$ sudo apt-get update
$ sudo apt-get install linux-image-extra-$(uname -r)
$ sudo apt-get install docker-engine
To pick a particular version of docker:
sudo apt-get install docker-engine=1.10.3-0~trusty
$ sudo service docker start
Add yourself to the docker group which runs as sudo:
$ sudo docker run hello-world
Test your Docker install again:
$ sudo usermod -aG docker mike
$ docker run hello-world
$ curl -L https://github.com/coreos/etcd/releases/download/v3.0.4/etcd-v3.0.4-linux-amd64.tar.gz -o etcd-v3.0.4-linux-amd64.tar.gz$ tar xzvf etcd-v3.0.4-linux-amd64.tar.gz Add
/home/mike/etcd-v3.0.4-linux-amd64to your path in your
~/.profileIn a first terminal run:
In a second terminal:
$ etcdctl set mykey "this is a test"
$ etcdctl get mykey
ForkingCreate your development forks on github for both
Cloning your ForksNormally, when cloning a github.com source repository, one uses the path $GOPATH/src/(path to source in github). This is for convenience when using go get type commands to pull down github hosted packages. However, for legacy reasons, Kubernetes packages are named k8s.io/kubernetes not github.com/kubernetes/kubernetes as you might expect. Thus, to make all of the development tooling work right and be able to find your package code on disk you’re going to have to put the clone in a k8s.io directory instead. Create the k8s.io/kubernetes source trees from your name/kubernetes forks using the following steps:
mike@mike-VirtualBox:~/go/src$ mkdir k8s.io
mike@mike-VirtualBox:~/go/src$ cd k8s.io
$ git clone https://github.com/mikebrow/kubernetes.git
$ git clone https://github.com/mikebrow/kubernetes.github.io.git
Result: source files are copied (cloned) from your forks into the
k8s.io/kubernetes.github.io.git directories. Git creates a master branch on your local hard disk for your git hub forks.
$ cd kubernetes
$ git status
On branch master
Your branch is up-to-date with 'origin/master'.
nothing to commit, working directory clean
Github Profile Recommendations
- Add an email to your public profile
- Enable two factor authentication
Git Client Configuration RecommendationsSet user.* entries to global and/or local:
Make it easier to push:
$ git config --local user.name "Full name here"
$ git config --local user.email "email address here"
$ git config --global user.name "Full name here"
$ git config --global user.email "email address here"
So you don’t have to remember your github credentials each time:
$ git config --global push.default simple
Add a remote named upstream for fetching / pushing changes between the cloned branch on your hard disk and kubernetes/kubernetes:
$ git config credential.helper store
$ git remote add upstream https://github.com/kubernetes/kubernetes.git
git config -lshould look like this:
user.name=Mike Brown email@example.com push.default=simple core.repositoryformatversion=0 core.filemode=true core.bare=false core.logallrefupdates=true remote.origin.url=https://github.com/mikebrow/kubernetes.git remote.origin.fetch=+refs/heads/*:refs/remotes/origin/* branch.master.remote=origin branch.master.merge=refs/heads/master remote.upstream.url=https://github.com/kubernetes/kubernetes.git remote.upstream.fetch=+refs/heads/*:refs/remotes/upstream/* credential.helper=storeThe following steps are recommended to be run by the Kubernetes team. Note: The following link adds a very large bucket of build & verify commands to git commits which will be subsequently run when you commit changes against your branch.
If you don’t link the pre-commit checks you risk pushing a PR that will not pass muster. For example, if you modified docs on the k8s.io/kubernetes fork, and you did not run the corresponding documentation scripts, you risk checking in improperly modified docs.
$ cd .git/hooks/
$ ln -s ../../hooks/pre-commit .
Git WorkflowThe following pic, shows a typical git workflow for finding and fixing an issue with an open source product on GitHub. While this flow was drafted for Docker, it also works for Kubernetes. https://github.com/kubernetes/kubernetes/issues
- If you have questions don’t be shy.
- If you want to work on it, say so.
- Adding a #dibs comment to the issue tells others that you are going to work on a resolution to the issue. Calling dibs is a colloquialism meaning others should recognize that you asked to work this issue first. It is the equivalent of raising your hand first to a question given to an audience.
- If you’re not sure whether you can fix it yourself… at least announce that you are investigating with a #investigating comment. You won’t be assigned and it won’t indicate that you have a resolution, but it lets others know that you might call dibs or open up a pull request later on.
Building KubernetesTo build binaries for running locally:
e2e Test note: With the default e2e test options, when the version of the client and server are out of sync the e2e tests won’t run. New: You can now set check_version_skew=false, to run the e2e with the versions out of skew.
The client/server may not always build depending on what you changed. To force both to be in sync (clean/remove build binaries) with:
$ sudo make
To generate a release:
$ sudo make clean
To generate a release without running tests:
$ make release
$ make release-skip-tests
Testing Kubernetes (unit and integration)Unit tests:
Run just the pkg/api tests:
$ sudo make test
mike@mike-VirtualBox:~/go/src/k8s.io/kubernetes$ sudo WHAT=../../k8s.io/kubernetes/pkg/api make test
$ sudo make test-integration
Kubernetes e2e TestsRunning the Kubernetes e2e tests requires using a cluster. The e2e tests can be run on any Kubernetes compatible cluster. This guide shows you how to create a single node cluster running in your Linux environment. If you followed the above instructions, this cluster will be running in your Ubuntu guest in your VM. We will use two terminals. In terminal 1, build Kubernetes:
Before running a local cluster make sure you have cfssl installed:
$ sudo make clean
$ sudo make
Bring up your local cluster:
$ go get -u github.com/cloudflare/cfssl/cmd/...
In terminal 2, configure kubectl to interact with your local cluster:
$ sudo PATH=$PATH KUBERNETES_PROVIDER=local hack/local-up-cluster.sh
Check status of your single node native local cluster:
$ export KUBERNETES_PROVIDER=local
$ cluster/kubectl.sh config set-cluster local --server=https://localhost:6443 --certificate-authority=/var/run/kubernetes/apiserver.crt
$ cluster/kubectl.sh config set-credentials myself --username=admin --password=admin
$ cluster/kubectl.sh config set-context local --cluster=local --user=myself
$ cluster/kubectl.sh config use-context local
To configure for running e2e tests on your single node local cluster add the following to your
$ cluster/kubectl.sh get nodes
NAME STATUS AGE VERSION
127.0.0.1 Ready 10m v1.6.0-alpha.0.2912+de59ede6b2f4c2-dirty
Check your environment:
mike@mike-VirtualBox:~/go/src/k8s.io/kubernetes$ env | grep kube
To run the e2e tests:
To run just the e2e Conformance tests:
$ KUBE_MASTER_IP=localhost:6443 KUBE_MASTER=https://localhost:6443 go run hack/e2e.go -v --test --test_args="--host=https://localhost:6443"
When finished using your Kubernetes cluster you can quickly bring it down by hitting ctrl-c in terminal 1.
$ KUBE_MASTER_IP=localhost:6443 KUBE_MASTER=https://localhost:6443 go run hack/e2e.go -v --test --test_args="--host=https://localhost:6443 --ginkgo.focus=through.*Conformance"
Running Documentation related scriptsAfter editing documentation related files in the kubernetes/kubernetes fork use before doing a git commit:
Alternatively you can call the following or do a commit and find out which of the following you need to run:
$ sudo hack/update-all.sh
note: requires godep to be installed for sudo as well as $user
$ sudo hack/update-generated-docs.sh
$ sudo hack/update-generated-swagger-docs.sh
$ sudo hack/update-swagger-spec.sh
kubernetes.github.io DocumentsMany of the kubernetes md files (documentation files) have been moved from the kubernetes/kubernetes repository to the kubernetes/kubernetes.github.io repository. For example, the documents published at the documentation web site http://kubernetes.io/docs/ are generated from the kubernetes/kubernetes.github.io repository. If you followed the earlier instructions you should have a directory named
~/go/src/k8s.io/kubernetes.github.iowhere these files are located. Follow these instructions to build and host a local version of the kubernetes documentation website:
http://kubernetes.io/editdocs/In this fashion you can edit the docs and get a preview of what the changes will look like up on kubertnetes.io.
Guidelines/Things to do:
gofmt -s -w file.goon each changed file before committing your changes.
golinton each changed file before committing your changes. (Kubernetes team does not currently use lint. But it doesn’t hurt.)
- Remember to update the documentation when creating or modifying features.
- Remember to add test cases to cover your changes.
- Commits that fix or close an issue should reference them in the commit message Closes #XXXX or Fixes #XXXX.
- After every commit, run the test suite and ensure it is passing.
- Sync and rebase frequently as you code to keep up with master.