Ansible Kubernetes basics (WIP)
This will start to show you how to use ansible and kubernetes clusters together
prior setup
You will need to get kubernetes installed correctly on your a machine. This is fairly easy to do on each system but it is depending on your operating systems. You can find more info about install kubernetes here. You should be able to run this and get the versions:
kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.2", GitCommit:"f6278300bebbb750328ac16ee6dd3aa7d3549568", GitTreeState:"clean", BuildDate:"2019-08-05T09:23:26Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.6-beta.0", GitCommit:"e7f962ba86f4ce7033828210ca3556393c377bcc", GitTreeState:"clean", BuildDate:"2020-01-15T08:18:29Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}
The next thing you will need is a version of golang installed. At the time of writing this here is my version:
go version
go version go1.14.1 darwin/amd64
The next thing is you need ansible installed:
ansible --version
ansible 2.9.10
The last package we will use is the openshift module which has kubernetes support:
pip install openshift
openshift==0.11.2
configuring our first test service in kubernetes
Now lets write a simple web server in golang so we can use it for our example and name it hello.go
package main
import (
"fmt"
"log"
"net/http"
)
func helloServer(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello your request is %s", r.URL.Path)
log.Printf("recieved request for path: %s", r.URL.Path)
}
func main() {
var addr string = ":8180"
http.HandleFunc("/", helloServer)
http.ListenAndServe(addr, nil)
}
Now compile your new web server:
go build hello.go
Now start your web server: ./hello
Now you should be able to curl your server:
curl localhost:8180
2020/07/12 19:40:54 recieved request for path: /
curl localhost:8180/hello
2020/07/12 19:41:18 recieved request for path: /hello
Now we are going to build a multi stage docker container. The advantages to this is that you have 1 container do the building and then you can copy the binary into the next container. This really reduces the size of your docker containers. Docker containers should try to be as small as possible in ordre to pull down faster on the kubernetes cluster. I still see docker containers that are very big. This multi stage setup can significantly reduce container size. More information can be found here.
Dockerfile:
# first stage
FROM golang:1-alpine as build
COPY hello.go hello.go
RUN go build hello.go
# second stage
FROM alpine:latest
COPY --from=build go/hello hello
EXPOSE 8180
ENTRYPOINT ["./hello"]
Now lets build the container:
docker build -t hello-go .
Sending build context to Docker daemon 7.364MB
Step 1/7 : FROM golang:1-alpine as build
---> 3289bf11c284
Step 2/7 : COPY hello.go hello.go
---> Using cache
---> af8ba0b2e7c3
Step 3/7 : RUN go build hello.go
---> Using cache
---> abf9fcf6a5d8
Step 4/7 : FROM alpine:latest
---> a24bb4013296
Step 5/7 : COPY --from=build go/hello hello
---> e7490e0492db
Step 6/7 : EXPOSE 8180
---> Running in 31bd5ffc69f4
Removing intermediate container 31bd5ffc69f4
---> 4212a43d83ec
Step 7/7 : ENTRYPOINT ["./hello"]
---> Running in cc3395b5efba
Removing intermediate container cc3395b5efba
---> 91531928fd1b
Successfully built 91531928fd1b
Successfully tagged hello-go:latest
Now when you do docker images you will see your 2 containers and the primary one is significantly
smaller.
hello-go latest 91531928fd1b 7 seconds ago 13.1MB
<none> <none> abf9fcf6a5d8 2 minutes ago 377MB
The 2nd container that has <none> is the container that did the actual build. If you know you are
ready to deploy your container and want to reclaim this pace you can do:
docker image prune
WARNING! This will remove all dangling images.
Are you sure you want to continue? [y/N] y
Deleted Images:
deleted: sha256:8d8bcc897b3423b1f0c7086b9056d45765616597b82bf7c3623bc3491d90fa57
deleted: sha256:7dbe8d79e9cda7c6c2363fafe6dc4dfbadb5f72802d7976a3ac6caab3e384ffb
deleted: sha256:81b03786ab969976848164135f91d1bfbe0802dee73cbbd6dae7dd1a70a896fb
deleted: sha256:f052a3a75925dc96aefd98cce1c38c431082d305674a961794754f09845c28dc
deleted: sha256:6182c01452845182816a67900c71de08135147b923dc4b4b207c2e07242c3ec5
deleted: sha256:f6f8ef4dd231a206b128c0c2b50b465eb89c0241d62f19b80068ae0c2dfe8a58
deleted: sha256:48168e2ac4b1a3d1f5393ea1b09f7d438e0222793ccd07cbe02240a2973ce8ff
deleted: sha256:d6f81a4eba0af3b6e4ab4a205c9cebfe1a9109a0490f0484e735243cf53da4fd
Total reclaimed space: 363.9MB
Now lets temporary start our docker container and make sure our curl commands still work:
docker run --name hello-go --rm -p 8180:8180 hello-go
curl localhost:8180
curl localhost:8180/hello
You can get out of the running container using ctrl-c
Now lets create a deployment for our new container:
kubectl create deployment hello-go --image=hello-go
You can get the status of the conatiner by doing:
kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-go-6dfc8bbd74-sz7pr 0/1 ImagePullBackOff 0 2m34s
You can get even more info about the pod running this:
kubectl describe pod
You can see it is having problems pullin the image by the READY and STATUS columns. The reason for
this is because it is trying to pull the image off a different docker registery. Lets fix that so
it can pull from the local machine:
```bash
kubectl describe pod hello-go-6dfc8bbd74-sz7pr
Name: hello-go-6dfc8bbd74-sz7pr
Namespace: default
Priority: 0
Node: docker-desktop/192.168.65.3
Start Time: Wed, 15 Jul 2020 08:23:25 -0500
Labels: app=hello-go
pod-template-hash=6dfc8bbd74
Annotations: <none>
Status: Pending
IP: 10.1.0.19
Controlled By: ReplicaSet/hello-go-6dfc8bbd74
Containers:
hello-go:
Container ID:
Image: hello-go
Image ID:
Port: <none>
Host Port: <none>
State: Waiting
Reason: ImagePullBackOff
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-z6pkr (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-z6pkr:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-z6pkr
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned default/hello-go-6dfc8bbd74-sz7pr to docker-desktop
Normal Pulling 3m3s (x4 over 4m32s) kubelet, docker-desktop Pulling image "hello-go"
Warning Failed 3m2s (x4 over 4m31s) kubelet, docker-desktop Failed to pull image "hello-go": rpc error: code = Unknown desc = Error response from daemon: pull access denied for hello-go, repository does not exist or may require 'docker login': denied: requested access to the resource is denied
Warning Failed 3m2s (x4 over 4m31s) kubelet, docker-desktop Error: ErrImagePull
Warning Failed 2m49s (x6 over 4m30s) kubelet, docker-desktop Error: ImagePullBackOff
Normal BackOff 2m35s (x7 over 4m30s) kubelet, docker-desktop Back-off pulling image "hello-go"
Lets edit the config and replace imagePullPolicy: Always to imagePullPolicy: IfNotPresent
kubectl edit deployment hello-go
As soon as you save it kubernetes will reload the config file and attempt to start the pod. Lets see if the pod is running now:
kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-go-697976685b-qhsxq 1/1 Running 0 9s
Success!
Now lets tell kubernetes to expose this port outside the cluster:
kubectl expose deployment hello-go --type=LoadBalancer --port=8180
Now lets check the status to make sure our service is up and running:
curl -I localhost:8180
HTTP/1.1 200 OK
Date: Wed, 15 Jul 2020 13:38:12 GMT
Content-Length: 23
Content-Type: text/plain; charset=utf-8
curl -I localhost:8180/bla
HTTP/1.1 200 OK
Date: Wed, 15 Jul 2020 13:38:17 GMT
Content-Length: 26
Content-Type: text/plain; charset=utf-8
To get the logs of our single pod you can do:
kubectl logs -l app=hello-go
2020/07/15 13:38:12 recieved request for path: /
2020/07/15 13:38:17 recieved request for path: /bla
Now lets scale the hello-go service to 2 containers running:
kubectl scale deployments/hello-go --replicas=2
Now you can see we have 2 pods running:
kubectl get deployment hello-go
NAME READY UP-TO-DATE AVAILABLE AGE
hello-go 2/2 2 2 21m
kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-go-697976685b-qhsxq 1/1 Running 0 9m59s
hello-go-697976685b-zvxfs 1/1 Running 0 15s
Now lets cleanup our testing go container from the kubernetes cluster:
kubectl delete service hello-go
kubectl delete deployment hello-go
check the status:
kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-go-697976685b-qhsxq 0/1 Terminating 0 14m
hello-go-697976685b-zvxfs 0/1 Terminating 0 4m49s
kubectl get pods
NAME READY STATUS RESTARTS AGE
hello-go-697976685b-qhsxq 0/1 Terminating 0 14m
hello-go-697976685b-zvxfs 0/1 Terminating 0 4m52s
kubectl get pods
No resources found.
remove our docker image:
docker rmi hello-go
Untagged: hello-go:latest
Deleted: sha256:91531928fd1bd3215c2d3e8f47c687403a4e26a816aeefbcef8e2ae20cee3c25
Deleted: sha256:4212a43d83ece3f23c6cb26492232fce53940e96a315ac90099e2327889539f0
Deleted: sha256:e7490e0492db45ed5df40a2b9b087ee2b0d9e0609c39036f2d076e2edbb052f1
Deleted: sha256:e5e59395d3aa48f4227eaea4a1eab4174657a0a16bbc19103fe5bf53ef49138b
Now lets connect ansible to our kubernetes cluster
first thing is to create an inventory file:
[localhost]
127.0.0.1 ansible_connection=local
now lets create a script that will get the uptime of our local machine main.yml
---
- hosts: localhost
gather_facts : false
tasks:
- name: get current uptime
command: uptime
register: uptime
changed_when: false
- name: print current uptime
debug:
msg: "{{ uptime.stdout }}"
Test our ansible setup:
ansible-playbook -i inventory main.yml
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************************
TASK [get current uptime] *********************************************************************************************************************************************************************************************************************************
[WARNING]: Platform darwin on host 127.0.0.1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
ok: [127.0.0.1]
TASK [print current uptime] *******************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": "17:16 up 36 days, 3:51, 4 users, load averages: 2.38 2.37 2.48"
}
PLAY RECAP ************************************************************************************************************************************************************************************************************************************************
127.0.0.1 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
So the first thing we need to do is build our docker container in ansible now. We will setup a var
in order to track the hash of our container. This way we can build it only when the image does not
exist on our local system. I will also introduce tags where you can specify a subset of commands
to run. Change your main.yml to this:
---
- hosts: localhost
gather_facts : false
vars:
image_name : hello-go
tasks:
- name: get current uptime
command: uptime
register: uptime
changed_when: false
tags: uptime
- name: print current uptime
debug:
msg: "{{ uptime.stdout }}"
tags: uptime
- name: print current docker container name
debug:
msg: " {{ image_name }} "
tags: docker_build
- name: get the exiting hash of our docker container if it is built
shell: |
docker images -q {{ image_name }}
register: image_hash
changed_when: false
tags: docker_build
- name: print current image hash
debug:
msg: "{{ image_hash.stdout }}"
tags: docker_build
- name: Build our hello-go docker container
shell: |
docker build -t {{ image_name }} ../cmd/hello/
when: not image_hash.stdout
tags: docker_build
With tags I can specify a single tag to run:
ansible-playbook -i inventory main.yml --tags "uptime"
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************************
TASK [get current uptime] *********************************************************************************************************************************************************************************************************************************
[WARNING]: Platform darwin on host 127.0.0.1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
ok: [127.0.0.1]
TASK [print current uptime] *******************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": "20:36 up 36 days, 7:11, 4 users, load averages: 3.01 3.13 3.05"
}
PLAY RECAP ************************************************************************************************************************************************************************************************************************************************
127.0.0.1 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Now lets only run our docker _build tags:
ansible-playbook -i inventory main.yml --tags "docker_build"
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************************
TASK [print current docker container name] ****************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": " hello-go "
}
TASK [get the exiting hash of our docker container if it is built] ****************************************************************************************************************************************************************************************
[WARNING]: Platform darwin on host 127.0.0.1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
ok: [127.0.0.1]
TASK [print current image hash] ***************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": ""
}
TASK [Build our hello-go docker container] ****************************************************************************************************************************************************************************************************************
changed: [127.0.0.1]
PLAY RECAP ************************************************************************************************************************************************************************************************************************************************
127.0.0.1 : ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
You can see that 1 was changed because it did not find the hash of the container. Now if you run it again you will see it skip the build:
ansible-playbook -i inventory main.yml --tags "docker_build"
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************************
TASK [print current docker container name] ****************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": " hello-go "
}
TASK [get the exiting hash of our docker container if it is built] ****************************************************************************************************************************************************************************************
[WARNING]: Platform darwin on host 127.0.0.1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
ok: [127.0.0.1]
TASK [print current image hash] ***************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": "563c1408fa75"
}
TASK [Build our hello-go docker container] ****************************************************************************************************************************************************************************************************************
skipping: [127.0.0.1]
PLAY RECAP ************************************************************************************************************************************************************************************************************************************************
127.0.0.1 : ok=3 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
You can also “skip” tags:
ansible-playbook -i inventory main.yml --skip-tags "uptime"
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************************
TASK [print current docker container name] ****************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": " hello-go "
}
TASK [get the exiting hash of our docker container if it is built] ****************************************************************************************************************************************************************************************
[WARNING]: Platform darwin on host 127.0.0.1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
ok: [127.0.0.1]
TASK [print current image hash] ***************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": "563c1408fa75"
}
TASK [Build our hello-go docker container] ****************************************************************************************************************************************************************************************************************
skipping: [127.0.0.1]
PLAY RECAP ************************************************************************************************************************************************************************************************************************************************
127.0.0.1 : ok=3 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
You can also group tasks together if want. This will run this entire playbook but on bigger playbooks this is very useful:
ansible-playbook -i inventory main.yml --tags "docker_build,uptime"
PLAY [localhost] ******************************************************************************************************************************************************************************************************************************************
TASK [get current uptime] *********************************************************************************************************************************************************************************************************************************
[WARNING]: Platform darwin on host 127.0.0.1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See
https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information.
ok: [127.0.0.1]
TASK [print current uptime] *******************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": "20:41 up 36 days, 7:16, 4 users, load averages: 2.18 2.86 2.95"
}
TASK [print current docker container name] ****************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": " hello-go "
}
TASK [get the exiting hash of our docker container if it is built] ****************************************************************************************************************************************************************************************
ok: [127.0.0.1]
TASK [print current image hash] ***************************************************************************************************************************************************************************************************************************
ok: [127.0.0.1] => {
"msg": "563c1408fa75"
}
TASK [Build our hello-go docker container] ****************************************************************************************************************************************************************************************************************
skipping: [127.0.0.1]
PLAY RECAP ************************************************************************************************************************************************************************************************************************************************
127.0.0.1 : ok=5 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
Now we are going to write some kubernetes inside ansible. More information about this module can be found here.