README.md

pupernetes - p8s

Table of Contents

• Goals
• Requirements
  • Runtime
    • Executables
    • Systemd
    • Resources
  • Development
    • Build
• Getting started
  • Download
  • Run
  • Stop
  • Systemd as job type
  • Command line docs
• Metrics
• Current limitations

Goals

Run a managed Kubernetes setup.

This project's purpose is to provide a simple Kubernetes setup to validate any software on top of it.

You can use it to validate a software dependency on Kubernetes itself or just to run some classic app workflows with argo.

As pupernetes runs in travis and circle-ci, it becomes very easy to integrate this tool in any Kubernetes project.

This project has been initially designed to perform the end to end testing of the datadog-agent.

Provides:

• etcd v3
• kubectl
• kubelet
• kube-apiserver
• kube-scheduler
• kube-controller-manager
• kube-proxy
• coredns

The default setup is secured with:

• Valid x509 certificates provided by an embedded vault PKI
  • Able to use the Kubernetes CSR and the service account root-ca
• HTTPS webhook to provide token lookups for the kubelet API
• RBAC

Requirements

Runtime

Executables

• tar
• unzip
• systemctl
• systemd-resolve (or a non-systemd managed /etc/resolv.conf)
• mount

Additionally any implicit requirements needed by the kubelet, like the container runtime and more. Currently only reporting docker, please see the current limitations.

Systemd

A recent systemd version is better to gain:

• systemd-resolve
• journalctl --since
• more convenient dbus API

Resources

• 4GB of memory is recommended
• 5GB of free disk space for the binaries and the container images

Development

Pupernetes must be run on linux (or linux VM).

Please see our ubuntu 18.04 notes about it.

To compile pupernetes, you need the following binaries:

• go 1.10
• make

Build

go get -u github.com/DataDog/pupernetes
cd ${GOPATH}/src/github.com/DataDog/pupernetes
make

Getting started

Download

You need to download the last version:

VERSION=0.5.0
curl -LOf https://github.com/DataDog/pupernetes/releases/download/v${VERSION}/pupernetes
chmod +x ./pupernetes
./pupernetes --help

Run

sudo ./pupernetes daemon run sandbox/

Note:

kubectl can be automatically installed by pupernetes.

You need to run the following command to add kubectl to the $PATH:

sudo ./pupernetes run sandbox/ --kubectl-link /usr/local/bin/kubectl

$ kubectl get svc,ds,deploy,job,po --all-namespaces

NAMESPACE     NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)         AGE
default       kubernetes   ClusterIP   192.168.254.1   <none>        443/TCP         3m
kube-system   coredns      ClusterIP   192.168.254.2   <none>        53/UDP,53/TCP   3m

NAMESPACE     NAME             DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
kube-system   kube-proxy       1         1         1         1            1           <none>          3m
kube-system   kube-scheduler   1         1         1         1            1           <none>          3m

NAMESPACE     NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
kube-system   coredns   1         1         1            1           3m

NAMESPACE     NAME                       READY     STATUS    RESTARTS   AGE
kube-system   coredns-747dbcf5df-p2lhq   1/1       Running   0          3m
kube-system   kube-controller-manager    1/1       Running   0          3m
kube-system   kube-proxy-wggdn           1/1       Running   0          3m
kube-system   kube-scheduler-92zrj       1/1       Running   0          3m

Stop

Gracefully stop it with:

• SIGINT
• SIGTERM
• --timeout
• curl -XPOST 127.0.0.1:8989/stop

Hyperkube versions

pupernetes can start a specific Kubernetes version with the flag --hyperkube-version=1.9.3.

These are the current supported versions:

• 1.11
• 1.10
• 1.9
• 1.8
• 1.7
• 1.6
• 1.5
• 1.4
• 1.3

Systemd as job type

It's possible to run pupernetes as a systemd service directly with the command line. In this case, pupernetes asks to systemd-dbus to be daemonised with the given arguments. See more info about it in the run command.

This command line is very convenient to run pupernetes in SaaS CI:

• travis
• circle-ci

Command line docs

The full documentation is available here.

Metrics

Pupernetes exposes prometheus metrics to improve the observability.

You can have a look at which metrics are available here.

Current limitations

• Container runtime
  • You need docker already up and running
  • You cannot use cri-containerd / crio without changing manually the systemd unit /run/systemd/system/p8s-kubelet.service
• Systemd
  • Currently working with systemd only
  • Could be containerized with extensive mounts
    • binaries
    • dbus
• Networking
  • The CNI bridge cannot be used yet
  • Kubernetes cluster IP range is statically set
• Secrets
  • IP SAN
    • Statically configured with the given Kubernetes cluster IP range
• Support for Custom Metrics
  • You can register an API Service for an External Metrics Provider. This is only supported for 1.10.x and 1.11.x.