Vytvoríme si install.sh v /root.
#/bin/bash -x apt-get install -y apt-transport-https ca-certificates curl software-properties-common curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" apt-get update apt-get install -y docker-ce iptables arptables ebtables wget -q https://storage.googleapis.com/golang/getgo/installer_linux chmod +x installer_linux ./installer_linux source /root/.bash_profile update-alternatives --set iptables /usr/sbin/iptables-legacy update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy update-alternatives --set arptables /usr/sbin/arptables-legacy update-alternatives --set ebtables /usr/sbin/ebtables-legacy apt-get update && apt-get install -y apt-transport-https curl curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - cat <<EOF | tee /etc/apt/sources.list.d/kubernetes.list deb https://apt.kubernetes.io/ kubernetes-xenial main EOF apt-get update apt-get install -y kubelet kubeadm kubectl mkdir -p /opt/k8s/fake cat > /opt/k8s/fake.sh <<EOF #!/bin/bash cd /opt/k8s/fake echo 0 > panic mount --bind panic /proc/sys/kernel/panic echo 0 > panic_on_oops mount --bind panic_on_oops /proc/sys/kernel/panic_on_oops echo 0 > overcommit_memory mount --bind overcommit_memory /proc/sys/vm/overcommit_memory mkdir block mount -o bind block/ /sys/block/ mount --make-rshared / EOF chmod +x /opt/k8s/fake.sh cat > /etc/systemd/system/fake.service <<EOF [Unit] Before=kubelet.service [Service] ExecStart=/opt/k8s/fake.sh [Install] WantedBy=default.target EOF chmod 644 /etc/systemd/system/fake.service systemctl daemon-reload systemctl enable fake.service systemctl start fake.service cat > /etc/docker/daemon.json <<EOF { "exec-opts": ["native.cgroupdriver=systemd"], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2" } EOF systemctl daemon-reload systemctl restart docker kubeadm config images pull
Spustíme inštaláciu a počkáme na úspešné dokončenie základnej inštalácie k8s:
chmod +x /root/install.sh /root/install.sh
Inštalácia je spravená tak, že systém funguje aj po reštarte, avšak trvá 3-5 minút kým znovu nabehnú všetky služby.
Master ma niekoľko špeciálnych krokov. Najprv inicializujeme kubernetes, následne pridáme network.
kubeadm init --pod-network-cidr=10.244.0.0/16 mkdir -p $HOME/.kube cp -i /etc/kubernetes/admin.conf $HOME/.kube/config chown $(id -u):$(id -g) $HOME/.kube/config wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml sed -i 's/vxlan/host-gw/g' kube-flannel.yml # Tento krok jsem musel preskocit, aby mit to fungovalo na Brnenskem VPS kubectl apply -f kube-flannel.yml
Odporúčam sledovať priebeh deploymentu a počkať, kým budú všetko v stave running s plným počtom:
kubectl get pods --all-namespaces
Vysledok by mal vyzerat cca:
# kubectl --namespace=kube-system get pods NAME READY STATUS RESTARTS AGE coredns-f9fd979d6-f9v99 1/1 Running 1 12m coredns-f9fd979d6-v7w2x 1/1 Running 1 12m etcd-vps3 1/1 Running 1 12m kube-apiserver-vps3 1/1 Running 1 12m kube-controller-manager-vps3 1/1 Running 1 12m kube-flannel-ds-zbc47 1/1 Running 1 12m kube-proxy-7zvc5 1/1 Running 1 12m kube-scheduler-vps3 1/1 Running 1 12m
Ak chceme aby aj master node slúžil ako worker, môžeme ho pridať:
kubectl taint nodes --all node-role.kubernetes.io/master-
Získame si najprv na master node token pre pridanie ďalšieho nodu do clustru. Získame tým príkaz ktorý iba copy-paste na worker node:
kubeadm token create --print-join-command # Ukazka vystupu prikazu kubeadm token create --print-join-command kubeadm join 37.205.14.241:6443 --token 53r0e7.21pznuukg755rpz3 --discovery-token-ca-cert-hash sha256:6be4cb960d16fae2dd7ce96c7a16fc585ce174973c04ded0f91df6cf86681e3a
Na master node môžeme sledovať stav nodov:
# kubectl get nodes NAME STATUS ROLES AGE VERSION vps3 Ready master 32m v1.19.4 vps4 Ready <none> 21m v1.19.4
Je treba zmenit strictARP na true, viz https://metallb.universe.tf/installation/
kubectl edit configmap -n kube-system kube-proxy kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.10.3/manifests/namespace.yaml kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.10.3/manifests/metallb.yaml
Config pro loadbalancer
apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: default protocol: layer2 addresses: - apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: default protocol: layer2 addresses: - 37.205.x.x/32 # IP addressa
U ingresu je treba zmenit servisu, co bere adresu z routeru, z NodePort na ClusterIP
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.4/deploy/static/provider/baremetal/deploy.yaml kubectl edit service -n ingress-nginx ingress-nginx-controller # edit type: NodePort -> type: LoadBalancer
Pro automaticke vystavovani certifikatu
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.6.0/cert-manager.yaml
Nasledne je treba vytvorit issuer:
kind: ClusterIssuer metadata: name: letsencrypt-prod namespace: cert-manager spec: acme: # The ACME server URL server: https://acme-v02.api.letsencrypt.org/directory # Email address used for ACME registration email: email@email.com # Name of a secret used to store the ACME account private key privateKeySecretRef: name: letsencrypt-prod # Enable the HTTP-01 challenge provider solvers: - http01: ingress: class: nginx
Pro otestovani muzeme nainstalovat napr dashboard.
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.3.1/aio/deploy/recommended.yaml
Dodatecne role + ingress pro dashboard
apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kubernetes-dashboard --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin-user namespace: kubernetes-dashboard --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: dashboard namespace: kubernetes-dashboard annotations: kubernetes.io/ingress.class: "nginx" cert-manager.io/cluster-issuer: "letsencrypt-prod" nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" spec: tls: - hosts: - k8s.domain.tld secretName: k8s.domain.tld rules: - host: k8s.domain.tld http: paths: - path: / pathType: Prefix backend: service: name: kubernetes-dashboard port: number: 443