Kubernetes - On-Premise

https://kubernetes.io/ko/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

 

1. Virtual box

 

ubuntu 20.04 설치 후 

해당 vm을 복제하여 node1, node2 생성

 

ubuntu server

2. Ip setup

sudo vim /etc/netplan/00-installer-config.yaml

# This is the network config written by 'subiquity'
network:
  ethernets:
    enp0s3:
      dhcp4: true
  version: 2
  
  => 
  
# This is the network config written by 'subiquity'
network:
  ethernets:
    enp0s3:
      dhcp4: false
      dhcp6: false
      addresses: [192.168.0.24/24]
      gateway4: 192.168.0.1
      nameservers:
        addresses: [8.8.8.8,8.8.4.4]
  version: 2
  
sudo netplan apply

 

 

3. Docker install

모든 노드에 도커 설치

https://docs.docker.com/engine/install/ubuntu/

 

Set up the repository

# Update the apt package index and install packages to allow apt to use a repository over HTTPS:

sudo apt-get update

sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

 

# Add Docker’s official GPG key:

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg


# Use the following command to set up the stable repository. 
# To add the nightly or test repository, add the word nightly or test (or both) after the word stable in the commands below. Learn about nightly and test channels.

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

 

Install Docker Engine

 

# Update the apt package index, and install the latest version of Docker Engine and containerd, or go to the next step to install a specific version:

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

 

4. kubeadm, kubelet, kubectl 

https://kubernetes.io/ko/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

 

# apt 패키지 색인을 업데이트하고, 쿠버네티스 apt 리포지터리를 사용하는 데 필요한 패키지를 설치한다.

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates curl


# 구글 클라우드의 공개 사이닝 키를 다운로드 한다.

sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg


# 쿠버네티스 apt 리포지터리를 추가한다.

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list


# apt 패키지 색인을 업데이트하고, kubelet, kubeadm, kubectl을 설치하고 해당 버전을 고정한다.

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

 

swap 기능 비활성화

https://askubuntu.com/questions/214805/how-do-i-disable-swap

sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
sudo sed -i '/ swap / s/^/#/' /etc/fstab

 

docker cgroup driver name change

https://stackoverflow.com/questions/43794169/docker-change-cgroup-driver-to-systemd

sudo vim /etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

sudo systemctl restart docker

 

5. Cluster

마스터 노드에서 

kubeadm init

# 설정이 잘못 되었을대 실행한다 
# kubeadm reset 

# 마지막 명령어 확인할것!

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.0.24:6443 --token jajoui.juipepec3zhntly3 \
	--discovery-token-ca-cert-hash sha256:6de9565152976b5946f82fd2112d673c5ae0c07beb10657ee88737ebb3ac987d

 

워커 노드에서 조인하여 클러스터 형성 

kubeadm join 192.168.0.24:6443 --token jajoui.juipepec3zhntly3 \
	--discovery-token-ca-cert-hash sha256:6de9565152976b5946f82fd2112d673c5ae0c07beb10657ee88737ebb3ac987d

 

 

kubectl get nodes 를 보면 NotReady 상태

podnetwork 설정을 해줘야 한다

https://kubernetes.io/docs/concepts/cluster-administration/addons/

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

 

 

Weavenet

https://www.weave.works/docs/net/latest/kubernetes/kube-addon/

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

root@master:~# kubectl get nodes
NAME     STATUS   ROLES                  AGE     VERSION
master   Ready    control-plane,master   12m     v1.22.4
node1    Ready    <none>                 6m11s   v1.22.4
node2    Ready    <none>                 5m47s   v1.22.4

 

Token create

kubeadm token create --print-join-command
kubeadm token list

 

Test

kubectl creaate deploy nx --image=nginx
kubectl get pod

kubectl expose deploy nx --type=NodePort --port=80 --target-port=80

 

트러블 슈팅

• kubeadm init 시 아래와 같은 에러 발생

[init] Using Kubernetes version: v1.26.2
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR CRI]: container runtime is not running: output: time="2023-03-15T16:55:44Z" level=fatal msg="validate service connection: CRI v1 runtime API is not implemented for endpoint \"unix:///var/run/containerd/containerd.sock\": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

https://github.com/containerd/containerd/issues/4581

Kubeadm unknown service runtime.v1alpha2.RuntimeService · Issue #4581 · containerd/containerd

sudo rm /etc/containerd/config.toml
sudo systemctl restart containerd
sudo kubeadm init