1. 规划

使用 kubeadm 安装 Kubernetes 集群并使用 containerd 作为容器运行时（container runtime）是一种常见的安装方法。

OS	配置	用途	备注
aliOS(172.17.197.69)	2核(vCPU) 4GiB 5 Mbps	k8s-master1	乌兰察布可用区 C
aliOS(172.17.197.68)	4核(vCPU) 10 GiB 5 Mbps GPU：NVIDIA T4/8	k8s-worker1	乌兰察布可用区 C

注：这是演示 k8s 集群安装的实验环境，配置较低，生产环境中我们的服务器配置至少都是 8C/16G 的基础配置。

2. 版本选择

Alibaba Cloud Linux 3.2104 LTS 64位

2.1. 配置主机名

所有节点执行

[root@k8s-master1 ~]# hostnamectl set-hostname k8s-master1
[root@k8s-worker1 ~]# hostnamectl set-hostname k8s-worker1

2.2. 关闭防火墙

所有节点执行

 # 关闭firewalld
2  [root@k8s-master1 ~]# systemctl stop firewalld
3  
4  # 关闭selinux
5  [root@k8s-master1 ~]# sed -i 's/enforcing/disabled/' /etc/selinux/config
6  [root@k8s-master1 ~]# setenforce 0
7   setenforce: SELinux is disabled
8  [root@k8s-master1 ~]#

2.3. 互做本地解析

所有节点执行

root@k8s-master1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
8.130.117.184 k8s-master1
8.130.92.114  k8s-worker1
[root@k8s-master1 ~]#

2.4. SSH 免密通信（可选）

master节点执行

root@k8s-master1 ~]# ssh-keygen

互发公钥

[root@k8s-master1 ~]# ssh-copy-id root@k8s-worker1

2.5. 加载 br_netfilter 模块

确保 br_netfilter 模块被加载 > 所有节点执行

# 加载模块
root@k8s-master1 ~]# modprobe br_netfilter
## 查看加载请看
[root@k8s-master1 ~]# lsmod | grep br_netfilter
br_netfilter           32768  0
bridge                270336  1 br_netfilter

# 永久生效
[root@k8s-master1 ~]# cat <<EOF | tee /etc/modules-load.d/k8s.conf
> br_netfilter
> EOF
br_netfilter
[root@k8s-master1 ~]#

2.6. 允许 iptables 检查桥接流量

所有节点执行

[root@k8s-master1 ~]# cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
> net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1
> EOF
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
[root@k8s-master1 ~]# sysctl --system

2.7. 关闭 swap

所有节点执行

# 临时关闭
[root@k8s-master1 ~]# swapoff -a

# 永久关闭
[root@k8s-master1 ~]# sed -ri 's/.*swap.*/#&/' /etc/fstab

2.8. 安装ipset及ipvsadm

安装ipset及ipvsadm

[root@k8s-master1 ~]# dnf install -y ipset ipvsadm

配置ipvsadm模块加载方式，添加需要加载的模块

$ cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack
EOF

授权、运行、检查是否加载

$ chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack

3. 安装containerd

所有节点执行

3.1. 安装 containerd

通过 yum 安装 containerd：

root@k8s-master1 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
Adding repo from: http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
[root@k8s-master1 ~]# yum install containerd -y

3.2. 配置 containerd

$ /usr/bin/containerd config default > /etc/containerd/config.toml
$ sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml
$ sed -i 's/sandbox_image = "registry.k8s.io\/pause:3.6"/sandbox_image = "registry.aliyuncs.com\/google_containers\/pause:3.9"/g' /etc/containerd/config.toml

注：记得确保配置修改成功，尤其是pause版本。

3.3. 启动containerd

启动并使 containerd 随系统启动：

root@k8s-master1 ~]# sudo systemctl start containerd
root@k8s-master1 ~]# sudo systemctl enable containerd
root@k8s-master1 ~]# ps -ef | grep containerd
root       37760       1  0 07:35 ?        00:00:00 /usr/bin/containerd
root       37807   36202  0 07:35 pts/0    00:00:00 grep --color=auto containerd
[root@k8s-master1 ~]#

说明containerd已启动成功。

4. 安装 kubeadm、kubelet

所有节点执行

4.1. 添加 k8s 镜像源

地址： https://developer.aliyun.com/mirror/kubernetes?spm=a2c6h.13651102.0.0.1cd01b116JYQIn

cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes-new/core/stable/v1.28/rpm/repodata/repomd.xml.key
EOF
setenforce 0
yum install -y --nogpgcheck kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet
echo 1 > /proc/sys/net/ipv4/ip_forward

4.2. 建立 k8s YUM 缓存

所有节点执行

[root@k8s-master1 ~]# yum makecache
alinux3-os                                                                                       1.3 MB/s | 3.8 kB     00:00    
alinux3-updates                                                                                  1.4 MB/s | 4.1 kB     00:00    
alinux3-module                                                                                   354 kB/s | 4.2 kB     00:00    
alinux3-plus                                                                                     645 kB/s | 3.0 kB     00:00    
alinux3-powertools                                                                               598 kB/s | 3.0 kB     00:00    
Extra Packages for Enterprise Linux 8 - x86_64                                                   1.8 MB/s | 4.4 kB     00:00    
Extra Packages for Enterprise Linux Modular 8 - x86_64                                           620 kB/s | 3.0 kB     00:00    
Kubernetes                                                                                        17 kB/s | 1.7 kB     00:00    
Metadata cache created.
[root@k8s-master1 ~]#

4.3. 安装 k8s 相关工具

所有节点执行

[root@k8s-master1 ~]# yum list kubelet --showduplicates
Last metadata expiration check: 0:01:46 ago on Wed 15 Jan 2025 10:55:44 PM CST.
Installed Packages
kubelet.x86_64                                           1.28.15-150500.1.1                                           @kubernetes
Available Packages
kubelet.aarch64                                          1.28.0-150500.1.1                                            kubernetes 
kubelet.ppc64le                                          1.28.0-150500.1.1                                            kubernetes 
kubelet.s390x                                            1.28.0-150500.1.1                                            kubernetes 
kubelet.src                                              1.28.0-150500.1.1                                            kubernetes 
kubelet.x86_64                                           1.28.0-150500.1.1                                            kubernetes 
kubelet.aarch64                                          1.28.1-150500.1.1                                            kubernetes 
kubelet.ppc64le                                          1.28.1-150500.1.1                                            kubernetes 
kubelet.s390x                                            1.28.1-150500.1.1                                            kubernetes 
kubelet.src                                              1.28.1-150500.1.1                                            kubernetes 
.....

设置crictl连接 containerd

$ crictl config --set runtime-endpoint=unix:///run/containerd/containerd.sock
$ crictl images ls

设置kubectl命令自动补全：

echo 'source <(kubectl completion bash)' >> ~/.bashrc
source ~/.bashrc

5. master节点

5.1. k8s 初始化

Master 节点执行

注：记得修改–apiserver-advertise-address与–kubernetes-version修改正确。

root@k8s-master ~]# kubeadm init \
  --apiserver-advertise-address=172.17.197.69 \
  --image-repository registry.aliyuncs.com/google_containers \
  --kubernetes-version v1.28.15 \
  --service-cidr=10.96.0.0/12 \
  --pod-network-cidr=10.244.0.0/16 \
  --ignore-preflight-errors=all \
  --cri-socket /var/run/containerd/containerd.sock

参数说明：

   --apiserver-advertise-address  # 集群master地址
   --image-repository             # 指定k8s镜像仓库地址
   --kubernetes-version           # 指定K8s版本（与kubeadm、kubelet版本保持一致）
   --service-cidr                 # Pod统一访问入口
   --pod-network-cidr             # Pod网络（与CNI网络保持一致）

初始化后输出如下内容，说明成功。

.....
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 172.17.197.69:6443 --token ipqfom.r9ltq4t3in53cd6w \
        --discovery-token-ca-cert-hash sha256:0f01242802eae4b69408c34070a3ad8d017229faba9f8b30623ed1e9dd69d66c

5.2. 根据输出提示创建相关文件

master节点上执行

[root@k8s-master ~]# mkdir -p $HOME/.kube
[root@k8s-master ~]# cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-master ~]# chown $(id -u):$(id -g) $HOME/.kube/config
[root@k8s-master ~]#

5.3. 查看 k8s 运行的容器

master节点上执行

[root@k8s-master1 ~]# kubectl  get pod -A -o wide
NAMESPACE     NAME                                  READY   STATUS    RESTARTS   AGE   IP              NODE          NOMINATED NODE   READINESS GATES
kube-system   coredns-66f779496c-lq5rj              0/1     Pending   0          26m   <none>          <none>        <none>           <none>
kube-system   coredns-66f779496c-q5pxk              0/1     Pending   0          26m   <none>          <none>        <none>           <none>
kube-system   etcd-k8s-master1                      1/1     Running   1          26m   172.17.197.69   k8s-master1   <none>           <none>
kube-system   kube-apiserver-k8s-master1            1/1     Running   1          26m   172.17.197.69   k8s-master1   <none>           <none>
kube-system   kube-controller-manager-k8s-master1   1/1     Running   1          26m   172.17.197.69   k8s-master1   <none>           <none>
kube-system   kube-proxy-88trd                      1/1     Running   0          26m   172.17.197.69   k8s-master1   <none>           <none>
kube-system   kube-scheduler-k8s-master1            1/1     Running   1          26m   172.17.197.69   k8s-master1   <none>           <none>
[root@k8s-master1 ~]#

5.4. 查看 k8s 节点

master节点上执行

root@k8s-master1 ~]# kubectl  get node -o wide
NAME          STATUS     ROLES           AGE   VERSION    INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                              KERNEL-VERSION           CONTAINER-RUNTIME
k8s-master1   NotReady   control-plane   36m   v1.28.15   172.17.197.69   <none>        Alibaba Cloud Linux 3.2104 U11 (OpenAnolis Edition)   5.10.134-18.al8.x86_64   containerd://1.6.32
[root@k8s-master1 ~]#

可看到当前只有 k8s-master 节点，而且状态是 NotReady（未就绪），因为我们还没有部署网络插件（kubectl apply -f [podnetwork].yaml），于是接着部署容器网络（CNI）。

5.5. 容器网络（CNI）部署

Master 节点执行

插件地址：https://kubernetes.io/docs/concepts/cluster-administration/addons/

该地址在 k8s-master 初始化成功时打印出来。

5.5.1. 选择一个主流的容器网络插件部署（Calico）

5.5.2. 下载yml文件

wget https://docs.projectcalico.org/manifests/calico.yaml --no-check-certificate

5.5.3. 看看该yaml文件所需要启动的容器

[root@k8s-master1 install-k8s-cluster]# cat calico.yaml |grep image
          image: docker.io/calico/cni:v3.25.0
          imagePullPolicy: IfNotPresent
          image: docker.io/calico/cni:v3.25.0
          imagePullPolicy: IfNotPresent
          image: docker.io/calico/node:v3.25.0
          imagePullPolicy: IfNotPresent
          image: docker.io/calico/node:v3.25.0
          imagePullPolicy: IfNotPresent
          image: docker.io/calico/kube-controllers:v3.25.0
          imagePullPolicy: IfNotPresent
[root@k8s-master1 install-k8s-cluster]#

5.5.4. 修改为国内可下载镜像

使用docker.m.daocloud.io代替docker.io

sed -i s/docker.io/docker.m.daocloud.io/g calico.yaml 
$ cat calico.yaml | grep image

5.5.5. 安装calico

[root@k8s-master1 calicodir]# kubectl  apply -f calico.yaml

5.5.6. 查看pod和node状态正常

[root@k8s-master1 calicodir]# kubectl  get pod -A
[root@k8s-master1 calicodir]# kubectl  get node

可以看出pod已全部Running， node已变成Ready状态，说明master节点已安装成功完成。

6. worker 节点

6.1. worker 节点加入 k8s 集群

所有 work 节点执行

复制k8s-master初始化屏幕输出的语句并在work节点执行

root@k8s-worker1 ~]# kubeadm join 172.17.197.69:6443 --token ipqfom.r9ltq4t3in53cd6w         --discovery-token-ca-cert-hash sha256:0f01242802eae4b69408c34070a3ad8d017229faba9f8b30623ed1e9dd69d66c
[preflight] Running pre-flight checks
        [WARNING FileExisting-tc]: tc not found in system path
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

6.2. 查询集群信息

Master节点上执行

[root@k8s-master1 calicodir]# kubectl  get pod -A -o wide
[root@k8s-master1 calicodir]# kubectl  get node -o wide

查看crictl和ctr是否安装成功。

[root@k8s-master1 calicodir]# crictl version
[root@k8s-master1 calicodir]# ctr version

7. 验证

Master节点上执行

k8s 集群部署 nginx 服务，并通过浏览器进行访问验证。

7.1. 创建Pod

[root@k8s-master1 calicodir]# kubectl create deployment nginx --image=docker.m.daocloud.io/library/nginx:latest
deployment.apps/nginx created
[root@k8s-master1 calicodir]# kubectl expose deployment nginx --port=80 --type=NodePort
service/nginx exposed
[root@k8s-master1 calicodir]# kubectl  get all
NAME                        READY   STATUS    RESTARTS   AGE
pod/nginx-fbf584587-pp6ks   1/1     Running   0          2m45s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        5h45m
service/nginx        NodePort    10.110.34.104   <none>        80:30884/TCP   28s

NAME                    READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx   1/1     1            1           2m45s

NAME                              DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-fbf584587   1         1         1       2m45s
[root@k8s-master1 calicodir]#

7.2. 访问nginx

7.2.1. curl访问

[root@k8s-master1 calicodir]# curl 10.110.34.104:80
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
[root@k8s-master1 calicodir]#

使用nodePort方式访问现象同上

至此：kubeadm方式的k8s集群已经部署完成。

微信扫一扫

作者：mospan
微信关注：墨斯潘園
本文出处：http://mospany.github.io/2024/01/15/kubeadm-install-k8s/
文章版权归本人所有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。

墨斯潘園

技术改变生活，学习成就未来

K8S项目实践(07): kubeadm安装k8s集群(containerd版)

1. 规划

2. 版本选择

2.1. 配置主机名

2.2. 关闭防火墙

2.3. 互做本地解析

2.4. SSH 免密通信（可选）

2.5. 加载 br_netfilter 模块

2.6. 允许 iptables 检查桥接流量

2.7. 关闭 swap

2.8. 安装ipset及ipvsadm

3. 安装containerd

3.1. 安装 containerd

3.2. 配置 containerd

3.3. 启动containerd

4. 安装 kubeadm、kubelet

4.1. 添加 k8s 镜像源

4.2. 建立 k8s YUM 缓存

4.3. 安装 k8s 相关工具

5. master节点

5.1. k8s 初始化

5.2. 根据输出提示创建相关文件

5.3. 查看 k8s 运行的容器

5.4. 查看 k8s 节点

5.5. 容器网络（CNI）部署

5.5.1. 选择一个主流的容器网络插件部署（Calico）

5.5.2. 下载yml文件

5.5.3. 看看该yaml文件所需要启动的容器

5.5.4. 修改为国内可下载镜像

5.5.5. 安装calico

5.5.6. 查看pod和node状态正常

6. worker 节点

6.1. worker 节点加入 k8s 集群

6.2. 查询集群信息

7. 验证

7.1. 创建Pod

7.2. 访问nginx

7.2.1. curl访问