kubernetes v1.18.5 集群安装说明

超、凢脫俗 2023-02-25 02:10 40阅读 0赞

### 前言： ###

>          Kubernetes（简称k8s），是一款目前最受欢迎的容器编排工具，由于阿里云**registry.aliyuncs.com/google\_containers**镜像地址库没有同步最新版本，为了方便后面集群搭建，我自己在[阿里云容器镜像服务][Link 1]弄了一个公开的**registry.cn-shanghai.aliyuncs.com/k8sgcrio\_containers**镜像地址库。

###  环境准备： ###

虚拟机安装或实体机安装：

[CentOS7精简版最小安装镜像（CentOS-7-x86\_64-Minimal-2009.iso）][CentOS7_CentOS-7-x86_64-Minimal-2009.iso]，我是用[Windows10自带的Hyper-V虚拟机平台搭建][Windows10_Hyper-V]的。也推荐大家使用，我觉得比我以前用的VMWare更方便。至少不需要额外去下载软件安装包。这点非常香。

<table style="width:607px;"> 
 <tbody> 
  <tr> 
   <td style="width:157px;">IP</td> 
   <td style="width:156px;">HOSTNAME</td> 
   <td style="width:140px;">CPU</td> 
   <td style="width:150px;">MEM</td> 
  </tr> 
  <tr> 
   <td style="width:157px;">192.168.1.100</td> 
   <td style="width:156px;">master</td> 
   <td style="width:140px;">&gt;=4C</td> 
   <td style="width:150px;">&gt;=8G</td> 
  </tr> 
  <tr> 
   <td style="width:157px;">192.168.1.101</td> 
   <td style="width:156px;">node01</td> 
   <td style="width:140px;">&gt;=2C</td> 
   <td style="width:150px;">&gt;=4G</td> 
  </tr> 
  <tr> 
   <td style="width:157px;">192.168.1.102</td> 
   <td style="width:156px;">node02</td> 
   <td style="width:140px;">&gt;=2C</td> 
   <td style="width:150px;">&gt;=4G</td> 
  </tr> 
 </tbody> 
</table>

### 进入正题： ###

1.设置主机名,其他主机照着葫芦画瓢,如果大家在安装的时候已经设置好相关hostname请忽略这一步。

hostnamectl set-hostname master

2.修改每台hosts文件，文件内容如下：

cat <<EOF >>/etc/hosts
    192.168.1.100 master
    192.168.1.101 node01
    192.168.1.102 node02
    EOF

3.关闭每台机器防火墙,postfix，selinux，swap。

systemctl disable firewalld && systemctl stop firewalld
    systemctl disable postfix && systemctl stop postfix
    setenforce 0 && sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
    swapoff -a && sed -i 's/.*swap.*/#&/' /etc/fstab

4.将桥接的IPv4流量传递到iptables的链【有一些ipv4的流量不能走iptables链【linux内核的一个过滤器，每个流量都会经过他，然后再匹配是否可进入当前应用进程去处理】，导致流量丢失】：配置k8s.conf文件（\#k8s.conf文件原来不存在，需要自己创建的）

cat > /etc/sysctl.d/k8s.conf <<EOF
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1
    EOF
    sysctl --system

5.配置CentOS国内阿里云源,安装所需软件包。其他节点同样操作一遍。

cd /etc/yum.repos.d/
    mkdir backup
    mv ./*.repo backup
    curl -O http://mirrors.aliyun.com/repo/Centos-7.repo
    curl -O https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
    cat <<EOF > /etc/yum.repos.d/kubernetes.repo
    [kubernetes]
    name=Kubernetes
    baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
    enabled=1
    gpgcheck=0
    EOF
    yum clean all && yum makecache
    yum -y install docker-ce docker-ce-cli containerd.io kubelet kubeadm kubectl
    systemctl enable docker --now
    cat > /etc/docker/daemon.json <<EOF
    {
      "exec-opts": ["native.cgroupdriver=systemd"],
      "log-driver": "json-file",
      "log-opts": {
        "max-size": "100m"
      },
      "storage-driver": "overlay2",
      "storage-opts": [
        "overlay2.override_kernel_check=true"
      ]
    }
    EOF
    mkdir -p /etc/systemd/system/docker.service.d
    systemctl daemon-reload && systemctl restart docker
    systemctl enable kubelet --now

6.在master节点初始化集群

kubeadm init --kubernetes-version=1.18.5 --apiserver-advertise-address=192.168.1.100  --service-cidr=10.1.0.0/16 --pod-network-cidr=10.244.0.0/16 --image-repository registry.cn-shanghai.aliyuncs.com/k8sgcrio_containers

输出内容：我这里忽略部分内容：

Your Kubernetes control-plane has initialized successfully!
    
    To start using your cluster, you need to run the following as a regular user:
    
      mkdir -p $HOME/.kube
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
    You should now deploy a pod network to the cluster.
    Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
      https://kubernetes.io/docs/concepts/cluster-administration/addons/
    
    Then you can join any number of worker nodes by running the following on each as root:
    
    kubeadm join 192.168.1.100:6443 --token lovamu.c224giryp09os8nb \
        --discovery-token-ca-cert-hash sha256:e1c74dd7a50f39c8e78012e422ef603d70d7af191cca7e1269f83dc5fc37189f

大家可以看到此时master节点初始化成功了。然后它告诉我们配置基础的kubectl 配置文件。直接复制它提示的命令即可完成后续步骤。

kubectl 配置文件：

mkdir -p $HOME/.kube
    cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    chown $(id -u):$(id -g) $HOME/.kube/config

[部署网络文件][Link 2]，我们首选([flannel][])：

大家可以尝试访问下这个链接：[https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml][https_raw.githubusercontent.com_coreos_flannel_master_Documentation_kube-flannel.yml]

不行的话复制我贴的内容吧！

kube-flannel.yml

---
    apiVersion: policy/v1beta1
    kind: PodSecurityPolicy
    metadata:
      name: psp.flannel.unprivileged
      annotations:
        seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default
        seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default
        apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default
        apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default
    spec:
      privileged: false
      volumes:
        - configMap
        - secret
        - emptyDir
        - hostPath
      allowedHostPaths:
        - pathPrefix: "/etc/cni/net.d"
        - pathPrefix: "/etc/kube-flannel"
        - pathPrefix: "/run/flannel"
      readOnlyRootFilesystem: false
      # Users and groups
      runAsUser:
        rule: RunAsAny
      supplementalGroups:
        rule: RunAsAny
      fsGroup:
        rule: RunAsAny
      # Privilege Escalation
      allowPrivilegeEscalation: false
      defaultAllowPrivilegeEscalation: false
      # Capabilities
      allowedCapabilities: ['NET_ADMIN']
      defaultAddCapabilities: []
      requiredDropCapabilities: []
      # Host namespaces
      hostPID: false
      hostIPC: false
      hostNetwork: true
      hostPorts:
      - min: 0
        max: 65535
      # SELinux
      seLinux:
        # SELinux is unused in CaaSP
        rule: 'RunAsAny'
    ---
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      name: flannel
    rules:
      - apiGroups: ['extensions']
        resources: ['podsecuritypolicies']
        verbs: ['use']
        resourceNames: ['psp.flannel.unprivileged']
      - apiGroups:
          - ""
        resources:
          - pods
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - nodes
        verbs:
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - nodes/status
        verbs:
          - patch
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      name: flannel
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: flannel
    subjects:
    - kind: ServiceAccount
      name: flannel
      namespace: kube-system
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: flannel
      namespace: kube-system
    ---
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: kube-flannel-cfg
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    data:
      cni-conf.json: |
        {
          "name": "cbr0",
          "cniVersion": "0.3.1",
          "plugins": [
            {
              "type": "flannel",
              "delegate": {
                "hairpinMode": true,
                "isDefaultGateway": true
              }
            },
            {
              "type": "portmap",
              "capabilities": {
                "portMappings": true
              }
            }
          ]
        }
      net-conf.json: |
        {
          "Network": "10.244.0.0/16",
          "Backend": {
            "Type": "vxlan"
          }
        }
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-amd64
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - amd64
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-amd64
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-amd64
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-arm64
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - arm64
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-arm64
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-arm64
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-arm
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - arm
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-arm
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-arm
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-ppc64le
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - ppc64le
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-ppc64le
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-ppc64le
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: kube-flannel-ds-s390x
      namespace: kube-system
      labels:
        tier: node
        app: flannel
    spec:
      selector:
        matchLabels:
          app: flannel
      template:
        metadata:
          labels:
            tier: node
            app: flannel
        spec:
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                      - key: kubernetes.io/os
                        operator: In
                        values:
                          - linux
                      - key: kubernetes.io/arch
                        operator: In
                        values:
                          - s390x
          hostNetwork: true
          tolerations:
          - operator: Exists
            effect: NoSchedule
          serviceAccountName: flannel
          initContainers:
          - name: install-cni
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-s390x
            command:
            - cp
            args:
            - -f
            - /etc/kube-flannel/cni-conf.json
            - /etc/cni/net.d/10-flannel.conflist
            volumeMounts:
            - name: cni
              mountPath: /etc/cni/net.d
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          containers:
          - name: kube-flannel
            image: registry.cn-shanghai.aliyuncs.com/quayio_containers/coreos-flannel:v0.12.0-s390x
            command:
            - /opt/bin/flanneld
            args:
            - --ip-masq
            - --kube-subnet-mgr
            resources:
              requests:
                cpu: "100m"
                memory: "50Mi"
              limits:
                cpu: "100m"
                memory: "50Mi"
            securityContext:
              privileged: false
              capabilities:
                 add: ["NET_ADMIN"]
            env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            volumeMounts:
            - name: run
              mountPath: /run/flannel
            - name: flannel-cfg
              mountPath: /etc/kube-flannel/
          volumes:
            - name: run
              hostPath:
                path: /run/flannel
            - name: cni
              hostPath:
                path: /etc/cni/net.d
            - name: flannel-cfg
              configMap:
                name: kube-flannel-cfg

kubectl get nodes
    NAME    STATUS     ROLES    AGE   VERSION
    master  NotReady   master   20m   v1.18.5

部署网络文件：

kubectl apply -f kube-flannel.yml

kubectl get nodes
    NAME    STATUS   ROLES    AGE   VERSION
    master   Ready    master   22m   v1.18.5

大家可以看到没有部署网络文件之前master节点状态处于NotReady状态，部署完之后处于Ready状态，证明master节点搭建成功！其他节点join即可。

kubeadm join 192.168.1.100:6443 --token lovamu.c224giryp09os8nb \
        --discovery-token-ca-cert-hash sha256:e1c74dd7a50f39c8e78012e422ef603d70d7af191cca7e1269f83dc5fc37189f

回到master节点继续查看nodes状态,至此搭建成功！大家有什么问题可以私聊我。

kubectl get nodes
    NAME    STATUS   ROLES    AGE   VERSION
    master  Ready    master   22m   v1.18.5
    node01  Ready    <none>   84s   v1.18.5
    node02  Ready    <none>   33s   v1.18.5

我感觉这种命令好多啊，是不是有什么可视化界面来控制和观察集群状况呢？没错，确实有，那就是k8s仪表盘。我会在下一篇介绍如何《[kubernetes部署dashboard][kubernetes_dashboard]》。

[Link 1]: https://cr.console.aliyun.com/
[CentOS7_CentOS-7-x86_64-Minimal-2009.iso]: http://mirrors.aliyun.com/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-2009.iso
[Windows10_Hyper-V]: http://www.cppcns.com/os/linux/241419.html
[Link 2]: https://kubernetes.io/docs/concepts/cluster-administration/addons/
[flannel]: https://github.com/coreos/flannel/blob/master/Documentation/kubernetes.md
[https_raw.githubusercontent.com_coreos_flannel_master_Documentation_kube-flannel.yml]: https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
[kubernetes_dashboard]: https://blog.csdn.net/ZouChengli/article/details/107250788