20_2.服务暴露-ingress (ingress-nginx)

我就是我 2022-10-07 05:45 107阅读 0赞

### 20\_2.服务暴露-ingress (ingress-nginx) ###

#### ingress-nginx ####

https://github.com/kubernetes/ingress-nginx

#### 准备镜像 ####

docker pull quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0

#### 资源配置清单 ####

参考：https://github.com/kubernetes/ingress-nginx/blob/master/deploy/static/provider/cloud/deploy.yaml  
参考：https://hub.fastgit.org/kubernetes/ingress-nginx/blob/nginx-0.30.0/deploy/static/mandatory.yaml

> https://hub.fastgit.org 为github的国内加速代理

##### namespace.yaml #####

apiVersion: v1
    kind: Namespace
    metadata:
      name: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---

##### configmap.yaml #####

kind: ConfigMap
    apiVersion: v1
    metadata:
      name: nginx-configuration
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: tcp-services
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: udp-services
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---

##### deployment.yaml #####

apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-ingress-controller
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          app.kubernetes.io/name: ingress-nginx
          app.kubernetes.io/part-of: ingress-nginx
      template:
        metadata:
          labels:
            app.kubernetes.io/name: ingress-nginx
            app.kubernetes.io/part-of: ingress-nginx
          annotations:
            prometheus.io/port: "10254"
            prometheus.io/scrape: "true"
        spec:
          # wait up to five minutes for the drain of connections
          terminationGracePeriodSeconds: 300
          serviceAccountName: nginx-ingress-serviceaccount
          nodeSelector:
            kubernetes.io/os: linux
          containers:
            - name: nginx-ingress-controller
              image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0
              args:
                - /nginx-ingress-controller
                - --configmap=$(POD_NAMESPACE)/nginx-configuration
                - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
                - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
                - --publish-service=$(POD_NAMESPACE)/ingress-nginx
                - --annotations-prefix=nginx.ingress.kubernetes.io
              securityContext:
                allowPrivilegeEscalation: true
                capabilities:
                  drop:
                    - ALL
                  add:
                    - NET_BIND_SERVICE
                # www-data -> 101
                runAsUser: 101
              env:
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
                - name: POD_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
              ports:
                - name: http
                  containerPort: 80
                - name: https
                  containerPort: 443
              livenessProbe:
                failureThreshold: 3
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                initialDelaySeconds: 10
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 10
              readinessProbe:
                failureThreshold: 3
                httpGet:
                  path: /healthz
                  port: 10254
                  scheme: HTTP
                periodSeconds: 10
                successThreshold: 1
                timeoutSeconds: 10
              lifecycle:
                preStop:
                  exec:
                    command:
                      - /wait-shutdown
    ---

##### service.yaml #####

apiVersion: v1
    kind: Service
    metadata:
      name: ingress-nginx
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    spec:
      type: NodePort
      ports:
        - name: http
          port: 80
          targetPort: 80
          protocol: TCP
          nodePort: 8480  # 设置对外http暴露入口为8480
        - name: https
          port: 443
          targetPort: 443
          protocol: TCP
          nodePort: 8443  # 设置对外https暴露入口为8443
      selector:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---

使用nodePort类型service暴露ingress-nginx服务，指定端口8480和8443，作为集群服务的总入口

> 和traefik一样，前置代理服务器可以配置流量负载到`nodeIP:8480`上，后续通过ingress规则分发业务流量给目标service

##### rbac.yaml #####

apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: nginx-ingress-serviceaccount
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRole
    metadata:
      name: nginx-ingress-clusterrole
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    rules:
      - apiGroups:
          - ""
        resources:
          - configmaps
          - endpoints
          - nodes
          - pods
          - secrets
        verbs:
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - nodes
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - services
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - ""
        resources:
          - events
        verbs:
          - create
          - patch
      - apiGroups:
          - "extensions"
          - "networking.k8s.io"
        resources:
          - ingresses
        verbs:
          - get
          - list
          - watch
      - apiGroups:
          - "extensions"
          - "networking.k8s.io"
        resources:
          - ingresses/status
        verbs:
          - update
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: Role
    metadata:
      name: nginx-ingress-role
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    rules:
      - apiGroups:
          - ""
        resources:
          - configmaps
          - pods
          - secrets
          - namespaces
        verbs:
          - get
      - apiGroups:
          - ""
        resources:
          - configmaps
        resourceNames:
          # Defaults to "<election-id>-<ingress-class>"
          # Here: "<ingress-controller-leader>-<nginx>"
          # This has to be adapted if you change either parameter
          # when launching the nginx-ingress-controller.
          - "ingress-controller-leader-nginx"
        verbs:
          - get
          - update
      - apiGroups:
          - ""
        resources:
          - configmaps
        verbs:
          - create
      - apiGroups:
          - ""
        resources:
          - endpoints
        verbs:
          - get
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: RoleBinding
    metadata:
      name: nginx-ingress-role-nisa-binding
      namespace: ingress-nginx
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: Role
      name: nginx-ingress-role
    subjects:
      - kind: ServiceAccount
        name: nginx-ingress-serviceaccount
        namespace: ingress-nginx
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: nginx-ingress-clusterrole-nisa-binding
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: nginx-ingress-clusterrole
    subjects:
      - kind: ServiceAccount
        name: nginx-ingress-serviceaccount
        namespace: ingress-nginx
    ---

#### 查看部署情况 ####

➜ ~  kubectl get all -n ingress-nginx -o wide
    NAME                                            READY   STATUS    RESTARTS   AGE     IP             NODE            NOMINATED NODE   READINESS GATES
    pod/nginx-ingress-controller-69c8b48fd6-wj7kk   1/1     Running   0          3m15s   172.20.102.3   172.10.10.102   <none>           <none>
    NAME                    TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                    AGE     SELECTOR
    service/ingress-nginx   NodePort   192.168.16.78   <none>        80:8480/TCP,443:8443/TCP   3m10s   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
    NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE     CONTAINERS                 IMAGES                                                 SELECTOR
    deployment.apps/nginx-ingress-controller   1/1     1            1           3m15s   nginx-ingress-controller   harbor.hzwod.com/k8s/nginx-ingress-controller:0.30.0   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
    NAME                                                  DESIRED   CURRENT   READY   AGE     CONTAINERS                 IMAGES                                                 SELECTOR
    replicaset.apps/nginx-ingress-controller-69c8b48fd6   1         1         1       3m15s   nginx-ingress-controller   harbor.hzwod.com/k8s/nginx-ingress-controller:0.30.0   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=69c8b48fd6

`kubelctl apply`上述资源后

➜  nginx-ingress curl http://172.10.10.101:8480
    <html>
    <head><title>404 Not Found</title></head>
    <body>
    <center><h1>404 Not Found</h1></center>
    <hr><center>nginx/1.17.8</center>
    </body>
    </html>

看见了一个nginx？？ 其实就是nginx pod

#### 配置ingress规则 ####

##### 目标服务 #####

➜  nginx-ingress kubectl get all -n default -l name=whoami -o wide
    NAME                          READY   STATUS    RESTARTS   AGE   IP             NODE            NOMINATED NODE   READINESS GATES
    pod/whoami-644f4f96df-f7dcj   1/1     Running   0          12m   172.20.102.3   172.10.10.102   <none>           <none>
    NAME             TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)   AGE   SELECTOR
    service/whoami   ClusterIP   192.168.255.128   <none>        80/TCP    17m   app=myapp,task=whoami

##### ingress规则 #####

whoami\_ingress.yaml

apiVersion: networking.k8s.io/v1beta1
    kind: Ingress
    metadata:
      name: whoamiingress
      annotations:
        kubernetes.io/ingress.class: "nginx"
    spec:
      rules:
      - http:
          paths:
            - path: /bar
              backend:
                serviceName: "whoami"
                servicePort: 80

##### 测试 #####

➜  nginx-ingress kubectl get ingress --all-namespaces
    NAMESPACE   NAME            HOSTS   ADDRESS           PORTS   AGE
    default     whoamiingress   *       192.168.175.255   80      12m
    ➜  nginx-ingress curl http://172.10.10.101:8480/bar
    Hostname: whoami-644f4f96df-f7dcj
    IP: 127.0.0.1
    IP: 172.20.102.3
    RemoteAddr: 172.20.102.4:45984
    GET /bar HTTP/1.1
    Host: 172.10.10.101:8480
    User-Agent: curl/7.29.0
    Accept: */*
    X-Forwarded-For: 172.10.10.101
    X-Forwarded-Host: 172.10.10.101:8480
    X-Forwarded-Port: 80
    X-Forwarded-Proto: http
    X-Real-Ip: 172.10.10.101
    X-Request-Id: 34cbd03ec296a29f6e4918bc2236906e
    X-Scheme: http

实现了通过ingress-nginx入口代理访问到了集群内部whoami的服务