采用rancher2+kubernetes+skywalking部署springcloud项目(四[istio服务网格化版本])

柔情只为你懂 2023-02-20 07:24 42阅读 0赞

**本系列文章目录**  
[(一)基础k8s yaml脚本发布][k8s yaml]  
[(二)helm+shell脚本优化大量冗余配置发布][helm_shell]  
[(三)jenkins用户审核的流水化方式部署][jenkins]  
[(四)service mesh(istio)服务网格化发布][service mesh_istio]  
[ (五)istio对项目进行金丝雀部署][_istio]

*原本这篇记录上周就该完成的，奈何种种原因再加上电脑配置低，环境跑着跑着就卡住了，后面把虚拟机全换到固态硬盘后环境稍微不卡，才将spring-boot-cloud全部署到了istio中*

下面为**通过istio来部署spring-boot-cloud项目的主要过程**

先来个部署好了后的kiali效果图：  
![spring-boot-cloud效果图][spring-boot-cloud]

# istio #

*istio作为一个非常热门的ServiceMesh框架,身为JAVA小弟的我为了避免后面有身边的朋友或同事聊到它时而我听不懂这个玩意而尴尬，所以特此决定来学习一把*

对于istio的介绍和学习教程，基本上通过istio官网来看就行了,地址如下：

[https://istio.io/][https_istio.io]  
![istio][]

在将spring-boot-cloud项目部署到istio之前，我也常常思考这些个问题：

*  在有了如此强大的kubernetes之后，为什么还有istio这个东西呢？
 *  istio解决了什么问题?
 *  istio解决了kubernetes中哪些不能解决的问题?

一头雾水的我在看了一些关于介绍istio的文章后感觉仍然是有些迷糊

如果你跟我一样仍然对istio有什么困惑的话，建议可以和我一样先将上面这些个问题搁在一边，先把istio官网中的[BookInfo项目][BookInfo]跑一遍再说

# 几种常见的部署方式 #

在了解istio的时候，可以顺带了解下以下几个概念

*  蛮力部署
 *  金丝雀部署
 *  滚动发布
 *  蓝绿部署
 *  A/B测试

具体的可以参考这个文章：https://www.cnblogs.com/apanly/p/8784096.html

上面的那个链接的文章写得很好，还有配图也很方便理解，以下是我对上面几种常见部署的方式的理解

*  蛮力部署(个人理解：类似于tomcat下换war包，会中断服务)
 *  金丝雀部署(个人理解：类似于在nginx下挂几个tomcat，tomcat中有一个是新版本的war包，其他tomcat下都是老版本的war包,如果新版本的war包测试通过后，就将其余tomcat下的war包也换成新版本的war包)
 *  滚动发布(个人理解：与金丝雀发布类似,一个金丝雀没有问题后继续发下一个金丝雀)
 *  蓝绿部署（个人理解：准备好两套环境，系统指向哪个环境由前面的网关来指定,网络切换的感觉-双服务器组）
 *  A/B测试（个人理解：根据客户端或用户的属性分阶段切换完成部署)

# istio安装 #

直接参考官网就行了  
https://istio.io/latest/docs/setup/getting-started/

如果网络好的话，执行执行curl命令下载就行

> curl -L https://istio.io/downloadIstio | sh -

网络不好的话，可以通过istio的github进行下载，我这里是通过链接下载当前最新版本的istio的：https://github.com/istio/istio/releases/tag/1.6.3

将istio下载好后，先解压，然后切换到istio目录，再将环境变量临时设置一下

[root@k8s-node1 istio-1.6.3]# pwd
    /root/istio/istio-1.6.3
    [root@k8s-node1 istio-1.6.3]# export PATH=$PWD/bin:$PATH
    [root@k8s-node1 istio-1.6.3]#

然后根据官网的方式进行安装

> istioctl install --set profile=demo

参数后面的profile说明：https://istio.io/latest/docs/setup/additional-setup/config-profiles/  
![istio-profiles][]

## 访问kiali ##

通过demo的profile安装好istio后，istio中会默认安装kiali,但默认情况下kiali没有暴露端口出来，想访问kiali的话是访问不到了

root@k8s-node1 istio-1.6.3]# kubectl get svc -n istio-system
    NAME                        TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                                                                      AGE
    grafana                     ClusterIP      10.43.118.62    <none>        3000/TCP                                                                     10m
    istio-egressgateway         ClusterIP      10.43.254.39    <none>        80/TCP,443/TCP,15443/TCP                                                     10m
    istio-ingressgateway        LoadBalancer   10.43.84.8      <pending>     15020:31026/TCP,80:31304/TCP,443:31640/TCP,31400:31249/TCP,15443:32596/TCP   10m
    istiod                      ClusterIP      10.43.199.240   <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP,53/UDP,853/TCP                         11m
    jaeger-agent                ClusterIP      None            <none>        5775/UDP,6831/UDP,6832/UDP                                                   10m
    jaeger-collector            ClusterIP      10.43.122.109   <none>        14267/TCP,14268/TCP,14250/TCP                                                10m
    jaeger-collector-headless   ClusterIP      None            <none>        14250/TCP                                                                    10m
    jaeger-query                ClusterIP      10.43.185.90    <none>        16686/TCP                                                                    10m
    kiali                       ClusterIP      10.43.182.219   <none>        20001/TCP                                                                    10m
    prometheus                  ClusterIP      10.43.37.102    <none>        9090/TCP                                                                     10m
    tracing                     ClusterIP      10.43.246.95    <none>        80/TCP                                                                       10m
    zipkin                      ClusterIP      10.43.201.182   <none>        9411/TCP                                                                     10m

在istio中想将某个服务暴露出来，我们这里可以配一个istio的gateway

gateway的介绍地址：

https://istio.io/latest/docs/reference/config/networking/gateway/

virtualService的介绍地址：  
https://istio.io/latest/docs/reference/config/networking/virtual-service/

apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: kiali-gateway
      namespace: istio-system
    spec:
      selector:
        istio: ingressgateway # use istio default controller
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "*"
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: kiali
      namespace: istio-system
    spec:
      hosts:
      - "*"
      gateways:
      - kiali-gateway
      http:
      - route:
        - destination:
            host: kiali
            port:
              number: 20001

执行kiali-gateway创建

[root@k8s-node1 istio-1.6.3]# kubectl apply -f kiali-gateway.yaml 
    gateway.networking.istio.io/kiali-gateway created
    virtualservice.networking.istio.io/kiali created
    [root@k8s-node1 istio-1.6.3]# kubectl get gateways -A
    NAMESPACE      NAME            AGE
    istio-system   kiali-gateway   21s
    [root@k8s-node1 istio-1.6.3]#

通过上面get svc命令可以看出istio-ingressgateway的端口为：31304

那么进行访问：http://192.168.113.147:31304/  
![kiali首页][kiali]  
输入默认的kiali登录名和密码:admin/  
![kiali登录后首页][kiali 1]

# 部署spring-boot-cloud项目 #

在istio装好后，接下来就是部署spring-boot-cloud了。

## 开启自动注入sidecar ##

开启namespace的istio自动注入功能

> kubectl label namespace default istio-injection=enabled

然后再确定下将要部署项目的namespace开启了istio-injection没有,  
使用命令：

> kubectl get namespace -L istio-injection

比如这里的：

[root@localhost spring-boot-cloud]# kubectl get namespace -L istio-injection
    NAME                   STATUS   AGE   ISTIO-INJECTION
    cattle-prometheus      Active   10d   
    cattle-system          Active   72d   
    default                Active   72d   enabled
    ingress-nginx          Active   72d   
    istio-system           Active   9d    disabled
    kube-node-lease        Active   72d   
    kube-public            Active   72d   
    kube-system            Active   72d   
    kubernetes-dashboard   Active   60d   
    my-space               Active   71d   
    security-scan          Active   72d

## 通过yaml部署spring-boot-cloud基础项目 ##

执行spring-boot-cloud的yaml前需要先将rabbitmq和skywalking安装好.

rabbitmq:  
https://raw.githubusercontent.com/puhaiyang/spring-boot-cloud/master/rabbitmq.yaml

skywalking:  
https://raw.githubusercontent.com/puhaiyang/spring-boot-cloud/master/skywalking-oap-server-with-h2.yaml

然后是spring-boot-cloud的yaml部署  
spring-boot-cloud.yaml:

#-------------registry-----------------
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: registry
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: registry
      strategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate
      template:
        metadata:
          labels:
            app: registry
        spec:
          initContainers:
            - name: init-skywalking-agent
              image: ccr.ccs.tencentyun.com/haiyang/skywalking-agent:7.0.0
              command:
                - 'sh'
                - '-c'
                - 'set -ex;mkdir -p /vmskywalking/agent;cp -r /skywalking/agent/* /vmskywalking/agent;'
              volumeMounts:
                - mountPath: /vmskywalking/agent
                  name: skywalking-agent
          containers:
            - image: ccr.ccs.tencentyun.com/spring-boot-cloud/registry:latest
              imagePullPolicy: Always
              name: registry
              ports:
                - containerPort: 8761
                  protocol: TCP
              volumeMounts:
                - mountPath: /opt/skywalking/agent
                  name: skywalking-agent
          volumes:
            - name: skywalking-agent
              emptyDir: {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: registry
    spec:
      ports:
        - name: http
          port: 8761
          protocol: TCP
          targetPort: 8761
      selector:
        app: registry
    ---
    #-------------config-----------------
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: config
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: config
      strategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate
      template:
        metadata:
          labels:
            app: config
        spec:
          initContainers:
            - name: init-skywalking-agent
              image: ccr.ccs.tencentyun.com/haiyang/skywalking-agent:7.0.0
              command:
                - 'sh'
                - '-c'
                - 'set -ex;mkdir -p /vmskywalking/agent;cp -r /skywalking/agent/* /vmskywalking/agent;'
              volumeMounts:
                - mountPath: /vmskywalking/agent
                  name: skywalking-agent
          containers:
            - image: ccr.ccs.tencentyun.com/spring-boot-cloud/config:latest
              imagePullPolicy: Always
              name: config
              ports:
                - containerPort: 8888
                  protocol: TCP
              volumeMounts:
                - mountPath: /opt/skywalking/agent
                  name: skywalking-agent
          volumes:
            - name: skywalking-agent
              emptyDir: {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: config
    spec:
      ports:
        - name: http
          port: 8888
          protocol: TCP
          targetPort: 8888
      selector:
        app: config
    #-------------svca-service-----------------
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: svca-service
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: svca-service
      strategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate
      template:
        metadata:
          labels:
            app: svca-service
        spec:
          initContainers:
            - name: init-skywalking-agent
              image: ccr.ccs.tencentyun.com/haiyang/skywalking-agent:7.0.0
              command:
                - 'sh'
                - '-c'
                - 'set -ex;mkdir -p /vmskywalking/agent;cp -r /skywalking/agent/* /vmskywalking/agent;'
              volumeMounts:
                - mountPath: /vmskywalking/agent
                  name: skywalking-agent
          containers:
            - image: ccr.ccs.tencentyun.com/spring-boot-cloud/svca-service:latest
              imagePullPolicy: Always
              name: svca-service
              ports:
                - containerPort: 8080
                  protocol: TCP
              volumeMounts:
                - mountPath: /opt/skywalking/agent
                  name: skywalking-agent
          volumes:
            - name: skywalking-agent
              emptyDir: {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: svca-service
    spec:
      ports:
        - name: http
          port: 8080
          protocol: TCP
          targetPort: 8080
      selector:
        app: svca-service
    #-------------svcb-service-----------------
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: svcb-service
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: svcb-service
      strategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate
      template:
        metadata:
          labels:
            app: svcb-service
        spec:
          initContainers:
            - name: init-skywalking-agent
              image: ccr.ccs.tencentyun.com/haiyang/skywalking-agent:7.0.0
              command:
                - 'sh'
                - '-c'
                - 'set -ex;mkdir -p /vmskywalking/agent;cp -r /skywalking/agent/* /vmskywalking/agent;'
              volumeMounts:
                - mountPath: /vmskywalking/agent
                  name: skywalking-agent
          containers:
            - image: ccr.ccs.tencentyun.com/spring-boot-cloud/svcb-service:latest
              imagePullPolicy: Always
              name: svcb-service
              ports:
                - containerPort: 8070
                  protocol: TCP
              volumeMounts:
                - mountPath: /opt/skywalking/agent
                  name: skywalking-agent
          volumes:
            - name: skywalking-agent
              emptyDir: {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: svcb-service
    spec:
      ports:
        - name: http
          port: 8070
          protocol: TCP
          targetPort: 8070
      selector:
        app: svcb-service
    #-------------auth-service-----------------
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: auth-service
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: auth-service
      strategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate
      template:
        metadata:
          labels:
            app: auth-service
        spec:
          initContainers:
            - name: init-skywalking-agent
              image: ccr.ccs.tencentyun.com/haiyang/skywalking-agent:7.0.0
              command:
                - 'sh'
                - '-c'
                - 'set -ex;mkdir -p /vmskywalking/agent;cp -r /skywalking/agent/* /vmskywalking/agent;'
              volumeMounts:
                - mountPath: /vmskywalking/agent
                  name: skywalking-agent
          containers:
            - image: ccr.ccs.tencentyun.com/spring-boot-cloud/auth-service:latest
              imagePullPolicy: Always
              name: auth-service
              ports:
                - containerPort: 5000
                  protocol: TCP
              volumeMounts:
                - mountPath: /opt/skywalking/agent
                  name: skywalking-agent
          volumes:
            - name: skywalking-agent
              emptyDir: {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: auth-service
    spec:
      ports:
        - name: http
          port: 5000
          protocol: TCP
          targetPort: 5000
      selector:
        app: auth-service
    #-------------gateway-----------------
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: gateway
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: gateway
      strategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate
      template:
        metadata:
          labels:
            app: gateway
        spec:
          initContainers:
            - name: init-skywalking-agent
              image: ccr.ccs.tencentyun.com/haiyang/skywalking-agent:7.0.0
              command:
                - 'sh'
                - '-c'
                - 'set -ex;mkdir -p /vmskywalking/agent;cp -r /skywalking/agent/* /vmskywalking/agent;'
              volumeMounts:
                - mountPath: /vmskywalking/agent
                  name: skywalking-agent
          containers:
            - image: ccr.ccs.tencentyun.com/spring-boot-cloud/gateway:latest
              imagePullPolicy: Always
              name: gateway
              ports:
                - containerPort: 8060
                  protocol: TCP
              env:
                - name: SKYWALKING_ADDR
                  value: skywalking-oap-server
                - name: APP_ID
                  value: gateway
              volumeMounts:
                - mountPath: /opt/skywalking/agent
                  name: skywalking-agent
          volumes:
            - name: skywalking-agent
              emptyDir: {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: gateway
    spec:
      ports:
        - name: http
          port: 8060
          protocol: TCP
          targetPort: 8060
      selector:
        app: gateway
    ---

将上面的内容执行下即可，与第一篇文章[采用rancher2+kubernetes+skywalking部署springcloud项目(一\[k8s yaml版本\])][rancher2_kubernetes_skywalking_springcloud_k8s yaml]相比，去掉了monitor和zipkin以及ingress

## 创建gateway与virtualService ##

与之前全用k8s部署spring-boot-cloud不同的是，istio对外访问需要使用gateway与virtualservice来对外暴露

virtual-service的参考文档为：  
https://istio.io/latest/docs/reference/config/networking/virtual-service/

其中对于gateways有如下的说明：

> The names of gateways and sidecars that should apply these routes. Gateways in other namespaces may be referred to by /; specifying a gateway with no namespace qualifier is the same as specifying the VirtualService’s namespace.

也就是说如果gateway和virtualService没有在同一个namespace的话，那么gateway所在的namespace必须写出来

而此处istio中的gateway是通过demo的profile来安装的，它默认安在了istio-system的namespace下，所以在下面的gateway中需要写上istio-system

apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: registry-gateway
      namespace: istio-system
    spec:
      selector:
        istio: ingressgateway # use istio default controller
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "registry.springcloud.com"
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: registry
    spec:
      hosts:
      - "registry.springcloud.com"
      gateways:
      - istio-system/registry-gateway #can omit the namespace if gateway is in same namespace as virtual service
      http:
      - route:
        - destination:
            host: registry
            port:
              number: 8761
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: gateway-gateway
      namespace: istio-system
    spec:
      selector:
        istio: ingressgateway # use istio default controller
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "gateway.springcloud.com"
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: gateway
    spec:
      hosts:
      - "gateway.springcloud.com"
      gateways:
      - istio-system/gateway-gateway #can omit the namespace if gateway is in same namespace as virtual service
      http:
      - route:
        - destination:
            host: gateway
            port:
              number: 8060
    ---

以及暴露出skyalking的界面

apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: skywalking-ui
      labels:
        app: skywalking-ui
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: skywalking-ui
      template:
        metadata:
          labels:
            app: skywalking-ui
        spec:
          containers:
            - name: skywalking-ui
              image: apache/skywalking-ui:latest
              imagePullPolicy: Always
              ports:
                - containerPort: 8080
                  name: httpport
              env:
                - name: SW_OAP_ADDRESS
                  value: skywalking-oap-server:12800
    ---
    #-----------------定义skywalking-ui的代理服务--------------
    apiVersion: v1
    kind: Service
    metadata:
      name: skywalking-ui
      labels:
        service: skywalking-ui
    spec:
      ports:
        - port: 8080
          name: httpport
          targetPort: 8080
      type: ClusterIP
      selector:
        app: skywalking-ui
    ---

添加对应的gateway

---
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      name: skywalking-ui-gateway
      namespace: istio-system
    spec:
      selector:
        istio: ingressgateway # use istio default controller
      servers:
      - port:
          number: 80
          name: http
          protocol: HTTP
        hosts:
        - "skywalking.springcloud.com"
    ---
    apiVersion: networking.istio.io/v1alpha3
    kind: VirtualService
    metadata:
      name: skywalking-ui
    spec:
      hosts:
      - "skywalking.springcloud.com"
      gateways:
      - istio-system/skywalking-ui-gateway #can omit the namespace if gateway is in same namespace as virtual service
      http:
      - route:
        - destination:
            host: skywalking-ui
            port:
              number: 8080
    ---

*因为istio网络的特殊性，对于skywalking这里的部署方式有个问题，这里先不进行深究。如果想在istio中用skywalking的话，直接参看skywalking官网中的adapter就可以了*

# 查看istio中spring-boot-cloud的监控 #

一切ok后开打kiali的界面查看下  
![kiali界面][kiali 2]  
如果从仅从这里看istio的特点的话，可能还是看不出它与k8s的区别，下一篇将会通过**使用istio对spring-boot-cloud项目进行金丝雀发布**以加深对istio实际中的作用，从而来对serviceMesh进行进一步的了解

[k8s yaml]: https://blog.csdn.net/puhaiyang/article/details/105738550
[helm_shell]: https://blog.csdn.net/puhaiyang/article/details/106925885
[jenkins]: https://blog.csdn.net/puhaiyang/article/details/106934404
[service mesh_istio]: https://blog.csdn.net/puhaiyang/article/details/106948850
[_istio]: https://blog.csdn.net/puhaiyang/article/details/107206635
[spring-boot-cloud]: https://img-blog.csdnimg.cn/20200630230017528.gif#pic_center
[https_istio.io]: https://istio.io/
[istio]: https://img-blog.csdnimg.cn/20200624174948575.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3B1aGFpeWFuZw==,size_16,color_FFFFFF,t_70#pic_center
[BookInfo]: https://istio.io/latest/docs/examples/bookinfo/
[istio-profiles]: https://img-blog.csdnimg.cn/20200624175152331.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3B1aGFpeWFuZw==,size_16,color_FFFFFF,t_70
[kiali]: https://img-blog.csdnimg.cn/20200624175232932.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3B1aGFpeWFuZw==,size_16,color_FFFFFF,t_70
[kiali 1]: https://img-blog.csdnimg.cn/20200624175455700.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3B1aGFpeWFuZw==,size_16,color_FFFFFF,t_70
[rancher2_kubernetes_skywalking_springcloud_k8s yaml]: https://puhaiyang.blog.csdn.net/article/details/105738550
[kiali 2]: https://img-blog.csdnimg.cn/20200630224159509.gif#pic_center