pod内访问kubernetes API server

深碍√TFBOYSˉ_ 2023-07-22 11:29 21阅读 0赞

本文接上一篇介绍如何从pod内部访问kubernetes API server。 所有的pod默认都关联上一个serviceAccount，只要该serviceAccount有权限访问你访问的资源对象，就可以直接访问。

我们使用default namespace中default serviceAccount， 提前创建了一个role并将default和该role进行roleBinding。

# 前提 #

先创建role， kubectl create -f role.yaml

[root@node1 yqyaml]# cat role.yaml
    kind: Role
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      namespace: default
      name: pod-reader
    rules:
      - apiGroups: [""] # The API group "" indicates the core API Group.
        resources:
        - configmaps
        - secrets
        - nodes
        - nodes/metrics
        - nodes/stats
        - nodes/log
        - nodes/spec
        - nodes/proxy
        - pods
        - services
        - resourcequotas
        - replicationcontrollers
        - limitranges
        - persistentvolumeclaims
        - persistentvolumes
        - namespaces
        - endpoints
        - proxy
        verbs:
        - list
        - watch
        - get
      - apiGroups:
        - extensions
        resources:
        - daemonsets
        - deployments
        - replicasets
        - ingresses
        verbs:
        - list
        - watch
      - apiGroups:
        - apps
        resources:
        - statefulsets
        - daemonsets
        - deployments
        - replicasets
        verbs:
        - list
        - watch
      - apiGroups:
        - batch
        resources:
        - cronjobs
        - jobs
        verbs:
        - list
        - watch
      - apiGroups:
        - autoscaling
        resources:
        - horizontalpodautoscalers
        verbs:
        - list
        - watch
      - apiGroups:
        - authentication.k8s.io
        resources:
        - tokenreviews
        verbs:
        - create
      - apiGroups:
        - authorization.k8s.io
        resources:
        - subjectaccessreviews
        verbs:
        - create
        nonResourceURLs: []
    [root@node1 yqyaml]#

然后绑定role和serviceAccount  
kubectl create -f roleBinding1.yaml

[root@node1 yqyaml]# cat roleBinding1.yaml
    # This role binding allows "test-deri" to read pods in the namespace "test-deri"
    kind: RoleBinding
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      name: default-role-binding
      namespace: default
    subjects:
      - kind: ServiceAccount # May be "User", "Group" or "ServiceAccount"
        name: default
    roleRef:
      kind: Role
      name: pod-reader
      apiGroup: rbac.authorization.k8s.io
    [root@node1 yqyaml]#

最后检查绑定关系. 检查已经创建的role和roleBinding

[root@node1 yqyaml]# kubectl get serviceaccount
    NAME         SECRETS   AGE
    default      1         18d
    myaccount2   1         8h
    test1        1         4h58m
    yqaccount    1         44h
    [root@node1 yqyaml]# kubectl get role
    NAME          AGE
    pod-reader    45h
    pod-reader2   45h
    [root@node1 yqyaml]# kubectl get rolebinding
    NAME                      AGE
    default-role-binding      45h
    default-role-binding2     44h
    yqaccount-role-binding2   44h
    [root@node1 yqyaml]#

解释：我的kubernetes上default空间已经创建很多role和serviceAccount，你只需要确保刚才提到的role pod-reader 和 rolebinding default-role-binding 存在就行。

# 简单的带有curl的镜像（非必须） #

我先制作了一个简单的镜像，镜像只有curl工具, 可以直接使用其他任何带有curl工具的镜像。略过本步

Dockerfile文件

FROM alpine:latest
    RUN apk add --update curl && rm -rf /var/cache/apk/*

然后执行命令（在Dockerfile文件所在目录执行， 后者指定Dockerfile文件路径）

docker build -t  local/curl:v1 .

执行完成后，查看生成的镜像：

[root@node1 dockerFiles]# docker images -a|grep local
    local/curl                                           v1                  fa8e179516ee        46 hours ago        6.95MB
    [root@node1 dockerFiles]#

# pod yaml文件 #

可以发下明我意的pod中镜像文件是local/curl:v1， 就是刚才生成的镜像。 它的运行命令是 \[ “/bin/sh”, “-c”, “sleep 60000” \]

[root@node1 yqyaml]# cat curlPodWithEnv.yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: curl-yqaccount-meta-status-pod
    spec:
      containers:
        - name: test-container
          image: local/curl:v1
          command: [ "/bin/sh", "-c", "sleep 60000" ]
          env:
            - name: SPECIAL_METADATA_NAME
              valueFrom:
                fieldRef:
                 fieldPath: metadata.name
            - name: SPECIAL_HOST_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
      restartPolicy: Never
    [root@node1 yqyaml]#

# 访问 API Server #

1， 先进入的pod的容器中

kubectl exec -it { podName}  /bin/sh

2， 执行 如下命令获得token  
token位置通过kubectl describe \{podName\} 就能看到。

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70]

TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)

3， 访问API server  
其中API server的地址和端口可以通过env命令获取

curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer $TOKEN" -s  https://10.1.0.1:443/api/v1/namespaces/default/pods/

需要强调的是因为我的role配置的不同，导致不能直接访问https://10.1.0.1:443， 但是可以访问https://10.1.0.1:443/api/v1/namespaces/default/pods/

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70 1]

# curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt -H "Authorization: Bearer $TOKEN" -s https://10.1.0.1:443/api
    { 
      "kind": "APIVersions",
      "versions": [
        "v1"
      ],
      "serverAddressByClientCIDRs": [
        { 
          "clientCIDR": "0.0.0.0/0",
          "serverAddress": "192.168.119.142:6443"
        }
      ]
    }/ #

还可以在pod内删除其他pod

curl -s  https://10.1.0.1/api/v1/namespaces/default/pods/{ xxxpod} -X DELETE  -H "Authorization: Bearer $TOKEN"    --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70 2]  
提示：我的default serviceAccount绑定了两个role，另外一个role是pod-reader2

[root@node1 yqyaml]# cat simpleRole.yaml
    kind: Role
    apiVersion: rbac.authorization.k8s.io/v1beta1
    metadata:
      namespace: default
      name: pod-reader2
    rules:
      - apiGroups:
        - '*'
        resources:
        - '*'
        verbs:
        - '*'

如取消关联pod-reader2， 按我的default serviceaccount可以访问获取pod list但是不能删除，可以参考如下结果  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70 3]

# curl #

curl的使用方法，请参考阮一峰老师的博客，http://www.ruanyifeng.com/blog/2019/09/curl-reference.html

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70]: /images/20230528/96f4faa51d4a4fc2a318f99c8925e484.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70 1]: /images/20230528/cb1d5c149e7044c19f15ac725f25e4bd.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70 2]: /images/20230528/a70fa920d8b848248b59c2d5b90f7fd5.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3J1c3NsZQ_size_16_color_FFFFFF_t_70 3]: /images/20230528/a84caaa15bd04f86b07f67a6e946cdc8.png