服务网格Istio安装部署

Istio服务网格安装部署

1.使用 Istioctl 安装Istio

1.1、Istio版本选择

版本选择参考：https://istio.io/latest/docs/releases/supported-releases/#support-status-of-istio%02releases

要根据自己k8s集群的版本选择合适的Istio版本：

1
2
3
4
5
6
7
8
9
[root@k8s-master01 ~]# kubectl get nodes
NAME           STATUS   ROLES    AGE   VERSION
k8s-master01   Ready    <none>   76d   v1.31.4
k8s-master02   Ready    <none>   76d   v1.31.4
k8s-master03   Ready    <none>   76d   v1.31.4
k8s-node01     Ready    <none>   76d   v1.31.4
k8s-node02     Ready    <none>   76d   v1.31.4
k8s-node03     Ready    <none>   76d   v1.31.4
#我这里的k8s版本是1.31.4，所以我选择安装的istio版本是1.24

1.2、下载Istio

从github下载Istio安装包：https://github.com/istio/istio/releases/tag/1.24.3

将下载好的安装包上传到k8s集群：

解压软件包：

1
2
3
4
5
6
[root@k8s-master01 26-istio]# tar -zxf istio-1.24.3-linux-amd64.tar.gz
[root@k8s-master01 26-istio]# ll
total 26284
drwxr-x--- 6 root root      115 Feb  6 09:30 istio-1.24.3
-rw-r--r-- 1 root root 26913981 Mar  1 21:16 istio-1.24.3-linux-amd64.tar.gz
-rw-r--r-- 1 root root        0 Mar  1 11:07 virtualService.yaml

将 解压后的 Istio 的客户端工具 istioctl，移动到/usr/local/bin 目录下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
#istioctl在解压目录中的bin目录中
[root@k8s-master01 26-istio]# cd istio-1.24.3/

#解压后的目录中的内容如下：
[root@k8s-master01 istio-1.24.3]# ll
total 28
drwxr-x---  2 root root    22 Feb  6 09:30 bin
-rw-r--r--  1 root root 11357 Feb  6 09:30 LICENSE
drwxr-xr-x  4 root root    36 Feb  6 09:30 manifests
-rw-r-----  1 root root   983 Feb  6 09:30 manifest.yaml
-rw-r--r--  1 root root  6927 Feb  6 09:30 README.md
drwxr-xr-x 27 root root  4096 Feb  6 09:30 samples
drwxr-xr-x  3 root root    57 Feb  6 09:30 tools

#将istio的客户端工具istioctl移动到/usr/local/bin/目录
[root@k8s-master01 istio-1.24.3]# mv bin/istioctl /usr/local/bin/

#查看版本，警告忽略，没启动pod导致的。
[root@k8s-master01 istio-1.24.3]# istioctl version
Istio is not present in the cluster: no running Istio pods in namespace "istio-system"
client version: 1.24.3

1.3、安装Istio

更多 Istio 安装配置：https://istio.io/latest/docs/setup/additional-setup/customize-installation/

之后通过定义 IstioOperator 资源，在 Kubernetes 中安装 Istio：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# 准备IstioOperator资源清单
[root@k8s-master01 26-istio]#  cat istio-operator.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
    name: istio-operator
    namespace: istio-system
spec:
    profile: default
    meshConfig:
        accessLogEncoding: JSON
        accessLogFile: /dev/stdout
    hub: registry.cn-beijing.aliyuncs.com/k8s-liujunwei
    tag: 1.24.3
    components:
        base:
            enabled: true
        pilot:
            enabled: true
        cni:
            enabled: false
        egressGateways:
            - name: istio-egressgateway
              enabled: false
        ingressGateways:
            - name: istio-ingressgateway
              enabled: true
              k8s:
                service:
                  type: NodePort
                  ports:
                  - name: status-port
                    port: 15020
                    nodePort: 30520
                  - name: http2
                    port: 80
                    targetPort: 8080
                    nodePort: 30080
                  - name: https
                    port: 443
                    targetPort: 8443
                    nodePort: 30443

安装Istio:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[root@k8s-master01 26-istio]# istioctl install -f istio-operator.yaml
        |\
        | \
        |  \
        |   \
      /||    \
     / ||     \
    /  ||      \
   /   ||       \
  /    ||        \
 /     ||         \
/______||__________\
____________________
  \__       _____/
     \_____/

WARNING: Istio 1.24.0 may be out of support (EOL) already: see https://istio.io/latest/docs/releases/supported-releases/ for supported releases
This will install the Istio 1.24.3 profile "default" into the cluster. Proceed? (y/N) y
✔ Istio core installed ⛵️
✔ Istiod installed 🧠
✔ Ingress gateways installed 🛬
✔ Installation complete
#看到上述提示信息表示成功安装。

#查看pod运行情况，pod运行在 istio-system命名空间
[root@k8s-master01 26-istio]# kubectl get pod -n istio-system
NAME                                   READY   STATUS    RESTARTS   AGE
istio-ingressgateway-9868bc46c-2tb9v   1/1     Running   0          5m34s
istiod-7ff54f76c8-8llzm                1/1     Running   0          5m42s

可能遇到的报错：（无法从官方拉取镜像）

当报错如上图，解决办法如下：

1
2
3
4
5
6
7
8
9
[root@k8s-master01 ~]# kubectl get deployments.apps  -n istio-system
NAME                   READY   UP-TO-DATE   AVAILABLE   AGE
istio-ingressgateway   1/1     1            1           3h32m
istiod                 1/1     1            1           3h32m

#当执行安装命令时会在istio-system命名空间创建deployment资源，通过edit来修改资源镜像下载地址
例如：
[root@k8s-master01 ~]#  kubectl edit -n istio-system deployments.apps istio-ingressgateway
#修改image参数的下载地址，如下图

查看安装的service和pod：

1
2
3
4
[root@k8s-master01 26-istio]# kubectl get svc -n istio-system
NAME                   TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                                      AGE
istio-ingressgateway   NodePort    10.96.24.231   <none>        15020:30520/TCP,80:30080/TCP,443:30443/TCP   6m34s
istiod                 ClusterIP   10.96.179.64   <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP        6m43s

在安装前已经通过配置文件istio-operator.yaml将service/istio-ingressgateway的类型改为了NodePort，如果没有改的话可以参考下面步骤将修改默认的LoadBalancer改为NodePort

将istio-ingressgateway这个service的LoadBalancer类型改为NodePort

1
2
3
4
5
6
7
8
9
10
11
[root@k8s-master01 ~]# kubectl edit svc -n istio-system istio-ingressgateway
 54     targetPort: 8443
 55   selector:
 56     app: istio-ingressgateway
 57     istio: ingressgateway
 58   sessionAffinity: None
 59   type: NodePort   #将原来的LoadBalancer改为NodePort
 60 status:
 61   loadBalancer: {}

#修改后的内容参考下图

卸载Istio：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
#要从集群中完全卸载 Istio，请运行以下命令：istioctl uninstall --purge  #参数 --purge 标志将删除所有 Istio 资源，包括可能与其他 Istio 控制平面共享的集群范围资源。
[root@k8s-master01 istio-1.24.3]# istioctl uninstall --purge
All Istio resources will be pruned from the cluster
Proceed? (y/N) y
  Removed apps/v1, Kind=Deployment/istiod.istio-system.
  Removed /v1, Kind=Service/istiod.istio-system.
  Removed /v1, Kind=ConfigMap/istio.istio-system.
  Removed /v1, Kind=ConfigMap/istio-sidecar-injector.istio-system.
  Removed /v1, Kind=Pod/istiod-6cc57f9897-gjj6c.istio-system.
  Removed /v1, Kind=ServiceAccount/istio-reader-service-account.istio-system.
  Removed /v1, Kind=ServiceAccount/istiod.istio-system.
  Removed rbac.authorization.k8s.io/v1, Kind=RoleBinding/istiod.istio-system.
  Removed rbac.authorization.k8s.io/v1, Kind=Role/istiod.istio-system.
  Removed policy/v1, Kind=PodDisruptionBudget/istiod.istio-system.
  Removed autoscaling/v2, Kind=HorizontalPodAutoscaler/istiod.istio-system.
  Removed admissionregistration.k8s.io/v1, Kind=MutatingWebhookConfiguration/istio-sidecar-injector..
  Removed admissionregistration.k8s.io/v1, Kind=ValidatingWebhookConfiguration/istio-validator-istio-system..
  Removed rbac.authorization.k8s.io/v1, Kind=ClusterRole/istio-reader-clusterrole-istio-system..
  Removed rbac.authorization.k8s.io/v1, Kind=ClusterRole/istiod-clusterrole-istio-system..
  Removed rbac.authorization.k8s.io/v1, Kind=ClusterRole/istiod-gateway-controller-istio-system..
  Removed rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding/istio-reader-clusterrole-istio-system..
  Removed rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding/istiod-clusterrole-istio-system..
  Removed rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding/istiod-gateway-controller-istio-system..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/authorizationpolicies.security.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/destinationrules.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/envoyfilters.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/gateways.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/peerauthentications.security.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/proxyconfigs.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/requestauthentications.security.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/serviceentries.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/sidecars.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/telemetries.telemetry.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/virtualservices.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/wasmplugins.extensions.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/workloadentries.networking.istio.io..
  Removed apiextensions.k8s.io/v1, Kind=CustomResourceDefinition/workloadgroups.networking.istio.io..
✔ Uninstall complete

1.4、Istio生产级高可用配置

Istio 的 istiod 和 ingressgateway 安装后，只有一个副本，生产环境建议两个以上的服务。 istio-ingressgateway和istiod是通过hpa来控制副本数的，可以通过如下配置更改 istiod 和 ingressgateway 的最小和最大副本数：

1
2
3
4
5
# 可以看到通过cpu来控制扩缩容，最小1个副本，最多5个副本，这属于默认配置
[root@k8s-master01 26-istio]# kubectl get hpa -n istio-system
NAME                   REFERENCE                         TARGETS       MINPODS   MAXPODS   REPLICAS   AGE
istio-ingressgateway   Deployment/istio-ingressgateway   cpu: 3%/80%   1         5         1          14m
istiod                 Deployment/istiod                 cpu: 0%/80%   1         5         1          14m

修改hpa的配置不能通过edit hpa来编辑，要编辑配置文件istio-operator.yaml来修改：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# 修改istio-operator.yaml文件来达到生产级别的配置，也可以在安装时直接用该配置
[root@k8s-master01 26-istio]# cat istio-operator.yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
    name: istio-operator
    namespace: istio-system
spec:
    profile: default
    meshConfig:
        accessLogEncoding: JSON
        accessLogFile: /dev/stdout
    hub: registry.cn-beijing.aliyuncs.com/k8s-liujunwei
    tag: 1.24.3
    components:
        base:
            enabled: true
        pilot:
            enabled: true
            k8s:
                resources:
                    limits:  #最高资源配置
                        memory: 2Gi  #生产环境建议给4Gi，资源充足可以配置8Gi
                        cpu: "2"  #生产环境建议给2核，资源充足可以配置4核
                    requests:  #最低的资源配置
                        memory: 512Mi  #生产环境建议给4Gi
                        cpu: "100m"   #生产环境建议给2核
                hpaSpec:
                    minReplicas: 2   #不配置默认是1
                    maxReplicas: 5   #不配置默认是5，5个足以应对高并发
        cni:
            enabled: false
        egressGateways:
            - name: istio-egressgateway
              enabled: false
        ingressGateways:
            - name: istio-ingressgateway
              enabled: true
              k8s:
                hpaSpec:
                    minReplicas: 2   #不配置默认是1
                    maxReplicas: 5   #不配置默认是5，5个足以应对高并发
                resources:
                    requests:
                        memory: 2Gi  #生产环境建议给4Gi，资源充足可以配置8Gi
                        cpu: "2"  #生产环境建议给2核，资源充足可以配置4核
                    limits:
                        memory: 512Mi  #生产环境建议给4Gi
                        cpu: "100m"   #生产环境建议给2核
                service:
                  type: NodePort
                  ports:
                  - name: status-port
                    port: 15020
                    nodePort: 30520
                  - name: http2
                    port: 80
                    targetPort: 8080
                    nodePort: 30080
                  - name: https
                    port: 443
                    targetPort: 8443
                    nodePort: 30443

更新配置：

1
2
3
4
5
6
7
[root@k8s-master01 26-istio]# istioctl upgrade -f istio-operator.yaml
WARNING: Istio 1.24.0 may be out of support (EOL) already: see https://istio.io/latest/docs/releases/supported-releases/ for supported releases
This will install the Istio 1.24.3 profile "default" into the cluster. Proceed? (y/N) y
✔ Istio core installed ⛵️
✔ Istiod installed 🧠
✔ Ingress gateways installed 🛬
✔ Installation complete

更新后查看pod副本数：

1
2
3
4
5
6
[root@k8s-master01 26-istio]# kubectl get po -n istio-system
NAME                                    READY   STATUS    RESTARTS   AGE
istio-ingressgateway-59c85d84cb-7dpnf   1/1     Running   0          19s
istio-ingressgateway-59c85d84cb-dgzqq   1/1     Running   0          19s
istiod-d55fd77f9-79vs8                  1/1     Running   0          57s
istiod-d55fd77f9-hqm6r                  1/1     Running   0          67s

2.配置自动注入Sidecar

2.1开启自动注入功能

修改 APIServer 的配置文件，添加 MutatingAdmissionWebhook,ValidatingAdmissionWebhook（如果 K8s 版本大于 1.16 默认已经开启）：

1
2
3
4
5
6
7
8
#kubeadmin安装的集群路径是/etc/kubernetes/manifests/kube-apiserver.yaml 
#二进制安装方式需要找到APIServer的Service 文件路径是/usr/lib/systemd/system/kube-apiserver.service

 - --enable-admission-plugins= MutatingAdmissionWebhook,ValidatingAdmissionWebhook # 本示例省略了其它配置项，在原本基础上追加这两项即可。
 
# 重启服务
systemctl daemon-reload
systemctl restart kube-apiserver

我的k8s集群版本是v1.31.4，默认已经开启了，这里就不需要上述配置了。

2.2自动注入方式

在Istio中，为Pod设置自动注入主要有两种方式：

①命名空间级别的注入

②为特定的资源单独注入

第一种方法（命名空间级别，最常用）：

使用kubectl命令给想要启用自动注入的命名空间添加istio-injection=enabled标签，之后，所有在该命名空间创建的新Pod都会自动注入Istio sidecar

1
2
# 为istio-test命名空间添加自动注入的标签istio-injection=enabled
kubectl label namespace istio-test istio-injection=enabled

第二种方法（Pod级别）：

在Deployment中定义pod的模板中（spec.template.metadata.labels） 添加sidecar.istio.io/inject: “true”标签，来实现为特定的pod注入。这样可以在不启用命名空间级别注入的情况下，选择性地为特定Pod注入sidecar**(不支持pod资源，要在高级资源中配置，如：Deployment或StatefulSet)**

1
2
3
4
5
6
7
8
9
10
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
        sidecar.istio.io/inject: "true"  #添加该注释启用自动注入，该注释为固定参数

2.2.1命名空间级别注入

案例演示：

需求：为istio-test命名空间中的pod配置自动注入

步骤如下：

①：首先确保 Istio 已经正确安装在你的 Kubernetes 集群中，并且 Istio 控制平面 (istiod) 正在运行。

检查命令：kubectl get pods -n istio-system

②：为istio-test命名空间添加标签

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#创建命名空间（如果存在就不用创建）
[root@k8s-master01 istio-1.24.3]# kubectl create ns istio-test
namespace/istio-test created

#查看命名空间是否存在
[root@k8s-master01 istio-1.24.3]# kubectl get ns istio-test
NAME         STATUS   AGE
istio-test   Active   12s

#为istio-test 命名空间添加 istio-injection=enabled 标签。
[root@k8s-master01 istio-1.24.3]# kubectl label ns istio-test istio-injection=enabled
namespace/istio-test labeled

#检查istio-test命名空间的标签
[root@k8s-master01 istio-1.24.3]# kubectl get ns istio-test --show-labels |grep istio-injection=enabled
istio-test   Active   3m17s   istio-injection=enabled,kubernetes.io/metadata.name=istio-test

③：现在，在这个命名空间中部署应用时，Istio 会自动注入 Sidecar。创建deployment资源：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
cat nginx-pod.yaml 
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: registry.cn-beijing.aliyuncs.com/k8s-liujunwei/nginx
        ports:
        - containerPort: 80

④创建资源

1
2
3
4
5
6
[root@k8s-master01 26-istio]# kubectl create -f nginx-pod.yaml -n istio-test
deployment.apps/nginx-deployment created
[root@k8s-master01 26-istio]# kubectl get po -n istio-test
NAME                                READY   STATUS     RESTARTS   AGE
nginx-deployment-754764cd45-6krsq   0/2     Init:0/1   0          6s
nginx-deployment-754764cd45-vm4w6   2/2     Running    0          6s

⑤验证 Sidecar 注入：

1
2
3
4
5
6
7
8
9
10
11
12
#查看pod中容器的数量来判断，如果READY列是2/2说明有两个容器，注入成功。
[root@k8s-master01 26-istio]# kubectl get po -n istio-test
NAME                                READY   STATUS     RESTARTS   AGE
nginx-deployment-754764cd45-6krsq   0/2     Init:0/1   0          6s
nginx-deployment-754764cd45-vm4w6   2/2     Running    0          6s

#也可以查看 Pod 的 YAML 描述：
[root@k8s-master01 26-istio]# kubectl get pod -n istio-test nginx-deployment-754764cd45-6krsq -o yaml
#会看到除了nginx容器外还有一个名为istio-proxy的容器，这就是 Istio 注入的 Sidecar。
    image: registry.cn-beijing.aliyuncs.com/k8s-liujunwei/proxyv2:1.24.3
    imagePullPolicy: IfNotPresent
    name: istio-proxy

2.2.2不要某个pod自动注入

在2.2.1中，默认创建在istio-test命名空间中的pod都会被自动注入，如果想要某个服务的pod不自动注入可以通过sidecar.istio.io/inject: “false”标签来取消自动注入，配置如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 在deploymetn级别资源中，定义pod的配置中添加标签sidecar.istio.io/inject: "false" 
[root@k8s-master01 26-istio]# cat nginx-deploy-notsidecar.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-deployment
  namespace: istio-test

  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
        sidecar.istio.io/inject: "false"   #添加该标签来取消该pod的自动注入
    spec:
      containers:
      - name: nginx
        image: registry.cn-beijing.aliyuncs.com/k8s-liujunwei/nginx
        ports:
        - containerPort: 80

创建资源，观察pod中容器的个数：

1
2
3
4
5
6
7
8
# 创建deployment
[root@k8s-master01 26-istio]# kubectl create -f nginx-deploy-notsidecar.yaml
deployment.apps/nginx-deployment created

#查看pod资源，此时pod中只有一个容器，READY是1/1
[root@k8s-master01 26-istio]# kubectl get po -n istio-test
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-5748b548dc-5d9ll   1/1     Running   0          28s

2.2.2pod级别注入

需求：只为default命名空间中的nginx-deploy这个deployment资源配置自动注入

步骤如下：在资源的yaml文件中，定义pod模板中添加sidecar.istio.io/inject: “true”标签

①配置资源yaml，为pod添加sidecar.istio.io/inject: “true”标签

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
vim nginx-deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels: #在定义pod的标签中添加 sidecar.istio.io/inject: "true"即可
        app: nginx
        sidecar.istio.io/inject: "true" #在标签中配置该参数
    spec:
      containers:
      - name: nginx
        image: registry.cn-beijing.aliyuncs.com/k8s-liujunwei/nginx
        ports:
        - containerPort: 80

②创建资源并验证

1
2
3
4
5
6
7
8
9
#创建这个deployment资源
[root@k8s-master01 26-istio]# kubectl create -f nginx-deploy.yaml
deployment.apps/nginx-deployment created
[root@k8s-master01 26-istio]# kubectl get pod
NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-5f78544f4b-6qmhn   1/2     Running   0          4s

#可以看到上述pod中READY列是2/2，说明有两个容器，已经成功注入。
#kubectl describe po nginx-deployment-5f78544f4b-6qmhn也能查到，创建了两个容器，如下图：

2.3关闭自动注入

如果想要关闭Namespace的自动注入，直接去除命名空间的标签的即可，已经注入的pod不受影响，只有新建或者在下次重建时，pod中不会再有istio-proxy的容器

1
2
3
4
# 删除前sleep-b745f884b-25gvf 中还是两个容器
[root@k8s-master01 sleep]# kubectl get po -n istio-test
NAME                                READY   STATUS    RESTARTS   AGE
sleep-b745f884b-25gvf               2/2     Running   0          7s

删除命名空间自动注入的标签：

1
2
3
# 删除命名空间istio-test的标签(istio-injection=enabled)
[root@k8s-master01 sleep]# kubectl label ns istio-test istio-injection-
namespace/istio-test unlabeled

查看pod，此时之前注入的pod不会受影响：

1
2
3
[root@k8s-master01 sleep]# kubectl get po -n istio-test
NAME                                READY   STATUS    RESTARTS   AGE
sleep-b745f884b-25gvf               2/2     Running   0          3m46s

更新pod，删除重建后的pod不在自动注入：

1
2
3
4
5
6
7
[root@k8s-master01 sleep]# kubectl delete po -n istio-test sleep-b745f884b-25gvf
pod "sleep-b745f884b-25gvf" deleted

# 可以看到删除重建的pod就不会自动注入了
[root@k8s-master01 sleep]# kubectl get po -n istio-test
NAME                                READY   STATUS    RESTARTS   AGE
sleep-b745f884b-nms8k               1/1     Running   0          7s

3.安装可视化工具Kiali

Kiali 为 Istio 提供了可视化的界面，可以在 Kiali 上进行观测流量的走向、调用链，同时还可以使用 Kiali 进行配置管理，给用户带来了很好的体验。

在kubernetes中安装Kiali工具，首先进入到Istio的安装包目录：

1
2
3
4
5
6
7
8
9
10
11
#进入istio的安装包目录，istio-1.24.3目录是上述1.2步骤中下载的软件包解压后生成的目录
[root@k8s-master01 istio-1.24.3]# cd /root/26-istio/istio-1.24.3
[root@k8s-master01 istio-1.24.3]# ll
total 28
drwxr-x---  2 root root     6 Mar  1 21:20 bin
-rw-r--r--  1 root root 11357 Feb  6 09:30 LICENSE
drwxr-xr-x  4 root root    36 Feb  6 09:30 manifests
-rw-r-----  1 root root   983 Feb  6 09:30 manifest.yaml
-rw-r--r--  1 root root  6927 Feb  6 09:30 README.md
drwxr-xr-x 27 root root  4096 Feb  6 09:30 samples
drwxr-xr-x  3 root root    57 Feb  6 09:30 tools

创建之前先修改一下镜像的下载地址：

1
2
3
4
5
6
7
vim /root/26-istio/istio-1.24.3/samples/addons/kiali.yaml
#大概347行
      containers:
      - image: "registry.cn-beijing.aliyuncs.com/k8s-liujunwei/kiali:v2.0"
        imagePullPolicy: IfNotPresent
        name: kiali
#将image中配置的镜像下载地址改为自己的仓库地址，前提是已经将镜像同步到了自己的仓库中，修改后的内容参考下图：

同时修改Service的类型，改为NodePort，也可以创建后edit资源编辑：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# 找到文件的Service资源，在spec下添加type: NodePort
apiVersion: v1
kind: Service
metadata:
  name: kiali
  namespace: "istio-system"
  labels:
    helm.sh/chart: kiali-server-2.0.0
    app: kiali
    app.kubernetes.io/name: kiali
    app.kubernetes.io/instance: kiali
    version: "v2.0.0"
    app.kubernetes.io/version: "v2.0.0"
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/part-of: "kiali"
  annotations:
spec:
  type: NodePort   #默认是ClusterIP，添加该参数把默认类型改为NodePort
  ports:
  - name: http
    appProtocol: http
    protocol: TCP
    port: 20001
  - name: http-metrics
    appProtocol: http
    protocol: TCP
    port: 9090
  selector:
    app.kubernetes.io/name: kiali
    app.kubernetes.io/instance: kiali

创建kiali服务：

1
2
3
4
5
6
7
[root@k8s-master01 istio-1.24.3]# kubectl create -f  samples/addons/kiali.yaml
serviceaccount/kiali created
configmap/kiali created
clusterrole.rbac.authorization.k8s.io/kiali created
clusterrolebinding.rbac.authorization.k8s.io/kiali created
service/kiali created
deployment.apps/kiali created

查看服务部署状态：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@k8s-master01 istio-1.24.3]# kubectl get svc,deployments,pod -n istio-system
NAME                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                      AGE
service/istio-ingressgateway   NodePort    10.96.156.156   <none>        15021:32272/TCP,80:32662/TCP,443:32330/TCP   28h
service/istiod                 ClusterIP   10.96.236.19    <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP        28h
service/kiali                  ClusterIP   10.96.232.129   <none>        20001/TCP,9090/TCP                           4m12s

NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/istio-ingressgateway   1/1     1            1           28h
deployment.apps/istiod                 1/1     1            1           28h
deployment.apps/kiali                  1/1     1            1           4m12s

NAME                                        READY   STATUS    RESTARTS   AGE
pod/istio-ingressgateway-59cb447668-5b5nx   1/1     Running   0          28h
pod/istiod-7ff54f76c8-rpj8p                 1/1     Running   0          28h
pod/kiali-95db6dc65-94zsr                   1/1     Running   0          4m12s

通过edit编辑Service ，将 Service 类型改成 NodePort，或者配置 Ingress 即可从外部访问 Kiali web控制台：

1
2
3
4
5
6
7
8
9
10
11
#这里为了方便直接把kiali的Service 类型改成 NodePort
[root@k8s-master01 istio-1.24.3]# kubectl edit  svc -n istio-system kiali
#大概在45行，将原来的type: ClusterIP改为type: NodePort
 41   selector:
 42     app.kubernetes.io/instance: kiali
 43     app.kubernetes.io/name: kiali
 44   sessionAffinity: None
 45   type: NodePort
 46 status:
 47   loadBalancer: {}
#修改后的内容参考下图显示

查看svc是否被修改：

1
2
[root@k8s-master01 istio-1.24.3]# kubectl get svc -n istio-system
#通过下图可以看到kiali的service类型已经被修改为NodePort

通过20001映射的30716即可访问到web界面：（任意一台k8s宿主机ip:30716）

除了 Kiali 之外，还需要一个链路追踪的工具，安装该工具可以在 Kiali 的 Workloads 页面，查看某个服务的 Traces 信息。直接安装即可：

1
2
3
4
5
6
7
8
9
10
11
12
#同样先修改镜像的下载地址
[root@k8s-master01 istio-1.24.3]# vim samples/addons/jaeger.yaml
#修改23行，把image的值改为registry.cn-beijing.aliyuncs.com/k8s-liujunwei/all-in-one:1.58
 20     spec:
 21       containers:
 22         - name: jaeger
 23           image: "docker.io/jaegertracing/all-in-one:1.58"
 24           env:
 25             - name: BADGER_EPHEMERAL
 26               value: "false"
 27             - name: SPAN_STORAGE_TYPE
 #修改后的内容参考下图：

修改好镜像的下载地址后创建该服务：

1
2
3
4
5
6
#创建服务
[root@k8s-master01 istio-1.24.3]# kubectl create -f samples/addons/jaeger.yaml
deployment.apps/jaeger created
service/tracing created
service/zipkin created
service/jaeger-collector created

查看创建的pod：

1
2
3
4
5
6
[root@k8s-master01 istio-1.24.3]# kubectl get po -n istio-system
NAME                                    READY   STATUS    RESTARTS   AGE
istio-ingressgateway-59cb447668-5b5nx   1/1     Running   0          29h
istiod-7ff54f76c8-rpj8p                 1/1     Running   0          29h
jaeger-78598c75b8-qpqn5                 1/1     Running   0          62s
kiali-95db6dc65-94zsr                   1/1     Running   0          22m

访问链路追踪的工具的web页面需要修改名字是tracing的Service：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# 通过edit直接修改svc的类型
[root@k8s-master01 istio-1.24.3]# kubectl edit  svc -n istio-system tracing
 15 spec:
 16   clusterIP: 10.96.10.101
 17   clusterIPs:
 18   - 10.96.10.101
 19   internalTrafficPolicy: Cluster
 20   ipFamilies:
 21   - IPv4
 22   ipFamilyPolicy: SingleStack
 23   ports:
 24   - name: http-query
 25     port: 80
 26     protocol: TCP
 27     targetPort: 16686
 28   - name: grpc-query
 29     port: 16685
 30     protocol: TCP
 31     targetPort: 16685
 32   selector:
 33     app: jaeger
 34   sessionAffinity: None
 35   type: NodePort   # 将默认值ClusterIP改为NodePort
 36 status:
 37   loadBalancer: {}
 
 # 修改后查看svc，web端通过30107端口访问
 [root@k8s-master01 istio-1.24.3]# kubectl get svc -n istio-system
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                          AGE
istio-ingressgateway   NodePort    10.96.24.231    <none>        15020:30520/TCP,80:30080/TCP,443:30443/TCP       32h
istiod                 ClusterIP   10.96.179.64    <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP            32h
jaeger-collector       ClusterIP   10.96.64.38     <none>        14268/TCP,14250/TCP,9411/TCP,4317/TCP,4318/TCP   70m
kiali                  NodePort    10.96.200.187   <none>        20001:30905/TCP,9090:30288/TCP                   6h31m
tracing                NodePort    10.96.10.101    <none>        80:30107/TCP,16685:31295/TCP                     70m
zipkin                 ClusterIP   10.96.65.213    <none>        9411/TCP

web端通过30107端口访问链路追踪页面，推荐正式环境用Skywalking做链路追踪：

4.安装Prometheus 和Grafana

Istio 默认暴露了很多监控指标，比如请求数量统计、请求持续时间以及 Service 和工作负载的指标，这些指标可以使用 Prometheus 进行收集，Grafana 进行展示。Istio 内置了 Prometheus 和 Grafana 的安装文件，直接安装即可（也可以使用外置的Prometheus 和 Grafana ），推荐使用Istio内置的Prometheus 和 Grafana 。

还是老样子，修改Prometheus 和Grafana 的镜像下载地址：

1
2
3
4
5
6
7
8
9
10
11
12
#Prometheus修改
[root@k8s-master01 istio-1.24.3]# vim samples/addons/prometheus.yaml
#修改497行和525行的镜像地址
495       containers:
496         - name: prometheus-server-configmap-reload
497           image: "registry.cn-beijing.aliyuncs.com/k8s-liujunwei/prometheus-config-reloader:v0.76.0"
498           imagePullPolicy: "IfNotPresent"
...
524         - name: prometheus-server
525           image: "registry.cn-beijing.aliyuncs.com/k8s-liujunwei/prometheus:v2.54.1"
526           imagePullPolicy: "IfNotPresent"
#修改后的内容参考下图

1
2
3
4
5
6
7
8
#Grafana 修改
[root@k8s-master01 istio-1.24.3]# vim samples/addons/grafana.yaml
#修改144行的镜像地址
 142       containers:
 143         - name: grafana
 144           image: "registry.cn-beijing.aliyuncs.com/k8s-liujunwei/grafana:11.2.2-security-01"
 145           imagePullPolicy: IfNotPresent
 #修改后的内容参考下图

修改好后创建资源：

1
2
3
4
5
6
7
8
9
10
11
12
13
[root@k8s-master01 istio-1.24.3]# kubectl create -f samples/addons/prometheus.yaml -f samples/addons/grafana.yaml
serviceaccount/prometheus created
configmap/prometheus created
clusterrole.rbac.authorization.k8s.io/prometheus created
clusterrolebinding.rbac.authorization.k8s.io/prometheus created
service/prometheus created
deployment.apps/prometheus created
serviceaccount/grafana created
configmap/grafana created
service/grafana created
deployment.apps/grafana created
configmap/istio-grafana-dashboards created
configmap/istio-services-grafana-dashboards created

查看创建的pod：

1
2
3
4
5
6
7
8
[root@k8s-master01 istio-1.24.3]# kubectl get po -n istio-system
NAME                                    READY   STATUS    RESTARTS   AGE
grafana-9dd4f7d69-zzr5x                 1/1     Running   0          3m12s
istio-ingressgateway-59cb447668-5b5nx   1/1     Running   0          29h
istiod-7ff54f76c8-rpj8p                 1/1     Running   0          29h
jaeger-78598c75b8-qpqn5                 1/1     Running   0          37m
kiali-95db6dc65-94zsr                   1/1     Running   0          58m
prometheus-5cc54f8878-4j5rp             2/2     Running   0          3m13s

同样的方式，将 Grafana和prometheus 的 Service 改成 NodePort 或者添加 Ingress，之后访问即可：

1
2
3
4
5
6
7
8
9
10
#这里采用修改NodePort方式，grafana的service名称就是grafana
[root@k8s-master01 istio-1.24.3]# kubectl edit svc -n istio-system grafana
 32   selector:
 33     app.kubernetes.io/instance: grafana
 34     app.kubernetes.io/name: grafana
 35   sessionAffinity: None
 36   type: ClusterIP  #将ClusterIP改为NodePort
 37 status:
 38   loadBalancer: {}
 #修改后的内容见下图：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
#这里采用修改NodePort方式，prometheus的service名称就是prometheus
[root@k8s-master01 istio-1.24.3]# kubectl edit svc -n istio-system prometheus
 21 spec:
 22   clusterIP: 10.96.3.25
 23   clusterIPs:
 24   - 10.96.3.25
 25   externalTrafficPolicy: Cluster
 26   internalTrafficPolicy: Cluster
 27   ipFamilies:
 28   - IPv4
 29   ipFamilyPolicy: SingleStack
 30   ports:
 31   - name: http
 32     nodePort: 31585
 33     port: 9090
 34     protocol: TCP
 35     targetPort: 9090
 36   selector:
 37     app.kubernetes.io/component: server
 38     app.kubernetes.io/instance: prometheus
 39     app.kubernetes.io/name: prometheus
 40   sessionAffinity: None
 41   type: ClusterIP   #将ClusterIP改为NodePort
 42 status:
 43   loadBalancer: {}
 #修改后的内容见下图：

查看service暴露的端口：

1
2
3
4
5
6
[root@k8s-master01 istio-1.24.3]# kubectl get svc -n istio-system grafana
NAME      TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
grafana   NodePort   10.96.35.228   <none>        3000:32648/TCP   3m56s
[root@k8s-master01 istio-1.24.3]# kubectl get svc -n istio-system prometheus
NAME         TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)          AGE
prometheus   NodePort   10.96.3.25   <none>        9090:31585/TCP   4m

可以看到grafana的9090端口已经映射为32648，用32648即可访问到web页面

可以看到prometheus的3000端口已经映射为31585，用31585即可访问到web页面