Linux - K8S - 调度策略 - Node调度

互联网 2022/1/17 7:08:37

5.调度策略 自定义调度策略 # 自定义一个demo-scheduler的资源策略配置文件 ​ [14:47:54 root@master1 scheduler]#mkdir /etc/kubernetes/scheduler [14:48:39 root@master1 scheduler]#cd /etc/kubernetes/scheduler ​ [14:49:20 root@master1 scheduler]#cat kubesch…
5.调度策略
自定义调度策略
# 自定义一个demo-scheduler的资源策略配置文件
​
[14:47:54 root@master1 scheduler]#mkdir /etc/kubernetes/scheduler
[14:48:39 root@master1 scheduler]#cd /etc/kubernetes/scheduler
​
[14:49:20 root@master1 scheduler]#cat kubeschedulerconfiguration.yaml 
apiVersion: kubescheduler.config.k8s.io/v1beta1
kind: KubeSchedulerConfiguration
clientConnection:
  kubeconfig: "/etc/kubernetes/scheduler.conf"
profiles:
- schedulerName: default-scheduler
- schedulerName: demo-scheduler
  plugins:
    filter:
      disabled:
      - name: NodeUnschedulable
    score:
      disabled:
      - name: NodeResourcesBalancedAllocation
        weight: 1
      - name: NodeResourcesLeastAllocated
        weight: 1
      enabled:
      - name: NodeResourcesMostAllocated
        weight: 5
​
# 应用前面定义的配置文件,注意提前备份好原本的kube-scheduler.yaml
[15:02:09 root@master1 manifests]#diff kube-scheduler.yaml kube-scheduler.yaml-bk 
16d15
<     - --config=/etc/kubernetes/scheduler/kubeschedulerconfiguration.yaml
20c19
<     #- --port=0
---
>     - --port=0
51,53d49
<     - mountPath: /etc/kubernetes/scheduler
<       name: schedconf
<       readOnly: true
64,67d59
<   - hostPath:
<       path: /etc/kubernetes/scheduler
<       type: DirectoryOrCreate
<     name: schedconf
​
# 查看结果
[15:03:56 root@master1 manifests]#kubectl get pod -n kube-system
NAME                                         READY   STATUS    RESTARTS         AGE
calico-kube-controllers-6fb865d84f-4lhbz     1/1     Running   15 (54m ago)     34d
calico-node-7hj44                            1/1     Running   6 (5d20h ago)    34d
calico-node-hk2r2                            1/1     Running   11 (5d20h ago)   34d
calico-node-kmmwm                            1/1     Running   12 (5d20h ago)   34d
calico-node-ns2ff                            1/1     Running   6 (5d20h ago)    34d
calico-node-qv7nn                            1/1     Running   6 (5d20h ago)    34d
coredns-76b4d8bc8f-d69q9                     1/1     Running   12 (5d20h ago)   34d
coredns-76b4d8bc8f-ndsg9                     1/1     Running   12 (5d20h ago)   34d
etcd-master1                                 1/1     Running   13 (5d20h ago)   34d
etcd-master2.noisedu.cn                      1/1     Running   14 (5d20h ago)   34d
etcd-master3.noisedu.cn                      1/1     Running   14 (5d20h ago)   34d
kube-apiserver-master1                       1/1     Running   14 (5d20h ago)   34d
kube-apiserver-master2.noisedu.cn            1/1     Running   15 (5d20h ago)   34d
kube-apiserver-master3.noisedu.cn            1/1     Running   15 (5d20h ago)   34d
kube-controller-manager-master1              1/1     Running   14 (5d20h ago)   34d
kube-controller-manager-master2.noisedu.cn   1/1     Running   6 (5d20h ago)    34d
kube-controller-manager-master3.noisedu.cn   1/1     Running   9 (5d20h ago)    34d
kube-proxy-6lw45                             1/1     Running   6 (5d20h ago)    34d
kube-proxy-9bjch                             1/1     Running   6 (5d20h ago)    34d
kube-proxy-b8g7m                             1/1     Running   11 (5d20h ago)   34d
kube-proxy-bbrxh                             1/1     Running   6 (5d20h ago)    34d
kube-proxy-pm6jk                             1/1     Running   12 (5d20h ago)   34d
kube-scheduler-master1                       1/1     Running   0                2m26s
kube-scheduler-master2.noisedu.cn            1/1     Running   7 (5d20h ago)    34d
kube-scheduler-master3.noisedu.cn            1/1     Running   7 (5d20h ago)    34d
​
# 测试,结果显示无法调度
10:33:53 root@master1 scheduler]#cat 01-scheduler-deployment-test.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-test
spec:
  replicas: 5
  selector:
    matchLabels:
      app: pod-test
  template:
    metadata:
      labels:
        app: pod-test
    spec:
      schedulerName: demo-scheduler
      containers:
      - name: nginxpod-test
        image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
        imagePullPolicy: IfNotPresent
[10:33:58 root@master1 scheduler]#kubectl apply -f 01-scheduler-deployment-test.yaml 
deployment.apps/deployment-test created
[10:34:13 root@master1 scheduler]#kubectl get pods -o wide
NAME                              READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
deployment-test-84444b586-68s7c   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
deployment-test-84444b586-69xfs   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
deployment-test-84444b586-j2vvt   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
deployment-test-84444b586-k74zt   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
deployment-test-84444b586-zb8vp   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
[10:34:19 root@master1 scheduler]#kubectl describe pod deployment-test-84444b586-j2vvt 
Name:           deployment-test-84444b586-j2vvt
Namespace:      default
Priority:       0
Node:           <none>
Labels:         app=pod-test
                pod-template-hash=84444b586
Annotations:    <none>
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  ReplicaSet/deployment-test-84444b586
Containers:
  nginxpod-test:
    Image:        10.0.0.55:80/mykubernetes/pod_test:v0.1
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-bb2zj (ro)
Volumes:
  kube-api-access-bb2zj:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:                      <none>
​
​
# 当不用这个自定义策略时,调度成功
[10:35:03 root@master1 scheduler]#cat 00-no-scheduler-deployment-test.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deployment-test
spec:
  replicas: 5
  selector:
    matchLabels:
      app: pod-test
  template:
    metadata:
      labels:
        app: pod-test
    spec:
      containers:
      - name: nginxpod-test
        image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
        imagePullPolicy: IfNotPresent
[10:35:08 root@master1 scheduler]#kubectl apply -f 00-no-scheduler-deployment-test.yaml 
deployment.apps/deployment-test created
[10:35:21 root@master1 scheduler]#kubectl get pods -o wide
NAME                              READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
deployment-test-7d8cb8c5d-2dkv9   1/1     Running   0          7s    10.244.4.3   node2.noisedu.cn   <none>           <none>
deployment-test-7d8cb8c5d-9hx8f   1/1     Running   0          7s    10.244.3.3   node1.noisedu.cn   <none>           <none>
deployment-test-7d8cb8c5d-f46p5   1/1     Running   0          7s    10.244.4.4   node2.noisedu.cn   <none>           <none>
deployment-test-7d8cb8c5d-sjctx   1/1     Running   0          7s    10.244.3.4   node1.noisedu.cn   <none>           <none>
deployment-test-7d8cb8c5d-vql7n   1/1     Running   0          7s    10.244.3.5   node1.noisedu.cn   <none>           <none>
​
 

节点调度 - 亲和nodeAffinity
# 调度到指定节点,注意节点的name一定要正确,下面先示范错误的节点名
[10:54:46 root@master1 scheduler]#cat 02-scheduler-pod-nodename.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: pod-nodename
spec:
  nodeName: node1
  containers:
  - name: demoapp
    image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
    imagePullPolicy: IfNotPresent
[13:49:01 root@master1 scheduler]#kubectl apply -f 02-scheduler-pod-nodename.yaml 
pod/pod-nodename created
[13:49:28 root@master1 scheduler]#kubectl get pod -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP       NODE    NOMINATED NODE   READINESS GATES
pod-nodename   0/1     Pending   0          5s    <none>   node1   <none>           <none>
[13:49:33 root@master1 scheduler]#kubectl get pod -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP       NODE    NOMINATED NODE   READINESS GATES
pod-nodename   0/1     Pending   0          9s    <none>   node1   <none>           <none>
[13:49:37 root@master1 scheduler]#
[13:49:55 root@master1 scheduler]#
[13:49:55 root@master1 scheduler]#kubectl delete -f 02-scheduler-pod-nodename.yaml 
pod "pod-nodename" deleted
​
# 改为正确的节点名
[13:50:26 root@master1 scheduler]#vim 02-scheduler-pod-nodename.yaml 
[13:51:06 root@master1 scheduler]#cat 02-scheduler-pod-nodename.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: pod-nodename
spec:
  nodeName: node1.noisedu.cn
  containers:
  - name: demoapp
    image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
    imagePullPolicy: IfNotPresent
[13:51:10 root@master1 scheduler]#kubectl apply -f 02-scheduler-pod-nodename.yaml 
pod/pod-nodename created
[13:51:15 root@master1 scheduler]#kubectl get pod -o wide
NAME           READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
pod-nodename   1/1     Running   0          6s    10.244.3.6   node1.noisedu.cn   <none>           <none>
​
#------------------------------------------------------------------------------
# 调度到标签为ssd的节点
[13:53:39 root@master1 scheduler]#cat 03-scheduler-pod-nodeselector.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: pod-nodeselector
spec:
  containers:
  - name: demoapp
    image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
    imagePullPolicy: IfNotPresent
  nodeSelector:
    node: ssd
[13:54:05 root@master1 scheduler]#kubectl get node --show-labels 
NAME                 STATUS   ROLES                  AGE   VERSION   LABELS
master1              Ready    control-plane,master   35d   v1.22.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master1,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=
master2.noisedu.cn   Ready    control-plane,master   35d   v1.22.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master2.noisedu.cn,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=
master3.noisedu.cn   Ready    control-plane,master   35d   v1.22.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master3.noisedu.cn,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=
node1.noisedu.cn     Ready    <none>                 35d   v1.22.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node1.noisedu.cn,kubernetes.io/os=linux
node2.noisedu.cn     Ready    <none>                 35d   v1.22.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=node2.noisedu.cn,kubernetes.io/os=linux
[13:54:19 root@master1 scheduler]#kubectl get node --show-labels | grep ssd
[13:54:24 root@master1 scheduler]#kubectl apply -f 03-scheduler-pod-nodeselector.yaml 
pod/pod-nodeselector created
​
# 由于没有存在为ssd标签的节点,所以pod一直处于pending状态
[13:54:42 root@master1 scheduler]#kubectl get pod -o wide
NAME               READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
pod-nodeselector   0/1     Pending   0          5s    <none>   <none>   <none>           <none>
[13:54:47 root@master1 scheduler]#kubectl describe pod pod-nodeselector 
Name:         pod-nodeselector
Namespace:    default
Priority:     0
Node:         <none>
Labels:       <none>
Annotations:  <none>
Status:       Pending
IP:           
IPs:          <none>
Containers:
  demoapp:
    Image:        10.0.0.55:80/mykubernetes/pod_test:v0.1
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-mkmfp (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  kube-api-access-mkmfp:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              node=ssd
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  11s   default-scheduler  0/5 nodes are available: 2 node(s) didn't match Pod's node affinity/selector, 3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
​
# 一旦给某个节点加上这个标签,pod就会成功
[13:56:58 root@master1 scheduler]#kubectl label node node2.noisedu.cn node=ssd
node/node2.noisedu.cn labeled
[13:57:22 root@master1 scheduler]#kubectl get pod -o wide
NAME               READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
pod-nodeselector   1/1     Running   0          30s   10.244.4.5   node2.noisedu.cn   <none>           <none>
[13:57:28 root@master1 scheduler]#kubectl describe pod pod-nodeselector 
Name:         pod-nodeselector
Namespace:    default
Priority:     0
Node:         node2.noisedu.cn/10.0.0.54
Start Time:   Sun, 16 Jan 2022 13:57:22 +0800
Labels:       <none>
Annotations:  cni.projectcalico.org/containerID: 74260ea9c42767d0f0b9fde2fdd74071c58e40fdd82be1af7afb69925ea438c5
              cni.projectcalico.org/podIP: 10.244.4.5/32
              cni.projectcalico.org/podIPs: 10.244.4.5/32
Status:       Running
IP:           10.244.4.5
IPs:
  IP:  10.244.4.5
Containers:
  demoapp:
    Container ID:   docker://d663a8a36145e37c54a9987755f95311b250c1e0f9b8137b4836aae8ba89c0a4
    Image:          10.0.0.55:80/mykubernetes/pod_test:v0.1
    Image ID:       docker-pullable://10.0.0.55:80/mykubernetes/pod_test@sha256:54402cda2ef15f45e4aafe98a5e56d4de076e3d4100c2a1bf1b780c787372fed
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sun, 16 Jan 2022 13:57:24 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tktqv (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-tktqv:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              node=ssd
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  35s   default-scheduler  0/5 nodes are available: 2 node(s) didn't match Pod's node affinity/selector, 3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
  Normal   Scheduled         11s   default-scheduler  Successfully assigned default/pod-nodeselector to node2.noisedu.cn
  Normal   Pulled            9s    kubelet            Container image "10.0.0.55:80/mykubernetes/pod_test:v0.1" already present on machine
  Normal   Created           9s    kubelet            Created container demoapp
  Normal   Started           9s    kubelet            Started container demoapp
​
​
​
​
​
​
​
节点表达式匹配matchExpressions
​
[13:59:01 root@master1 scheduler]#cat 04-scheduler-pod-node-required-affinity.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: node-required-affinity
spec:
  containers:
  - name: demoapp
    image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
    imagePullPolicy: IfNotPresent
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: env
            operator: In
            values:
            - dev
            - test
[14:01:06 root@master1 scheduler]#kubectl apply -f 04-scheduler-pod-node-required-affinity.yaml 
pod/node-required-affinity created
[14:01:33 root@master1 scheduler]#kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
node-required-affinity   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
[14:01:39 root@master1 scheduler]#kubectl describe pod node-required-affinity 
Name:         node-required-affinity
Namespace:    default
Priority:     0
Node:         <none>
Labels:       <none>
Annotations:  <none>
Status:       Pending
IP:           
IPs:          <none>
Containers:
  demoapp:
    Image:        10.0.0.55:80/mykubernetes/pod_test:v0.1
    Port:         <none>
    Host Port:    <none>
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-g52g2 (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  kube-api-access-g52g2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  12s   default-scheduler  0/5 nodes are available: 2 node(s) didn't match Pod's node affinity/selector, 3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
​
# 由于当前无任何的标签匹配到,所以pod一直处于pending
# 添加标签env=test到node1.noisedu.cn
[14:01:45 root@master1 scheduler]#kubectl label node node1.noisedu.cn env=test
node/node1.noisedu.cn labeled
[14:02:45 root@master1 scheduler]#kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
node-required-affinity   1/1     Running   0          75s   10.244.3.7   node1.noisedu.cn   <none>           <none>
[14:02:48 root@master1 scheduler]#kubectl describe pod node-required-affinity 
Name:         node-required-affinity
Namespace:    default
Priority:     0
Node:         node1.noisedu.cn/10.0.0.53
Start Time:   Sun, 16 Jan 2022 14:02:45 +0800
Labels:       <none>
Annotations:  cni.projectcalico.org/containerID: 925088d019790f61bf46bf4e6949ff386bd145f53fcc328c669eaad2dcd130a1
              cni.projectcalico.org/podIP: 10.244.3.7/32
              cni.projectcalico.org/podIPs: 10.244.3.7/32
Status:       Running
IP:           10.244.3.7
IPs:
  IP:  10.244.3.7
Containers:
  demoapp:
    Container ID:   docker://5df82eeef3c5d5d3ec1de8a36665f7f8ef5faac112c3f2fa2442fbf019d273bf
    Image:          10.0.0.55:80/mykubernetes/pod_test:v0.1
    Image ID:       docker-pullable://10.0.0.55:80/mykubernetes/pod_test@sha256:54402cda2ef15f45e4aafe98a5e56d4de076e3d4100c2a1bf1b780c787372fed
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sun, 16 Jan 2022 14:02:46 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-g52g2 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-g52g2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  78s   default-scheduler  0/5 nodes are available: 2 node(s) didn't match Pod's node affinity/selector, 3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
  Normal   Scheduled         6s    default-scheduler  Successfully assigned default/node-required-affinity to node1.noisedu.cn
  Normal   Pulled            5s    kubelet            Container image "10.0.0.55:80/mykubernetes/pod_test:v0.1" already present on machine
  Normal   Created           5s    kubelet            Created container demoapp
  Normal   Started           5s    kubelet            Started container demoapp
  
# 发现此时匹配成功,调度到node1.noisedu.cn
 

软亲和preferredDuringSchedulingIgnoredDuringExecution
[14:04:47 root@master1 scheduler]#cat 05-scheduler-pod-node-preferred-affinity.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: node-preferred-affinity
spec:
  containers:
  - name: demoapp
    image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
    imagePullPolicy: IfNotPresent
  affinity:
    nodeAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 50
        preference:
          matchExpressions:
          - key: env
            operator: In
            values:
            - test
      - weight: 20
        preference:
          matchExpressions:
          - key: env
            operator: In
            values:
            - dev
[14:07:11 root@master1 scheduler]#kubectl apply -f 05-scheduler-pod-node-preferred-affinity.yaml 
pod/node-preferred-affinity created
[14:07:26 root@master1 scheduler]#kubectl get pod -o wide
NAME                      READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
node-preferred-affinity   1/1     Running   0          5s    10.244.3.8   node1.noisedu.cn   <none>           <none>
​
# 因为指定了preferredDuringSchedulingIgnoredDuringExecution这个属性,即使不满足任何条件也可以调度到任意节点
# 我们重新给node1.noisedu.cn和node2.noisedu.cn加上标签,然后重新创建这个pod查看效果
[14:09:19 root@master1 scheduler]#kubectl delete  -f 05-scheduler-pod-node-preferred-affinity.yaml 
pod "node-preferred-affinity" deleted
[14:12:46 root@master1 scheduler]#kubectl label node node1.noisedu.cn env=dev
node/node1.noisedu.cn labeled
[14:13:04 root@master1 scheduler]#kubectl label node node2.noisedu.cn env=test
node/node2.noisedu.cn labeled
[14:13:12 root@master1 scheduler]#kubectl apply -f 05-scheduler-pod-node-preferred-affinity.yaml 
pod/node-preferred-affinity created
[14:13:22 root@master1 scheduler]#kubectl get pod -o wide
NAME                      READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
node-preferred-affinity   1/1     Running   0          5s    10.244.4.6   node2.noisedu.cn   <none>           <none>
[14:13:27 root@master1 scheduler]#kubectl describe pod node-preferred-affinity 
Name:         node-preferred-affinity
Namespace:    default
Priority:     0
Node:         node2.noisedu.cn/10.0.0.54
Start Time:   Sun, 16 Jan 2022 14:13:22 +0800
Labels:       <none>
Annotations:  cni.projectcalico.org/containerID: 113cd980d6dee836fcf307ae8214235636c7c35c4476b5a37e5a4c92693ea61b
              cni.projectcalico.org/podIP: 10.244.4.6/32
              cni.projectcalico.org/podIPs: 10.244.4.6/32
Status:       Running
IP:           10.244.4.6
IPs:
  IP:  10.244.4.6
Containers:
  demoapp:
    Container ID:   docker://7a520abf227cdc993a2b6288ecc1b6a35da7179e1f4a9a4443e817a88baa96f9
    Image:          10.0.0.55:80/mykubernetes/pod_test:v0.1
    Image ID:       docker-pullable://10.0.0.55:80/mykubernetes/pod_test@sha256:54402cda2ef15f45e4aafe98a5e56d4de076e3d4100c2a1bf1b780c787372fed
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sun, 16 Jan 2022 14:13:24 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-b7m2v (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-b7m2v:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  11s   default-scheduler  Successfully assigned default/node-preferred-affinity to node2.noisedu.cn
  Normal  Pulled     10s   kubelet            Container image "10.0.0.55:80/mykubernetes/pod_test:v0.1" already present on machine
  Normal  Created    9s    kubelet            Created container demoapp
  Normal  Started    9s    kubelet            Started container demoapp
​
# 此时发现因为node2.noisedu.cn的权重比较大,所以调度到node2这个节点,权重比较也是软亲和和硬亲和的最重要区别。
​
​
# 资源调度实验,我们对CPU和内存做了资源限制,要求满足资源的node才给调度
[14:18:56 root@master1 scheduler]#cat 06-scheduler-pod-node-resourcefits-affinity.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: node-resourcefits-affinity
spec:
  replicas: 2
  selector:
    matchLabels:
      app: podtest
  template:
    metadata:
      labels:
        app: podtest
    spec:
      containers:
      - name: podtest
        image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: 2
            memory: 2Gi
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: env
                operator: Exists
[14:19:14 root@master1 scheduler]#kubectl apply -f 06-scheduler-pod-node-resourcefits-affinity.yaml 
deployment.apps/node-resourcefits-affinity created
[14:19:19 root@master1 scheduler]#kubectl get pod -o wide
NAME                                          READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
node-resourcefits-affinity-84fd5f6f9c-qmmlc   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
node-resourcefits-affinity-84fd5f6f9c-wpqt6   0/1     Pending   0          6s    <none>   <none>   <none>           <none>
[14:19:25 root@master1 scheduler]#kubectl describe pod
poddisruptionbudgets.policy  pods                         podsecuritypolicies.policy   podtemplates                 
[14:19:25 root@master1 scheduler]#kubectl describe pod node-resourcefits-affinity-84fd5f6f9c-qmmlc 
Name:           node-resourcefits-affinity-84fd5f6f9c-qmmlc
Namespace:      default
Priority:       0
Node:           <none>
Labels:         app=podtest
                pod-template-hash=84fd5f6f9c
Annotations:    <none>
Status:         Pending
IP:             
IPs:            <none>
Controlled By:  ReplicaSet/node-resourcefits-affinity-84fd5f6f9c
Containers:
  podtest:
    Image:      10.0.0.55:80/mykubernetes/pod_test:v0.1
    Port:       <none>
    Host Port:  <none>
    Requests:
      cpu:        2
      memory:     2Gi
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-ljtxk (ro)
Conditions:
  Type           Status
  PodScheduled   False 
Volumes:
  kube-api-access-ljtxk:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason            Age   From               Message
  ----     ------            ----  ----               -------
  Warning  FailedScheduling  22s   default-scheduler  0/5 nodes are available: 2 Insufficient cpu, 2 Insufficient memory, 3 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate.
​
# 我们发现满足2 CPU和内存的node都没有,所以pod处于pending。
​
# 修改CPU为0.2个,内存和100Mi,重新创建pod
[14:23:17 root@master1 scheduler]#cat 06-scheduler-pod-node-resourcefits-affinity.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: node-resourcefits-affinity
spec:
  replicas: 2
  selector:
    matchLabels:
      app: podtest
  template:
    metadata:
      labels:
        app: podtest
    spec:
      containers:
      - name: podtest
        image: 10.0.0.55:80/mykubernetes/pod_test:v0.1
        imagePullPolicy: IfNotPresent
        resources:
          requests:
            cpu: 0.2
            memory: 100Mi
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: env
                operator: Exists
[14:23:22 root@master1 scheduler]#kubectl apply -f 06-scheduler-pod-node-resourcefits-affinity.yaml 
deployment.apps/node-resourcefits-affinity created
[14:23:26 root@master1 scheduler]#kubectl get pod -o wide
NAME                                         READY   STATUS    RESTARTS   AGE   IP           NODE               NOMINATED NODE   READINESS GATES
node-resourcefits-affinity-778bdb685-76x5s   1/1     Running   0          6s    10.244.3.9   node1.noisedu.cn   <none>           <none>
node-resourcefits-affinity-778bdb685-h54vp   1/1     Running   0          6s    10.244.4.7   node2.noisedu.cn   <none>           <none>
[14:23:32 root@master1 scheduler]#kubectl describe pod node-resourcefits-affinity-778bdb685-76x5s 
Name:         node-resourcefits-affinity-778bdb685-76x5s
Namespace:    default
Priority:     0
Node:         node1.noisedu.cn/10.0.0.53
Start Time:   Sun, 16 Jan 2022 14:23:26 +0800
Labels:       app=podtest
              pod-template-hash=778bdb685
Annotations:  cni.projectcalico.org/containerID: f615b27cbf0fd22f4b08ba1675192b789c88646e38cf8db7c7df33add1409ae4
              cni.projectcalico.org/podIP: 10.244.3.9/32
              cni.projectcalico.org/podIPs: 10.244.3.9/32
Status:       Running
IP:           10.244.3.9
IPs:
  IP:           10.244.3.9
Controlled By:  ReplicaSet/node-resourcefits-affinity-778bdb685
Containers:
  podtest:
    Container ID:   docker://5ede69880785fe10fe4e12e6c0ef88a116fff69b40fe69aa0dc7f5ff4f8a8c44
    Image:          10.0.0.55:80/mykubernetes/pod_test:v0.1
    Image ID:       docker-pullable://10.0.0.55:80/mykubernetes/pod_test@sha256:54402cda2ef15f45e4aafe98a5e56d4de076e3d4100c2a1bf1b780c787372fed
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sun, 16 Jan 2022 14:23:28 +0800
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:        200m
      memory:     100Mi
    Environment:  <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-n9rk2 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-n9rk2:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  16s   default-scheduler  Successfully assigned default/node-resourcefits-affinity-778bdb685-76x5s to node1.noisedu.cn
  Normal  Pulled     15s   kubelet            Container image "10.0.0.55:80/mykubernetes/pod_test:v0.1" already present on machine
  Normal  Created    15s   kubelet            Created container podtest
  Normal  Started    15s   kubelet            Started container podtest
​
# 发现成功,由于node1和node2都存在env这个label,所以随机调度。
[14:24:41 root@master1 scheduler]#kubectl get node --show-labels | grep env
node1.noisedu.cn     Ready    <none>                 35d   v1.22.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,env=dev,kubernetes.io/arch=amd64,kubernetes.io/hostname=node1.noisedu.cn,kubernetes.io/os=linux
node2.noisedu.cn     Ready    <none>                 35d   v1.22.1   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,env=test,kubernetes.io/arch=amd64,kubernetes.io/hostname=node2.noisedu.cn,kubernetes.io/os=linux
​
 

  

随时随地学软件编程-关注百度小程序和微信小程序
关于找一找教程网

本站文章仅代表作者观点,不代表本站立场,所有文章非营利性免费分享。
本站提供了软件编程、网站开发技术、服务器运维、人工智能等等IT技术文章,希望广大程序员努力学习,让我们用科技改变世界。
[Linux - K8S - 调度策略 - Node调度]http://www.zyiz.net/tech/detail-286369.html

赞(0)
关注微信小程序
程序员编程王-随时随地学编程

扫描二维码或查找【程序员编程王】

可以随时随地学编程啦!

技术文章导航 更多>