Kubernetes之报错already present on machine
LiuSw Lv6
 2022-12-20 15:02:38      502 字  2 分钟  

Kubernetes之报错already present on machine

pod节点报错already present on machine，应该是数据损坏，处理思路：

  • 将其他matser或者node打上其他标签；
  • 将报错pod个数调度到0；
  • 修改pod的亲和性改为刚刚打得标签，让这个pod调度到其他节点；

现在报错： 

1
2
3
Warning  Failed          7m52s                  kubelet, smyk8s-h3c-master-01  Error: Error response from daemon: error creating overlay mount to /data/docker/overlay2/cb7546231a100f02b414080eb065e88769876b79e72736d1e4ec941c69b9b3a4-init/merged: no such file or directory
Warning  Failed          7m28s (x2 over 7m39s)  kubelet, smyk8s-h3c-master-01  (combined from similar events): Error: Error response from daemon: error creating overlay mount to /data/docker/overlay2/6272d607de946411c5ea81c1277345830ff49af7d94fc5ada83a683fe02e937a-init/merged: no such file or directory
Normal   Pulled          90s (x39 over 9m16s)   kubelet, smyk8s-h3c-master-01  Container image "osixia/openldap:1.3.0" already present on machine

解决方案

根据版主(kubesphere论坛)的指导总结下解决过程：

1.这个pod运行只在master上的，master节点有3个，其中master1的数据被删无法启动，需要调度到master2和master3上启动，这里是对master2和master3打个自定义标签node-role.kubernetes.io/openldap

1
2
kubectl label node master-02 node-role.kubernetes.io/openldap
kubectl label node master-03 node-role.kubernetes.io/openldap

实例关闭 

1
kubectl -n kubesphere-system scale sts openldap --replicas=0

修改sts 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
kubectl -n kubesphere-system edit sts openldap
#在亲和性上修改为：
nodeAffinity:
  preferredDuringSchedulingIgnoredDuringExecution:
  - preference:
      matchExpressions:
      - key: node-role.kubernetes.io/openldap # 修改此处
        operator: In
        values:
        - ""
    weight: 100

#保存退出，

#启动实例等待openldap服务启动完成验证分配节点是否正常
kubectl -n kubesphere-system scale sts openldap --replicas=2
kubectl get po -n kubesphere-system -o wide |grep open

#openldap启动完成后重启account服务
kubectl rollout restart deployment ks-account -n kubesphere-system

2.重置管理员密码 

1
2
3
4
5
6
7
8
#查看account服务的pod name
kubectl get po -n kubesphere-system  |grep ks-account

#进入容器
kubectl exec -it ks-account-5d8c49d4bc-4rz29 -n kubesphere-system sh

#执行初始化账号
packet='PUT /kapis/iam.kubesphere.io/v1alpha2/users/admin HTTP/1.1\r\nHost: ks-account.kubesphere-system.svc:9090\r\nUser-Agent: curl/7.54.0\r\nAccept: */*\r\nContent-Type: application/json\r\nContent-Length: 105\r\n\r\n{"username": "admin","email":"admin@kubesphere.io","cluster_role": "cluster-admin","password":"P@88w0rd"}'; echo -ne $packet | nc ks-account.kubesphere-system.svc 80
 评论