12. ETCD节点故障修复

# 安装etcdctl命令
```bash
yum install etcd-client
```

# 配置环境变量指定证书路径
```bash
export ETCDCTL CACERT=/etc/kubernetes/pki/etcd/ca.crt
export ETCDCTL CERT=/etc/kubernetes/pki/etcd/server.crt
export ETCDCTL KEY=/etc/kubernetes/pki/etcd/server.key
```

# 查看故障的etcd节点
```bash
# kubectl -n kube-system get pods grep etcd
etcd-master-01       1/1     Running              1(28d ago)       28d
etcd-master-02       1/1     Running              45(3d1h ago)     3d1h
etcd-master-03       0/1     CrashLoopBackOff     56(5m ago)       28d
# ETCDCTL API=3 etcdctl member list
2f324c77a17e8946,started,master03,https://1.182.90.28:2380，https://1.182.90.28:2379
57b3f718b2e0ed8a,started,master01,https://1.182.90.40:2380,https://1.182.90.40:2379
a8407c8a16e3df20,started,master02,https://1.182.90.27:2380，https://1.182.90.27:2379
```

# 根据节点ID删除掉etcd节点
```bash
# kubectl -n kube-system delete pods etcd-master03
# ETCDCTL API=3 etcdctl member list
2f324c77a17e8946，started,masterg3,https://1.182.90.28:2380，https://1.182.90.28:2379
57b3f718b2e0ed8a,started,master01,https://1.182.90.40:2388，https://1.182.90.40:2379
a8407c8a16e3df20,started,master02,https://1.182.90.27:2380,https://1.182.90.27:2379
# ETCDCTL API=3 etcdctl member remove 2f324c77a17e8946
Member 2f324c77a17e8946 removed from cluster c70ccff1085a1494
```

# 添加etcd节点
```bash
# ETCDCTL API=3 etcdctl member add master03Member 75912c3fec2d6713 added to cluster c70ccff1085a1484
ETCD NAME="master03"
ETCD INITIAL CLUSTER="master01=https://1.182.90.40:2380,master03=https://1.182.90.28:2380
ETCD INITIAL CLUSTER STATE="existing'
```