1. 准备工作

# 考试报名资料
报名文档：[https://training.linuxfoundation.cn/news/308](https://training.linuxfoundation.cn/news/308) 考纲：[https://training.linuxfoundation.cn/certificates/1](https://training.linuxfoundation.cn/certificates/1) 考试常见问题：[https://docs.linuxfoundation.org/tc-docs/certification/lf-handbook2/taking-the-exam](https://docs.linuxfoundation.org/tc-docs/certification/lf-handbook2/taking-the-exam)

# 考试小技巧

## 查看k8s资源列表
```yaml
kubectl api-resources --namespaced=true  ##查看哪些资源在命令空间 
kubectl api-resources --namespaced=false  ##查看哪些资源不在命令空间
```

## 查看kubectl示例命令
```yaml
[root@k8s-master ~]# kubectl create clusterrole -h
Create a cluster role.

Examples:
  # Create a cluster role named "pod-reader" that allows user to perform "get", "watch" and "list" on pods
  kubectl create clusterrole pod-reader --verb=get,list,watch --resource=pods
  
  # Create a cluster role named "pod-reader" with ResourceName specified
  kubectl create clusterrole pod-reader --verb=get --resource=pods --resource-name=readablepod
--resource-name=anotherpod
  
  # Create a cluster role named "foo" with API Group specified
  kubectl create clusterrole foo --verb=get,list,watch --resource=rs.extensions
  
  # Create a cluster role named "foo" with SubResource specified
  kubectl create clusterrole foo --verb=get,list,watch --resource=pods,pods/status
  
  # Create a cluster role name "foo" with NonResourceURL specified
  kubectl create clusterrole "foo" --verb=get --non-resource-url=/logs/*
  
  # Create a cluster role name "monitoring" with AggregationRule specified
  kubectl create clusterrole monitoring --aggregation-rule="rbac.example.com/aggregate-to-monitoring=true"

Options:
    ……
    --verb=[]:
        Verb that applies to the resources contained in the rule

Usage:
  kubectl create clusterrole NAME --verb=verb --resource=resource.group [--resource-name=resourcename]
[--dry-run=server|client|none] [options]

Use "kubectl options" for a list of global command-line options (applies to all commands).
```

## 快速生成yaml文件
```bash
# 不创建资源，将yaml导出到文件
[root@k8s-master ~]# kubectl create deployment nginx --image=nginx -o yaml --dry-run > nginx.yaml
W0516 11:15:34.619473  469162 helpers.go:639] --dry-run is deprecated and can be replaced with --dry-run=client.
[root@k8s-master ~]# cat nginx.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}
# get已有资源，导出yaml格式文件
[root@k8s-master ~]# kubectl get svc myapp -o yaml > myapp.svc
[root@k8s-master ~]# cat myapp.svc 
apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"myapp","namespace":"default"},"spec":{"ports":[{"port":80,"targetPort":80}],"selector":{"app":"myapp"},"type":"ClusterIP"}}
  creationTimestamp: "2023-05-11T11:18:49Z"
  name: myapp
  namespace: default
  resourceVersion: "95775"
  uid: 94d82f07-fad5-4713-8e26-67a22baa5816
spec:
  clusterIP: 10.106.161.89
  clusterIPs:
  - 10.106.161.89
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: myapp
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: {}
```

## 资源清单编辑
考试过程中涉及到修改pod配置的情况，记得导出yaml文件后备份，以免修改错误无法恢复。 从官网复制yaml文件到vim编辑器可能存在过多空白行的情况，可以执行`:g/^$/d`删除空白行

# 重点文档路径
考试过程中允许查阅官方文档，并复制相关yaml配置，可以提前记住考点对应的文档路径以及相关配置的大致位置，避免考试过程中搜索查找文档浪费时间。

## RBAC
文档路径：参考——>API访问控制——>使用RBAC鉴权 链接：[https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/rbac/](https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/rbac/)

## 升级k8s版本
文档路径：任务——>管理集群——>用kubeadm进行管理——>升级kubeadm集群 链接：[https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/](https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/)

## ETCD备份与恢复
文档路径：任务——>管理集群——>为 Kubernetes 运行 etcd 集群 链接：[https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/configure-upgrade-etcd/](https://kubernetes.io/zh-cn/docs/tasks/administer-cluster/configure-upgrade-etcd/)

## pod增加sidecar容器
文档路径：概念——>集群管理——>日志架构 链接：[https://kubernetes.io/zh-cn/docs/concepts/cluster-administration/logging/](https://kubernetes.io/zh-cn/docs/concepts/cluster-administration/logging/)

## pod指定节点调度
文档路径：任务——>配置 Pods 和容器——>将 Pod 分配给节点 链接：[https://kubernetes.io/zh-cn/docs/tasks/configure-pod-container/assign-pods-nodes/](https://kubernetes.io/zh-cn/docs/tasks/configure-pod-container/assign-pods-nodes/)

## 网络策略
文档路径：概念——>服务、负载均衡和联网——>网络策略 链接：[https://kubernetes.io/zh-cn/docs/concepts/services-networking/network-policies/](https://kubernetes.io/zh-cn/docs/concepts/services-networking/network-policies/)

## SVC暴露服务
文档路径：概念——>服务、负载均衡和联网——>服务（Service） 链接：[https://kubernetes.io/zh-cn/docs/concepts/services-networking/service/](https://kubernetes.io/zh-cn/docs/concepts/services-networking/service/)

## Ingress
文档路径：概念——>服务、负载均衡和联网——>Ingress 链接：[https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/](https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/)

## pod使用pv
文档路径：任务——>配置 Pods 和容器——>配置 Pod 以使用 PersistentVolume 作为存储 链接：[https://kubernetes.io/zh-cn/docs/tasks/configure-pod-container/configure-persistent-volume-storage/](https://kubernetes.io/zh-cn/docs/tasks/configure-pod-container/configure-persistent-volume-storage/)

# 练习环境准备

## 部署k8s
部署一个3节点k8s，版本为1.24.X，操作系统为ubuntu(centos也可以)，容器运行时使用Containerd。参考文章：

- 环境准备：[https://www.cuiliangblog.cn/detail/section/15186285](https://www.cuiliangblog.cn/detail/section/15186285)
- 安装containerd：[https://www.cuiliangblog.cn/detail/section/99861101](https://www.cuiliangblog.cn/detail/section/99861101)
- 部署kubernetes：[https://www.cuiliangblog.cn/detail/section/15188146](https://www.cuiliangblog.cn/detail/section/15188146)

本环境操作系统为Rocky Linux 8.7 内核版本为4.18.0-425.13.1.el8_7.x86_64，k8s集群版本为1.24.13，containerd版本为1.6.4。集群环境如下：

| 主机名 | ip | 角色 | 其他操作 |
| --- | --- | --- | --- |
| k8s-master | 192.168.10.20 | master节点 | 节点打上NoSchedule的污点 |
| k8s-work1 | 192.168.10.21 | work节点 | 单独部署etcd服务，模拟etcd数据备份与恢复 |
| k8s-work2 | 192.168.10.22 | work节点 | 停止kubelet服务，并关闭开机自启动 |

## 部署ingress-NGINX
参考文章：[https://www.cuiliangblog.cn/detail/section/15188441](https://www.cuiliangblog.cn/detail/section/15188441)

## 部署共享存储
参考文章：[https://www.cuiliangblog.cn/detail/section/116191364](https://www.cuiliangblog.cn/detail/section/116191364)

## 部署metrics-server
参考文章：[https://www.cuiliangblog.cn/detail/section/15189166](https://www.cuiliangblog.cn/detail/section/15189166)

## 部署etcd服务
在work1节点单独部署一套etcd模拟考试题目的etcd备份恢复操作。

- 下载安装etcd软件
```bash
# 下载软件包
[root@k8s-work1 ~]# wget https://github.com/etcd-io/etcd/releases/download/v3.4.23/etcd-v3.4.23-linux-amd64.tar.gz
[root@k8s-work1 ~]# ls
etcd-v3.4.23-linux-amd64.tar.gz

# 解压到指定目录
[root@k8s-work1 ~]# tar -zxf etcd-v3.4.23-linux-amd64.tar.gz -C /usr/local
[root@k8s-work1 ~]# cd /usr/local/etcd-v3.4.23-linux-amd64/
[root@k8s-work1 etcd-v3.4.23-linux-amd64]# ls
Documentation  README-etcdctl.md  README.md  READMEv2-etcdctl.md  etcd  etcdctl

# 添加环境变量
[root@k8s-work1 etcd-v3.4.23-linux-amd64]# vim /etc/profile
export PATH="$PATH:/usr/local/etcd-v3.4.23-linux-amd64"
[root@k8s-work1 etcd-v3.4.23-linux-amd64]# source /etc/profile

# 验证
[root@k8s-work1 etcd-v3.4.23-linux-amd64]# etcdctl version
etcdctl version: 3.4.23
API version: 3.4
[root@k8s-work1 etcd-v3.4.23-linux-amd64]# etcd --version
etcd Version: 3.4.23
Git SHA: c8b7831
Go Version: go1.17.13
Go OS/Arch: linux/amd64
```

- 配置证书
```bash
# 下载安装cfssl
[root@k8s-work1 ~]# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.3/cfssl_1.6.3_linux_amd64
[root@k8s-work1 ~]# wget https://github.com/cloudflare/cfssl/releases/download/v1.6.3/cfssljson_1.6.3_linux_amd64
[root@k8s-work1 ~]# mv cfssl_1.6.3_linux_amd64 /usr/bin/cfssl
[root@k8s-work1 ~]# mv cfssljson_1.6.3_linux_amd64 /usr/bin/cfssljson
[root@k8s-work1 ~]# chmod +x /usr/bin/{cfssl,cfssljson}
[root@k8s-work1 ~]# cfssl version
Version: 1.6.3
Runtime: go1.18
[root@k8s-work1 ~]# cd /etc/etcd/pki

# 创建 CA 证书
[root@k8s-work1 pki]# cat > ca-config.json <<EOF
{
 "signing": {
 "default": {
 "expiry": "43800h"
 },
 "profiles": {
 "server": {
 "expiry": "43800h",
 "usages": [
 "signing",
 "key encipherment",
 "server auth"
 ]
 },
 "client": {
 "expiry": "43800h",
 "usages": [
 "signing",
 "key encipherment",
 "client auth"
 ]
 },
 "peer": {
 "expiry": "43800h",
 "usages": [
 "signing",
 "key encipherment",
 "server auth",
 "client auth"
 ]
 }
 }
 }
}
EOF
[root@k8s-work1 pki]# cat > ca-csr.json <<EOF
{
 "CN": "Etcd",
 "key": {
 "algo": "rsa",
 "size": 2048
 },
 "names": [
 {
 "C": "CN",
 "ST": "BeiJing",
 "L": "BeiJing",
 "O": "Etcd",
 "OU": "CA"
 }
 ]
}
EOF
[root@k8s-work1 pki]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
[root@k8s-work1 pki]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem

# 生成服务器端证书
[root@k8s-work1 pki]# cat > server-csr.json <<EOF
{
 "CN": "server",
 "hosts": [
 "127.0.0.1",
 "192.168.10.21"
 ],
 "key": {
 "algo": "ecdsa",
 "size": 256
 },
 "names": [
 {
 "C": "CN",
 "L": "BeiJing",
 "ST": "BeiJing"
 }
 ]
}
EOF
[root@k8s-work1 pki]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=server server-csr.json | cfssljson -bare server
[root@k8s-work1 pki]# ls server*
server.csr server-csr.json server-key.pem server.pem

# 生成客户端证书
[root@k8s-work1 pki]# cat > client-csr.json <<EOF
{
 "CN": "client",
 "hosts": [
 ""
 ],
 "key": {
 "algo": "ecdsa",
 "size": 256
 },
 "names": [
 {
 "C": "CN",
 "L": "BeiJing",
 "ST": "BeiJing"
 }
 ]
}
EOF
[root@k8s-work1 pki]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=client client-csr.json | cfssljson -bare client
[root@k8s-work1 pki]# ls client*
client.csr client-csr.json client-key.pem client.pem

# 更新系统证书库
[root@k8s-work1 pki]# dnf install ca-certificates -y  
[root@k8s-work1 pki]# update-ca-trust
```

- 添加配置文件
```bash
# 创建数据目录与配置文件目录
[root@k8s-work1 ~]# mkdir -p /etc/etcd
[root@k8s-work1 ~]# mkdir -p /var/lib/etcd

# systemd 服务配置文件
[root@k8s-work1 ~]# cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=notify
EnvironmentFile=/etc/etcd/etcd.conf
ExecStart=/usr/local/etcd-v3.4.23-linux-amd64/etcd --config-file=/etc/etcd/etcd.conf
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

# 创建etcd配置文件
[root@k8s-work1 ~]# cat > /etc/etcd/etcd.conf <<EOF
# 节点名称
name: "etcd1"
# 数据存储目录
data-dir: "/var/lib/etcd"
# 对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点
advertise-client-urls: "https://192.168.10.21:2379"
# 监听客户端请求的地址列表
listen-client-urls: "https://192.168.10.21:2379,https://127.0.0.1:2379"
# 监听URL，用于节点之间通信监听地址
listen-peer-urls: "https://192.168.10.21:2380"
# 服务端之间通讯使用的地址列表,该节点同伴监听地址，这个值会告诉集群中其他节点
initial-advertise-peer-urls: "https://192.168.10.21:2380"
# etcd启动时，etcd集群的节点地址列表
initial-cluster: "etcd1=https://192.168.10.21:2380"
# etcd集群的初始集群令牌
initial-cluster-token: 'etcd-cluster'
# etcd集群初始化的状态，new代表新建集群，existing表示加入现有集群
initial-cluster-state: 'new'
# 日志配置
logger: zap

# 客户端加密
client-transport-security:
  cert-file: "/etc/etcd/pki/server.pem"
  key-file: "/etc/etcd/pki/server-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/pki/ca.pem"

# 节点加密
peer-transport-security:
  cert-file: "/etc/etcd/pki/server.pem"
  key-file: "/etc/etcd/pki/server-key.pem"
  client-cert-auth: True
  trusted-ca-file: "/etc/etcd/pki/ca.pem"
EOF

# 启动etcd服务并添加开机自启动
[root@k8s-work1 ~]# systemctl daemon-reload
[root@k8s-work1 ~]# systemctl start etcd
[root@k8s-work1 ~]# systemctl enable etcd 
```

- 访问验证
```bash
[root@k8s-work1 ~]# ETCDCTL_API=3 etcdctl --endpoints=https://192.168.10.21:2379 --cacert=/etc/etcd/pki/ca.pem --cert=/etc/etcd/pki/client.pem --key=/etc/etcd/pki/client-key.pem endpoint status --cluster -w table
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|          ENDPOINT          |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| https://192.168.10.21:2379 | f8faaf98bca480d2 |  3.4.23 |   20 kB |      true |      false |         2 |          4 |                  4 |        |
+----------------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
[root@k8s-work1 ~]# ETCDCTL_API=3 etcdctl --endpoints=https://192.168.10.21:2379 --cacert=/etc/etcd/pki/ca.pem --cert=/etc/etcd/pki/client.pem --key=/etc/etcd/pki/client-key.pem put /foo/bar "hello world"
OK   
[root@k8s-work1 ~]# ETCDCTL_API=3 etcdctl --endpoints=https://192.168.10.21:2379 --cacert=/etc/etcd/pki/ca.pem --cert=/etc/etcd/pki/client.pem --key=/etc/etcd/pki/client-key.pem get /foo/bar
/foo/bar
hello world
```

- 创建备份快照文件
```bash
[root@k8s-work1 ~]# mkdir -p /data/backup/
[root@k8s-work1 ~]# ETCDCTL_API=3 etcdctl snapshot save /data/backup/etcd-snapshot-previous.db --endpoints=https://127.0.0.1:2379 --cacert=/etc/etcd/pki/ca.pem --cert=/etc/etcd/pki/client.pem --key=/etc/etcd/pki/client-key.pem
{"level":"info","ts":1684481727.6203024,"caller":"snapshot/v3_snapshot.go:119","msg":"created temporary db file","path":"/data/backup/etcd-snapshot.db.part"}
{"level":"info","ts":"2023-05-19T15:35:27.677+0800","caller":"clientv3/maintenance.go:200","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":1684481727.678215,"caller":"snapshot/v3_snapshot.go:127","msg":"fetching snapshot","endpoint":"https://127.0.0.1:2379"}
{"level":"info","ts":"2023-05-19T15:35:27.714+0800","caller":"clientv3/maintenance.go:208","msg":"completed snapshot read; closing"}
{"level":"info","ts":1684481727.7291808,"caller":"snapshot/v3_snapshot.go:142","msg":"fetched snapshot","endpoint":"https://127.0.0.1:2379","size":"20 kB","took":0.106096152}
{"level":"info","ts":1684481727.7293956,"caller":"snapshot/v3_snapshot.go:152","msg":"saved","path":"/data/backup/etcd-snapshot.db"}
Snapshot saved at ls 
```

## 其他资源创建
```bash
[root@k8s-master ~]# kubectl create ns app-team1
namespace/app-team1 created
[root@k8s-master ~]# kubectl create ns my-app
namespace/my-app created

[root@k8s-master ~]# kubectl create deployment front-end --image=nginx
deployment.apps/front-end created
[root@k8s-master ~]# kubectl create deployment loadbalancer --image=nginx
deployment.apps/loadbalancer created
[root@k8s-master ~]# kubectl create deployment cpu-utilizer --image=docker.elastic.co/logstash/logstash:8.7.1 --replicas=3
deployment.apps/cpu-utilizer created

[root@k8s-master ~]# cat pod.yaml
apiVersion: v1
kind: Pod
metadata:
 name: legacy-app
spec:
 containers:
 - name: count
 image: busybox
 args:
 - /bin/sh 
 - -c 
 - > 
 i=0; 
 while true; 
 do 
 echo "$i: $(date)" >> /var/log/legacy-app.log; 
 sleep 1; 
 done 
 volumes:
 - name: varlog 
 emptyDir: {}
[root@k8s-master ~]# kubectl apply -f pod.yaml 
pod/legacy-app created
[root@k8s-master ~]# cat > filebeat.yaml <<EOF
apiVersion: v1
kind: Pod
metadata:
 name: filebeat
spec:
 containers:
 - name: filebeat
 image: docker.elastic.co/beats/filebeat:8.7.1
EOF
[root@k8s-master ~]# kubectl apply -f filebeat.yaml 
pod/filebeat created
```

## work1节点打标签
有一道题目是pod调度到直接标签的节点，需要提起在work1节点打标签
```bash
[root@k8s-master ~]# kubectl label nodes k8s-work1 disktype=ssd
node/k8s-work1 labeled
[root@k8s-master ~]# kubectl get nodes --show-labels | grep ssd
k8s-work1 Ready <none> 10d v1.24.13 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,disktype=ssd,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-work1,kubernetes.io/os=linux
```

## 停止work2服务
有一道处理节点NotReady的题目，就是kubelet服务未启动导致。
```bash
[root@k8s-work2 ~]# systemctl stop kubelet
[root@k8s-work2 ~]# systemctl disable kubelet
Removed /etc/systemd/system/multi-user.target.wants/kubelet.service.
```

## master节点打上污点
有一道题目是统计有多少个worker nodes已经准备就绪（不包括被打上Taint NoSchedule的节点）
```bash
[root@k8s-master ~]# kubectl taint nodes k8s-master key=value:NoSchedule
node/k8s-master tainted
```