Backup kubernetes backup etcd data

1. Practice goals

First prepare one in the environment pod1, Then back up etcd data , Then delete pod1. The backup data contains pod1 Of , So by restoring etcd After the data , You can still see pod1.

2. Practice environment

This experiment consists of 2 Taiwan machine , Namely vms71 and vms72.

[[email protected] ~]# kubectl get nodes
NAME            STATUS   ROLES                  AGE    VERSION
vms71.rhce.cc   Ready    control-plane,master   2d6h   v1.23.2
vms72.rhce.cc   Ready    <none>                 2d6h   v1.23.2
[[email protected] ~]#

3. stay vms71 Installation on etcd software package

[[email protected] ~]# yum install etcd -y
	... Output ...
  complete ！
[[email protected] ~]#

4. Preparation before experiment

Now in the current environment, there is a name called pod1 Of pod Running .

[[email protected] ~]# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
pod1   1/1     Running   0          42h
[[email protected] ~]#

In order to reduce the length of commands during backup and recovery , Here we first define some variables .

[[email protected] ~]# export ETCDCTL_API=3
[[email protected] ~]# cert=/etc/kubernetes/pki/etcd/server.crt
[[email protected] ~]# key=/etc/kubernetes/pki/etcd/server.key 
[[email protected] ~]# cacert=/etc/kubernetes/pki/etcd/ca.crt 
[[email protected] ~]#

5. The backup data

Back up the data according to the following command .

[[email protected] ~]# etcdctl --endpoints=https://192.168.26.71:2379 --cacert=$cacert --cert=$cert --key=$key snapshot save snap1.data
*Snapshot saved at snap1.data*
[[email protected] ~]#

At this time, the backup data includes pod1 This pod Of .

Delete after backup pod1.

[[email protected] ~]# kubectl delete pod pod1
pod "pod1" deleted
[[email protected] ~]# kubectl get pods
No resources found in default namespace.
[[email protected] ~]# 

The current namespace does not exist pod 了 , But there are in the backup data .

6. Restore data

1. close kubernetes Its core components include etcd.

Because they are static pod The way it works , And the static directory is /etc/kubernetes/manifests/, So just cut this directory to another location , Cut here to / Next .

[[email protected] ~]# mv /etc/kubernetes/manifests/ /
[[email protected] ~]#

2. Delete the original data .

[[email protected] ~]# rm -rf /var/lib/etcd/
[[email protected] ~]#

3. Restore data .

[[email protected] ~]# etcdctl --endpoints=https://192.168.26.71:2379 --cacert=$cacert --cert=$cert --key=$key snapshot restore snap1.data --data-dir=/var/lib/etcd/ --name=vms71.rhce.cc --initial-cluster=vms71.rhce.cc=https://192.168.26.71:2380 --initial-advertise-peer-urls https://192.168.26.71:2380
*2022-02-28 16:58:25.518748 I | mvcc: restore compact to 67960
2022-02-28 16:58:25.526988 I | etcdserver/membership: added member ec884fe58e76ba1a [https://192.168.26.71:2380] to cluster 6ba0775b22f7fb90*
[[email protected] ~]#

In this way, the data recovery is completed , Most of the parameters of this command can be changed to the original /etc/kubernetes/manifests/etcd.yaml current /manifests/etcd.yaml Found in .
Modify owner and group

[[email protected] ~]# chown -R etcd.etcd /var/lib/etcd/
[[email protected] ~]#

4. restart kubelet.

[[email protected] ~]# systemctl restart kubelet
[[email protected] ~]#

5. verification pod Whether it has been restored .

[[email protected] ~]# kubectl get pods
NAME   READY   STATUS              RESTARTS   AGE
pod1   0/1     ContainerCreating   1          42h
[[email protected] ~]# 
[[email protected] ~]# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
pod1   1/1     Running   0          42h
[[email protected] ~]#