Using gatekeeper to restrict kubernetes to create specific types of resources

Under normal circumstances, users should log in k8s Create a resource such as pod when , First apiserver Will detect
1. adopt token Or the certificate authenticates whether this user can log in k8s
2. It will check whether this user has sufficient permissions
stay k8s By default , If the user can log in and has permission , You can create resources correctly . But we can use the access controller （admission control） Or dynamic admission controller （Dynamic Admission Control） Customize a series of rules , Further restrict what you want to create pod Compliance .

OPA（ The full name is Open Policy Agent ） yes CNCF An incubated project , It is an open source strategy engine , Can be used for docker、envoy、ssh、kubernetes And other application customization strategies .

and gatekeeper Is based on OPA by kubernetes A customized project , Various strategies can be implemented . These strategies act through the admission controller kubernetes Upper .

install gatekeeper

First go to the following address to download the latest manifest file
https://github.com/open-policy-agent/gatekeeper/tree/master/deploy
And then in master Execute the following command to install gatekeeper.

[email protected]:~/demo4# kubectl apply -f gatekeeper.yaml 
namespace/gatekeeper-system created
... Output ...
poddisruptionbudget.policy/gatekeeper-controller-manager created
validatingwebhookconfiguration.admissionregistration.k8s.io/gatekeeper-validating-webhook-configuration created
[email protected]:~/demo4#

After installation , Will automatically create a file named gatekeeper-system The namespace of , In this namespace, a series of pod, Confirm these pod All States are running .

[email protected]:~/demo4# kubectl get pods -n gatekeeper-system 
NAME                                                                                READY   STATUS    RESTARTS   AGE
gatekeeper-audit-7b67879df4-sm4fw               1/1    Running    0  30s
gatekeeper-controller-manager-6cb56f759f-96jmn  1/1     Running   0  30s
gatekeeper-controller-manager-6cb56f759f-hnxc4  1/1     Running   0   30s
gatekeeper-controller-manager-6cb56f759f-sv5zp  1/1     Running   0   30s
[email protected]:~/demo4#

gatekeeper A series of resource types will be created .

[email protected]:~/demo4# kubectl get crd | grep gatekeep
configs.config.gatekeeper.sh                          2022-05-09T11:51:01Z
constraintpodstatuses.status.gatekeeper.sh            2022-05-09T11:51:01Z
constrainttemplatepodstatuses.status.gatekeeper.sh    2022-05-09T11:51:01Z
constrainttemplates.templates.gatekeeper.sh           2022-05-09T11:51:01Z
providers.externaldata.gatekeeper.sh                  2022-05-09T11:51:01Z
[email protected]:~/demo4#

Create an image for testing
At all nodes nginx The image is relabeled as hub.c.163.com/library/nginx

[email protected]:~# nerdctl tag nginx hub.c.163.com/library/nginx
[email protected]:~#

Now there are no restrictions on which image to use , So if you use mirroring nginx hub.c.163.com/library/nginx establish pod It can also be created .
establish pod1.yaml The contents are as follows .

[email protected]:~/demo4# cat pod1.yaml 
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: pod1
  name: pod1
spec:
  terminationGracePeriodSeconds: 0
  containers:
  - image: hub.c.163.com/library/nginx
    imagePullPolicy: IfNotPresent
    name: pod1
    resources: {
    }
  dnsPolicy: ClusterFirst
  restartPolicy: Always
status: {
    }

[email protected]:~/demo4# 

Start to create pod.

[email protected]:~/demo4# kubectl apply -f pod1.yaml 
pod/pod1 created
[email protected]:~/demo4# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
pod1   1/1     Running   0          3s
[email protected]:~/demo4# 

You can see it here pod Can be created , That is, it can be used normally now hub.c.163.com/library/nginx This is a mirror image . Let's take this pod Delete .

[email protected]:~/demo4# kubectl delete pod pod1
pod "pod1" deleted
[email protected]:~/demo4#

Prohibit the use of specific images

We first customize a type as blacklistimages Of CRD The resource type , Used to prohibit hub.c.163.com The mirror image of the beginning .
notes ： As if there were pod This resource type , We can create pod1、pod2、pod3 Such as these pod resources . We need to create a resource type before we can create a specific resource .
Check if... Exists first blacklistimages Type of resource .

[email protected]:~/demo4# kubectl get blacklistimages
error: the server doesn't have a resource type "blacklistimages"
[email protected]s61:~/demo4# 

Prompt here does not exist blacklistimages This resource type , So first through CRD Create this type . Create below gatekeeper-blk-type.yaml The contents are as follows .

[email protected]:~/demo4# cat gatekeeper-blk-type.yaml
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: blacklistimages
spec:
  crd:
    spec:
      names:
        kind: BlacklistImages
  targets:
  - rego: |
      package k8strustedimages

      images {
    
        image := input.review.object.spec.containers[_].image
        not startswith(image, "hub.c.163.com/")
      }

      violation[{
    "msg": msg}] {
    
        not images
        msg := " Unavailable image !"
      }
    target: admission.k8s.gatekeeper.sh
[email protected]:~/demo4#

It is specified here if hub.c.163.com The beginning of the mirror , Then the prompt “ Unavailable image ！”.
Create this resource .

[email protected]:~/demo4# kubectl apply -f gatekeeper-blk-type.yaml 
constrainttemplate.templates.gatekeeper.sh/blacklistimages created
[email protected]:~/demo4# 
[email protected]:~/demo4# kubectl get blacklistimages
No resources found
[email protected]:~/demo4# 

Here's a hint "No resources found", Indicates that the resource type already exists blacklistimages 了 , But no resources have been created under this type .
Now create a name pod-blk-img Of BlacklistImages.

[email protected]:~/demo4# cat gatekeeper-blacklist.yaml 
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: BlacklistImages
metadata:
  generation: 1
  managedFields:
  name: pod-blk-img
  resourceVersion: "14449"
spec:
  match:
    kinds:
    - apiGroups:
      - ""
      kinds:
      - Pod
[email protected]:~/demo4#

What is defined here pod-blk-img Will be used for pod The creation of , First name it pod-blk-img Of blacklistimages created .

[email protected]:~/demo4# kubectl apply -f gatekeeper-blacklist.yaml 
blacklistimages.constraints.gatekeeper.sh/pod-blk-img  created
[email protected]:~/demo4#

[email protected]:~/demo4# kubectl get blacklistimages
NAME              AGE
pod-blk-img       74s
[email protected]:~/demo4# 

Now create again pod1.

[email protected]:~/demo4# kubectl apply -f pod1.yaml 
Error from server ([pod-blk-img]  Unavailable image !): error when creating "pod1.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [pod-blk-img]  Unavailable image !
[email protected]:~/demo4#

here pod Create failure , It means that it can no longer be used hub.c.163.com The mirror image of .

Delete pod-blk-img.

[email protected]:~/demo4# kubectl delete -f gatekeeper-blacklist.yaml 
blacklistimages.constraints.gatekeeper.sh "pod-blk-img" deleted
[email protected]:~/demo4# 

Delete blacklistimages This type of resource .

[email protected]:~/demo4# kubectl delete -f gatekeeper-blk-type.yaml 
constrainttemplate.templates.gatekeeper.sh "blacklistimages" deleted
[email protected]:~/demo4#

It is forbidden to create LB Type of svc

Now let's do another exercise , Used to prohibit creation LoadBalancer Type of service.
The first pod1 created .

[email protected]:~/demo4# kubectl apply -f pod1.yaml 
pod/pod1 created
[email protected]:~/demo4# kubectl get pods
NAME   READY   STATUS    RESTARTS   AGE
pod1   1/1     Running   0          2s
[email protected]:~/demo4# 

Without any restrictions , So let's create one LoadBalancer Type of service.

[email protected]:~/demo4# kubectl expose --name=svc1 pod pod1 --port=80 --type=LoadBalancer
service/svc1 exposed
[email protected]:~/demo4# kubectl get svc svc1
NAME   TYPE          CLUSTER-IP       EXTERNAL-IP      PORT(S)       AGE
svc1   LoadBalancer  10.110.158.128   192.168.26.240   80:32109/TCP   5s
[email protected]:~/demo4#

At this time, you can create a type of LoadBalancer Of service Of , Let's take this service Delete .

[email protected]:~/demo4# kubectl delete svc svc1
service "svc1" deleted
[email protected]:~/demo4#

Now we are going to create a CRD Resource type of lbtypesvcnotallowed, First determine whether this resource type exists .

[email protected]:~/demo4# kubectl get lbtypesvcnotallowed
error: the server doesn't have a resource type "lbtypesvcnotallowed"
[email protected]:~/demo4# 

Report errors , It means that it doesn't exist yet lbtypesvcnotallowed This resource type .

First create the resource type lbtypesvcnotallowed, To write aa-tmp.yaml The contents are as follows .

[email protected]:~/demo4# cat aa-tmp.yaml
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: lbtypesvcnotallowed
spec:
  crd:
    spec:
      names:
        kind: LBTypeSvcNotAllowed
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package kubernetes.admission
        violation[{
    "msg": msg}] {
    
                    input.review.kind.kind = "Service"
                    input.review.operation = "CREATE"
                    input.review.object.spec.type = "LoadBalancer"
                    msg := " Creation of LB Type of service!"
        }
[email protected]:~/demo4#

Here we create a name called lbtypesvcnotallowed Resource type of , It is forbidden to create LB Type of service, If created , Will have a " Creation of LB Type of service!" Report errors .
Next, create a resource type .

[email protected]:~/demo4# kubectl apply -f aa-tmp.yaml 
constrainttemplate.templates.gatekeeper.sh/lbtypesvcnotallowed created
[email protected]:~/demo4# 
[email protected]:~/demo4# kubectl get lbtypesvcnotallowed
No resources found
[email protected]:~/demo4# 

It already exists lbtypesvcnotallowed This resource type , But there are no resources below .
The name created below is deny-create-lb-type-svc Of lbtypesvcnotallowed, To write bb-contraint.yaml The contents are as follows .

[email protected]:~/demo4# cat bb-contraint.yaml 
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: LBTypeSvcNotAllowed
metadata:
  name: deny-create-lb-type-svc
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Service"]
    namespaces:
      - "default"
[email protected]:~/demo4#

Create this resource .

[email protected]:~/demo4# kubectl apply -f bb-contraint.yaml
lbtypesvcnotallowed.constraints.gatekeeper.sh/deny-create-lb-type-svc created
[email protected]:~/demo4# 
[email protected]:~/demo4# kubectl get lbtypesvcnotallowed
NAME                      AGE
deny-create-lb-type-svc   4s
[email protected]:~/demo4#

Create again LoadBalancer Of service.

[email protected]:~/demo4# kubectl expose --name=svc1 pod pod1 --port=80 --type=LoadBalancer
Error from server ([deny-create-lb-type-svc]  Creation of LB Type of service!): admission webhook "validation.gatekeeper.sh" denied the request: [deny-create-lb-type-svc]  Creation of LB Type of service!
[email protected]:~/demo4#

You can see here that creation is no longer allowed .
Delete lbtypesvcnotallowed, Then the following resources will be deleted together .

[email protected]:~/demo4# kubectl delete -f aa-tmp.yaml 
constrainttemplate.templates.gatekeeper.sh "lbtypesvcnotallowed" deleted
[email protected]:~/demo4#