昨天收到一个朋友的信息，说不小心把集群的业务namespace干掉了，导致整个业务都停滞了，问我有没有禁止删除namespace的方案。
在我的记忆里，Kubernetes的准入里并没有这个控制器，所以我就给他说需要自己开发一个准入控制器来实现自己的目标。
作为人，何为正确！我不能只脱裤子，不放屁。所以这里也整理了一下如何自定义Kubernetes的准入控制器。
理论介绍准入控制器（Admission Controller）位于 API Server 中，在对象被持久化之前，准入控制器拦截对 API Server 的请求，一般用来做身份验证和授权。其中包含两个特殊的控制器：MutatingAdmissionWebhook 和 ValidatingAdmissionWebhook。
MutatingAdmissionWebhook ：用于变更请求对象，比如istio为每个Pod注入sidecar，就是通过它实现。ValidatingAdmissionWebhook：用于验证请求对象整个准入控制器的流程如下：
当 API 请求进入时，mutating 和 validating 控制器使用配置中的外部 webhooks 列表并发调用，规则如下：
如果所有的 webhooks 批准请求，准入控制链继续流转。如果有任意一个 webhooks 阻止请求，那么准入控制请求终止，并返回第一个 webhook 阻止的原因。其中，多个 webhooks 阻止也只会返回第一个 webhook 阻止的原因。如果在调用 webhook 过程中发生错误，那么请求会被终止或者忽略 webhook。准入控制器是在 API Server 的启动参数中配置的。一个准入控制器可能属于以上两者中的一种，也可能两者都属于。
我们在部署 Kubernetes 集群的时候都会默认开启一系列准入控制器，如果没有设置这些准入控制器的话可以说你的 Kubernetes 集群就是在裸奔，应该叫管理员为集群添加准入控制器。
代码实现实现逻辑在开发之前先大致了解一下准入控制器的Webhook的大致实现逻辑：
Webhook是一个标准的HTTP服务，接收HTTP请求接收到的请求是一个AdmissionReview对象然后我们自定义的Hook会处理这个AdmissionReview对象处理完过后再返回一个AdmissionReview对象，这里面会包含处理结果AdmissionReview的结构体如下：
// AdmissionReview describes an admission review request/response.type AdmissionReview struct { metav1.TypeMeta `json:",inline"`// Request describes the attributes for the admission request.// +optional Request *AdmissionRequest `json:"request,omitempty" protobuf:"bytes,1,opt,name=request"`// Response describes the attributes for the admission response.// +optional Response *AdmissionResponse `json:"response,omitempty" protobuf:"bytes,2,opt,name=response"`}从代码的命名中可以很清晰的看出，在请求发送到 WebHook 时我们只需要关注内部的 AdmissionRequest(实际入参)，在我们编写的 WebHook 处理完成后只需要返回包含有 AdmissionResponse(实际返回体) 的 AdmissionReview 对象即可；总的来说 AdmissionReview 对象是个套壳，请求是里面的 AdmissionRequest，响应是里面的 AdmissionResponse。
（1）首先创建一个HTTP Server，监听端口，接收请求
package mainimport ("context""flag""github.com/joker-bai/validate-namespace/http" log "k8s.io/klog/v2""os""os/signal""syscall")var ( tlscert, tlskey, port string)func main() { flag.StringVar(&tlscert, "tlscert", "/etc/certs/cert.pem", "Path to the TLS certificate") flag.StringVar(&tlskey, "tlskey", "/etc/certs/key.pem", "Path to the TLS key") flag.StringVar(&port, "port", "8443", "The port to listen") flag.Parse() server := http.NewServer(port)go func() {if err := server.ListenAndServeTLS(tlscert, tlskey); err != nil { log.Errorf("Failed to listen and serve: %v", err) } }() log.Infof("Server running in port: %s", port)// listen shutdown signal signalChan := make(chan os.Signal, 1) signal.Notify(signalChan, syscall.SIGINT, syscall.SIGTERM) <-signalChan log.Info("Shutdown gracefully...")if err := server.Shutdown(context.Background()); err != nil { log.Error(err) }}由于准入控制器和Webhook之间需要使用TLS进行通信，所以上面监听的端口是TLS端口，通过server.ListenAndServeTLS实现，后续在部署服务的时候需要把证书挂到相应的目录中。
（2）定义Handler，将请求分发到具体的处理方法
package httpimport ("fmt""github.com/joker-bai/validate-namespace/namespace""net/http")// NewServer creates and return a http.Serverfunc NewServer(port string) *http.Server {// Instances hooks nsValidation := namespace.NewValidationHook()// Routers ah := newAdmissionHandler() mux := http.NewServeMux() mux.Handle("/healthz", healthz()) mux.Handle("/validate/delete-namespace", ah.Serve(nsValidation))return &http.Server{ Addr: fmt.Sprintf(":%s", port), Handler: mux, }}实现admissionHandler，主要作用是将http body的内容解析成AdmissionReview对象，然后调用具体的Hook处理，再将结果放到AdmissionReview中，返回给客户端。
package httpimport ("encoding/json""fmt""io""net/http""github.com/douglasmakey/admissioncontroller""k8s.io/api/admission/v1beta1" admission "k8s.io/api/admission/v1beta1" meta "k8s.io/apimachinery/pkg/apis/meta/v1""k8s.io/apimachinery/pkg/runtime""k8s.io/apimachinery/pkg/runtime/serializer" log "k8s.io/klog/v2")// admissionHandler represents the HTTP handler for an admission webhooktype admissionHandler struct { decoder runtime.Decoder}// newAdmissionHandler returns an instance of AdmissionHandlerfunc newAdmissionHandler() *admissionHandler {return &admissionHandler{ decoder: serializer.NewCodecFactory(runtime.NewScheme()).UniversalDeserializer(), }}// Serve returns a http.HandlerFunc for an admission webhookfunc (h *admissionHandler) Serve(hook admissioncontroller.Hook) http.HandlerFunc {return func(w http.ResponseWriter, r *http.Request) { w.Header().Set("Content-Type", "application/json")if r.Method != http.MethodPost { http.Error(w, fmt.Sprint("invalid method only POST requests are allowed"), http.StatusMethodNotAllowed)return }if contentType := r.Header.Get("Content-Type"); contentType != "application/json" { http.Error(w, fmt.Sprint("only content type 'application/json' is supported"), http.StatusBadRequest)return } body, err := io.ReadAll(r.Body)if err != nil { http.Error(w, fmt.Sprintf("could not read request body: %v", err), http.StatusBadRequest)return }var review admission.AdmissionReviewif _, _, err := h.decoder.Decode(body, nil, &review); err != nil { http.Error(w, fmt.Sprintf("could not deserialize request: %v", err), http.StatusBadRequest)return }if review.Request == nil { http.Error(w, "malformed admission review: request is nil", http.StatusBadRequest)return } result, err := hook.Execute(review.Request)if err != nil { log.Error(err) w.WriteHeader(http.StatusInternalServerError)return } admissionResponse := v1beta1.AdmissionReview{ Response: &v1beta1.AdmissionResponse{ UID: review.Request.UID, Allowed: result.Allowed, Result: &meta.Status{Message: result.Msg}, }, } res, err := json.Marshal(admissionResponse)if err != nil { log.Error(err) http.Error(w, fmt.Sprintf("could not marshal response: %v", err), http.StatusInternalServerError)return } log.Infof("Webhook [%s - %s] - Allowed: %t", r.URL.Path, review.Request.Operation, result.Allowed) w.WriteHeader(http.StatusOK) w.Write(res) }}func healthz() http.HandlerFunc {return func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(http.StatusOK) w.Write([]byte("ok")) }}上面处理是通过hook.Execute来处理请求，这是admissionController内部实现的一个结构体，它为每个操作定义了一个方法，如下：
// AdmitFunc defines how to process an admission requesttype AdmitFunc func(request *admission.AdmissionRequest) (*Result, error)// Hook represents the set of functions for each operation in an admission webhook.type Hook struct { Create AdmitFunc Delete AdmitFunc Update AdmitFunc Connect AdmitFunc}我们就需要实现具体的AdmitFunc，并注册。
（3）将自己实现的方法注册到Hook中。
package namespaceimport ("github.com/douglasmakey/admissioncontroller")// NewValidationHook delete namespace validation hookfunc NewValidationHook() admissioncontroller.Hook {return admissioncontroller.Hook{ Delete: validateDelete(), }}（4）实现具体的AdmitFunc
package namespaceimport ("github.com/douglasmakey/admissioncontroller" log "k8s.io/klog/v2""k8s.io/api/admission/v1beta1")func validateDelete() admissioncontroller.AdmitFunc {return func(r *v1beta1.AdmissionRequest) (*admissioncontroller.Result, error) {if r.Kind.Kind == "Namespace" { log.Info("You cannot delete namespace: ", r.Name)return &admissioncontroller.Result{Allowed: false}, nil } else {return &admissioncontroller.Result{Allowed: true}, nil } }}这里实现很简单，如果Kind为Namespace，就拒绝操作。
部署测试上面完成了业务逻辑开发，下面就把它部署到Kubernetes集群测试一番。
部署（1）编写Dockerfile，将应用打包成镜像
FROM golang:1.17.5 AS build-envENV GOPROXY https://goproxy.cnADD . /go/src/appWORKDIR /go/src/appRUN go mod tidyRUN cd cmd && GOOS=linux GOARCH=amd64 go build -v -a -ldflags '-extldflags "-static"' -o /go/src/app/app-server /go/src/app/cmd/main.goFROM registry.cn-hangzhou.aliyuncs.com/coolops/ubuntu:22.04ENV TZ=Asia/ShanghaiCOPY --from=build-env /go/src/app/app-server /opt/app-serverWORKDIR /optEXPOSE 80CMD [ "./app-server" ]（2）创建TLS证书，使用脚本进行创建
#!/bin/bashset -eusage() { cat <<EOFGenerate certificate suitable for use with an sidecar-injector webhook service.This script uses k8s' CertificateSigningRequest API to a generate acertificate signed by k8s CA suitable for use with sidecar-injector webhookservices. This requires permissions to create and approve CSR. Seehttps://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster fordetailed explantion and additional instructions.The server key/cert k8s CA cert are stored in a k8s secret.usage: ${0} [OPTIONS]The following flags are required. --service Service name of webhook. --namespace Namespace where webhook service and secret reside. --secret Secret name for CA certificate and server certificate/key pair.EOF exit 1}while [[ $# -gt 0 ]]; do case ${1} in --service) service="$2" shift ;; --secret) secret="$2" shift ;; --namespace) namespace="$2" shift ;; *) usage ;; esac shiftdone[ -z ${service} ] && service=validate-delete-namespace[ -z ${secret} ] && secret=validate-delete-namespace-tls[ -z ${namespace} ] && namespace=defaultif [ ! -x "$(command -v openssl)" ]; then echo "openssl not found" exit 1ficsrName=${service}.${namespace}tmpdir=$(mktemp -d)echo "creating certs in tmpdir ${tmpdir} "cat <<EOF >> ${tmpdir}/csr.conf[req]req_extensions = v3_reqdistinguished_name = req_distinguished_name[req_distinguished_name][ v3_req ]basicConstraints = CA:FALSEkeyUsage = nonRepudiation, digitalSignature, keyEnciphermentextendedKeyUsage = serverAuthsubjectAltName = @alt_names[alt_names]DNS.1 = ${service}DNS.2 = ${service}.${namespace}DNS.3 = ${service}.${namespace}.svcEOFopenssl genrsa -out ${tmpdir}/server-key.pem 2048openssl req -new -key ${tmpdir}/server-key.pem -subj "/CN=${service}.${namespace}.svc" -out ${tmpdir}/server.csr -config ${tmpdir}/csr.conf# clean-up any previously created CSR for our service. Ignore errors if not present.kubectl delete csr ${csrName} 2>/dev/ || true# create server cert/key CSR and send to k8s APIcat <<EOF | kubectl create -f -apiVersion: certificates.k8s.io/v1beta1kind: CertificateSigningRequestmetadata: name: ${csrName}spec: groups: - system:authenticated request: $(cat ${tmpdir}/server.csr | base64 | tr -d '\n') usages: - digital signature - key encipherment - server authEOF# verify CSR has been createdwhile true; do kubectl get csr ${csrName} if [ "$?" -eq 0 ]; then break fidone# approve and fetch the signed certificatekubectl certificate approve ${csrName}# verify certificate has been signedfor x in $(seq 10); do serverCert=$(kubectl get csr ${csrName} -o jsonpath='{.status.certificate}') if [[ ${serverCert} != '' ]]; then break fi sleep 1doneif [[ ${serverCert} == '' ]]; then echo "ERROR: After approving csr ${csrName}, the signed certificate did not appear on the resource. Giving up after 10 attempts." >&2 exit 1fiecho ${serverCert} | openssl base64 -d -A -out ${tmpdir}/server-cert.pem# create the secret with CA cert and server cert/keykubectl create secret generic ${secret} \ --from-file=key.pem=${tmpdir}/server-key.pem \ --from-file=cert.pem=${tmpdir}/server-cert.pem \ --dry-run -o yaml | kubectl -n ${namespace} apply -f -（3）编写Deployment部署服务
apiVersion: apps/v1kind: Deploymentmetadata:name: validate-delete-namespacelabels:app: validate-delete-namespacespec:replicas: 1selector:matchLabels:app: validate-delete-namespacetemplate:metadata:labels:app: validate-delete-namespacespec:containers:- name: serverimage: registry.cn-hangzhou.aliyuncs.com/coolops/validate-delete-namespace:latestimagePullPolicy: AlwayslivenessProbe:httpGet:path: /healthzport: 8443scheme: HTTPSports:- containerPort: 8443volumeMounts:- name: tls-certsmountPath: /etc/certsreadOnly: truevolumes:- name: tls-certssecret:secretName: validate-delete-namespace-tls---apiVersion: v1kind: Servicemetadata:name: validate-delete-namespacespec:selector:app: validate-delete-namespaceports:- port: 443targetPort: 8443（4）部署Webhook
apiVersion: admissionregistration.k8s.io/v1beta1kind: ValidatingWebhookConfigurationmetadata: name: validate-delete-namespacewebhooks: - name: validate-delete-namespace.default.svc.cluster.local clientConfig: service: namespace: default name: validate-delete-namespace path: "/validate/delete-namespace" caBundle: "${CA_BUNDLE}" rules: - operations: - DELETE apiGroups: - "" apiVersions: - "v1" resources: - namespaces failurePolicy: Ignore这里有一个${CA_BUNDLE}占位符，在创建Webhook的时候要将其替换掉，使用如下命令：
cat ./validate-delete-namespace.yaml | sh ./patch-webhook-ca.sh > ./webhook.yaml然后创建webhook.yaml即可。
kubectl apply -f webhook.yaml上面的所有文件都在代码库里，可以直接使用脚本进行部署。
# sh deploy.sh creating certs in tmpdir /tmp/tmp.SvMHWcPI6x Generating RSA private key, 2048 bit long modulus..........................................+++.............................................................+++e is 65537 (0x10001)certificatesigningrequest.certificates.k8s.io/validate-delete-namespace.default createdNAME AGE REQUESTOR CONDITIONvalidate-delete-namespace.default 0s kubernetes-admin Pendingcertificatesigningrequest.certificates.k8s.io/validate-delete-namespace.default approvedsecret/validate-delete-namespace-tls createdCreating k8s admission deploymentdeployment.apps/validate-delete-namespace createdservice/validate-delete-namespace createdvalidatingwebhookconfiguration.admissionregistration.k8s.io/validate-delete-namespace created执行完成过后，可以查看具体的信息。
# kubectl get poNAME READY STATUS RESTARTS AGEvalidate-delete-namespace-74c9b8b7bd-5g9zv 1/1 Running 0 3s# kubectl get secretNAME TYPE DATA AGEdefault-token-kx5wf kubernetes.io/service-account-token 3 72dvalidate-delete-namespace-tls Opaque 2 53s# kubectl get ValidatingWebhookConfigurationNAME CREATED ATvalidate-delete-namespace 2022-06-24T09:39:26Z测试（1）首先打开webhook的pod日志
# kubectl logs validate-delete-namespace-74c9b8b7bd-5g9zv -fI0624 17:39:27.858753 1 main.go:30] Server running in port: 8443（2）创建一个namespace并删除
# kubectl create ns joker# kubectl get ns | grep jokerjoker Active 4h5m# kubectl delete ns jokerError from server: admission webhook "validate-delete-namespace.default.svc.cluster.local" denied the request without explanation# kubectl get ns | grep jokerjoker Active 4h5m可以发现我们的删除操作被拒绝了，并且查看namespace还存在。
我们也可以到日志中查看，如下：
# kubectl logs validate-delete-namespace-74c9b8b7bd-5g9zv -fI0624 17:39:27.858753 1 main.go:30] Server running in port: 84432022/06/24 17:43:34 You cannot delete namespace: jokerI0624 17:43:34.664945 1 handler.go:94] Webhook [/validate/delete-namespace - DELETE] - Allowed: false2022/06/24 17:43:34 You cannot delete namespace: jokerI0624 17:43:34.667043 1 handler.go:94] Webhook [/validate/delete-namespace - DELETE] - Allowed: false上面就是简单的实现了一个准入控制器，
只要思想不滑坡，办法总比困难多。
感谢万能的百度，感谢牛逼的网友。
参考https://www.qikqiak.com/post/k8s-admission-webhookhttps://github.com/douglasmakey/admissioncontrollerhttps://mritd.com/2020/08/19/write-a-dynamic-admission-control-webhook/最后，求关注。如果你还想看更多优质原创文章，欢迎关注我们的「运维开发故事」。如果我的文章对你有所帮助，还请帮忙一下，你的支持会激励我输出更高质量的文章，非常感谢！
你还可以把我的设为「星标」，这样当文章更新时，你会在第一时间收到推送消息，避免错过我的文章更新。
标签：控制器 runningman