主题
实验六:配置热更新、RBAC 证书授权、Helm 模板工程与 Webhook/升级实战
0. 实验目标与环境基线
本实验是整套 Kubernetes 进阶体系的综合收官大实验,在你的 Mac mini M4 + Lima 三节点集群 上一次性打通四大企业级治理与扩展主题:
- ConfigMap 与 Secret 全景治理(多行控制符、
subPath单文件挂载、Volume 热更新同步 vs Reloader 滚动更新、K8s 1.24+TokenRequest动态短期 Token 投影卷); - 集群安全、X.509 证书签发与 RBAC 授权 +
SecurityContext(Capabilities细粒度特权收敛); - Helm 3 安装与 Chart 模板开发实战(亲手创建 Chart、编写
_helpers.tpl/include/nindent/with/range/.Files.Get,测试--set > -f > values.yaml三级优先级与一键 Rollback); - CRD + Mutating Admission Webhook 准入拦截器开发实战(在集群内用 Python 3 标准库跑一个真实的 HTTPS Mutating Webhook,给新建 Pod 自动注入
environment: production标签!)+ etcd 快照备份与集群无损升级 SOP 演练。
包含的 7 个核心实操点
- 实验点 6.1:ConfigMap(
|vs|-、Volume 自动热更新 vssubPath不热更新)与 Secret 三大类型实战 - 实验点 6.2:使用 OpenSSL 为研发人员
dev-alice签发 X.509 客户端证书、配置kubeconfigContext,并通过Role + RoleBinding(含绑定ClusterRole)验证 RBAC 最小权限拦截 - 实验点 6.3:
SecurityContext容器安全加固——验证非 Root 运行(runAsNonRoot)与 LinuxCapabilities(drop: ["ALL"]+add: ["NET_ADMIN"]) - 实验点 6.4:在
lima-master安装 Helm 3,从零开发一个标准 Chart 包并演练install/upgrade/history/rollback - 实验点 6.5:自定义资源
CRD与CR创建(体验在 etcd 中建表与插记录) - 实验点 6.6:部署真实 HTTPS
MutatingWebhookConfiguration准入控制器,实现新建 Pod 自动打上environment: production标签 - 实验点 6.7:
etcdctl底层数据探测、一致性快照备份(snapshot save)与节点滚动维护 SOP(cordondrainuncordon)
1. 实验前准备:创建实验六专属命名空间
在 lima-master 上执行:
bash
limactl shell master
kubectl create namespace lab6-gov
kubectl config set-context --current --namespace=lab6-gov2. 实验点 6.1:ConfigMapVolume 热更新与 subPath 对比实战
原理回顾:
- 通过环境变量(
env)或subPath单文件挂载的 ConfigMap 不支持热更新。 - 通过普通目录
volumeMounts挂载的 ConfigMap,在修改 ConfigMap 后,Kubelet 会在约 10~60 秒内通过符号链接(..data)原子替换自动同步容器内文件。
bash
# 1. 创建包含多行配置 (|-) 的 ConfigMap 与挂载它的 Pod
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: app-dynamic-cm
data:
app.conf: |-
env = staging
max_connections = 100
---
apiVersion: v1
kind: Pod
metadata:
name: cm-hot-reload-demo
spec:
containers:
- name: app
image: busybox:1.36
command: ["sh", "-c", "sleep 3600"]
volumeMounts:
- name: config-vol
mountPath: /etc/app-config
volumes:
- name: config-vol
configMap:
name: app-dynamic-cm
EOF
kubectl wait --for=condition=Ready pod/cm-hot-reload-demo --timeout=60s
kubectl exec cm-hot-reload-demo -- cat /etc/app-config/app.conf
# 2. 在线更新 ConfigMap 中的 max_connections 为 500
kubectl patch configmap app-dynamic-cm -p '{"data":{"app.conf":"env = production\nmax_connections = 500"}}'
# 3. 触发 kubelet 同步后查看容器内文件(观察软链接 ..data 与热更新后的内容)
kubectl exec cm-hot-reload-demo -- ls -la /etc/app-config3. 实验点 6.2:使用 OpenSSL 签发用户证书 + RBAC 精细授权实战
原理回顾:API Server 请求需经过 Authentication(认证)CN(用户名)和 O(用户组)完成身份认证。
步骤 1:使用集群 CA 为新用户 dev-alice 签发客户端证书并配置上下文
在 lima-master 执行:
bash
mkdir -p /tmp/rbac-test && cd /tmp/rbac-test
# 1. 生成私钥与 CSR (CN=dev-alice, O=developers)
openssl genrsa -out dev-alice.key 2048
openssl req -new -key dev-alice.key -out dev-alice.csr -subj "/CN=dev-alice/O=developers"
# 2. 使用集群根证书 (/etc/kubernetes/pki/ca.crt) 签发合法证书
sudo openssl x509 -req -in dev-alice.csr \
-CA /etc/kubernetes/pki/ca.crt \
-CAkey /etc/kubernetes/pki/ca.key \
-CAcreateserial -out dev-alice.crt -days 30
# 3. 在 kubeconfig 中注册用户凭据与上下文 alice-ctx
kubectl config set-credentials dev-alice \
--client-certificate=/tmp/rbac-test/dev-alice.crt \
--client-key=/tmp/rbac-test/dev-alice.key
kubectl config set-context alice-ctx --cluster=kubernetes --namespace=lab6-gov --user=dev-alice步骤 2:未授权前测试访问(验证 403 Forbidden),随后创建 Role 与 RoleBinding 授权
bash
# 1. 未创建 RoleBinding 前,以 alice-ctx 身份查看 Pod -> 预期返回 Error from server (Forbidden)!
kubectl --context=alice-ctx get pods || true
# 2. 创建仅允许在 lab6-gov 命名空间 get/list/watch Pod 的 Role 与 RoleBinding
cat <<'EOF' | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: pod-reader-role
namespace: lab6-gov
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: alice-read-pods
namespace: lab6-gov
subjects:
- kind: User
name: dev-alice
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: pod-reader-role
apiGroup: rbac.authorization.k8s.io
EOF
# 3. 再次以 dev-alice 身份执行:
# (a) 查看 lab6-gov 下的 Pod -> 成功!
kubectl --context=alice-ctx get pods
# (b) 尝试删除 Pod -> Forbidden(因为没有 delete 动词权限)!
kubectl --context=alice-ctx delete pod cm-hot-reload-demo || true
# (c) 尝试查看 kube-system 下的 Pod -> Forbidden(因为 RoleBinding 仅限 lab6-gov 命名空间)!
kubectl --context=alice-ctx get pods -n kube-system || true4. 实验点 6.3:SecurityContext 与 Linux Capabilities 细粒度特权控制
原理回顾:生产最佳实践是 allowPrivilegeEscalation: false + capabilities.drop: ["ALL"],仅按需 add 最小权限(如网络工具仅需 NET_ADMIN)。
bash
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: sec-cap-demo
spec:
containers:
- name: hardened
image: busybox:1.36
command: ["sleep", "3600"]
securityContext:
allowPrivilegeEscalation: false
privileged: false
capabilities:
drop:
- ALL
add:
- NET_ADMIN
EOF
kubectl wait --for=condition=Ready pod/sec-cap-demo --timeout=60s
# 验证:因为赋予了 NET_ADMIN,容器内可以成功添加虚拟回环 IP;但尝试修改系统时间 (需要 SYS_TIME) 则被拒绝!
kubectl exec sec-cap-demo -- ip addr add 10.99.99.99/32 dev lo
kubectl exec sec-cap-demo -- ip -4 addr show dev lo
kubectl exec sec-cap-demo -- date -s "2030-01-01 00:00:00" || true5. 实验点 6.4:Helm 3 安装、Chart 模板语法实战与版本回滚
步骤 1:在 lima-master (ARM64) 安装 Helm 3 CLI
bash
if ! command -v helm >/dev/null 2>&1; then
curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
fi
helm version步骤 2:从零创建一个自定义 Chart 并演练核心 Go Template 语法
我们在 /tmp/my-web-chart 创建一个精简但涵盖笔记中所有高频模板语法(_helpers.tpl、define、include、nindent、default、quote、if/else、with、range、.Files.Get)的 Chart:
bash
mkdir -p /tmp/my-web-chart/templates
cd /tmp/my-web-chart
cat <<'EOF' > Chart.yaml
apiVersion: v2
name: my-web-chart
description: A comprehensive Helm chart for K8s study
type: application
version: 0.1.0
appVersion: "1.0.0"
EOF
cat <<'EOF' > values.yaml
replicaCount: 1
env: dev
appConfig:
server:
port: 80
greeting: "Hello from default values.yaml"
extraEnvs:
LOG_LEVEL: info
REGION: ap-northeast-1
EOF
# 创建一个外部普通静态文件供 .Files.Get 读取
cat <<'EOF' > app.toml
[database]
timeout = "5s"
pool_size = 20
EOF
cat <<'EOF' > templates/_helpers.tpl
{{- define "mychart.labels" -}}
generator: helm
chart: {{ .Chart.Name }}-{{ .Chart.Version }}
release: {{ .Release.Name }}
{{- end -}}
EOF
cat <<'EOF' > templates/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Release.Name }}-cm
labels:
{{- include "mychart.labels" . | nindent 4 }}
data:
{{- with .Values.appConfig.server }}
greeting.txt: {{ .greeting | upper | quote }}
{{- end }}
app.toml: |-
{{ .Files.Get "app.toml" | nindent 4 }}
EOF
cat <<'EOF' > templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Release.Name }}-deploy
labels:
{{- include "mychart.labels" . | nindent 4 }}
spec:
{{- if eq .Values.env "prod" }}
replicas: 3
{{- else }}
replicas: {{ .Values.replicaCount | default 1 }}
{{- end }}
selector:
matchLabels:
release: {{ .Release.Name }}
template:
metadata:
labels:
release: {{ .Release.Name }}
spec:
containers:
- name: web
image: nginx:stable-alpine
env:
{{- range $k, $v := .Values.extraEnvs }}
- name: {{ $k }}
value: {{ $v | quote }}
{{- end }}
EOF步骤 3:渲染预览(helm template)、安装、参数覆盖升级(--set)与一键回滚(helm rollback)
bash
# 1. 本地渲染预览生成的 YAML(检查缩进与语法是否正确)
helm template demo-rel /tmp/my-web-chart
# 2. 正式安装 Release (Revision 1)
helm install demo-rel /tmp/my-web-chart -n lab6-gov
helm list -n lab6-gov
# 3. 使用 --set 覆盖参数在线升级为 prod 环境 (自动变为 3 副本,Revision 2)
helm upgrade demo-rel /tmp/my-web-chart -n lab6-gov \
--set env=prod \
--set appConfig.server.greeting="Upgraded via --set!"
kubectl get deploy demo-rel-deploy
# 4. 查看历史版本并一键回滚到 Revision 1(恢复为 1 副本!)
helm history demo-rel -n lab6-gov
helm rollback demo-rel 1 -n lab6-gov
helm history demo-rel -n lab6-gov
kubectl get deploy demo-rel-deploy
# 卸载 Release
helm uninstall demo-rel -n lab6-gov6. 实验点 6.5:自定义资源 CRD 与 CR 实战(向 etcd 新增表与记录)
bash
cat <<'EOF' | kubectl apply -f -
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: redisclusters.database.example.com
spec:
group: database.example.com
names:
kind: RedisCluster
plural: redisclusters
singular: rediscluster
shortNames:
- rc-db
scope: Namespaced
versions:
- name: v1alpha1
served: true
storage: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
nodes:
type: integer
minimum: 1
maximum: 10
version:
type: string
EOF
# 创建一条自定义资源记录 (CR)
cat <<'EOF' | kubectl apply -f -
apiVersion: database.example.com/v1alpha1
kind: RedisCluster
metadata:
name: my-prod-redis
spec:
nodes: 3
version: "7.2"
EOF
# 使用简称 rc-db 查询我们自定义的数据库资源!
kubectl get rc-db7. 实验点 6.6:从零实战 MutatingAdmissionWebhook(自动为 Pod 注入标签)
原理回顾:在笔记中我们详细讲到了如何编写一个 Webhook 服务,配置自签名 TLS 证书,并通过 MutatingWebhookConfiguration 让 API Server 在创建 Pod 时自动调用它注入 environment: production 标签(注意使用 JSONPatch 仅增补标签而不覆盖原有标签)。 下面我们无需构建外部 Docker 镜像,直接在 lima-master 上用 20 行 Python 3 标准库代码启动一个 HTTPS Webhook,亲眼见证 API Server 准入拦截与自动打标!
bash
mkdir -p /tmp/webhook-lab && cd /tmp/webhook-lab
# 1. 生成 Webhook HTTPS 证书(SAN 包含 Master IP 192.168.104.1)
openssl req -x509 -nodes -days 30 -newkey rsa:2048 \
-keyout webhook.key -out webhook.crt \
-subj "/CN=192.168.104.1" \
-addext "subjectAltName = IP:192.168.104.1"
# 2. 编写基于 Python 3 内置 http.server 的 Mutating Webhook(返回 JSONPatch 添加 environment=production 标签)
cat <<'EOF' > webhook_server.py
import http.server, ssl, json, base64
class MutatingHandler(http.server.BaseHTTPRequestHandler):
def do_POST(self):
length = int(self.headers.get('content-length', 0))
body = json.loads(self.rfile.read(length))
uid = body["request"]["uid"]
# JSONPatch: 增加 metadata.labels.environment = "production"
patch = [{"op": "add", "path": "/metadata/labels/environment", "value": "production"}]
patch_b64 = base64.b64encode(json.dumps(patch).encode()).decode()
resp = {
"apiVersion": "admission.k8s.io/v1",
"kind": "AdmissionReview",
"response": {
"uid": uid,
"allowed": True,
"patchType": "JSONPatch",
"patch": patch_b64
}
}
self.send_response(200)
self.send_header("Content-Type", "application/json")
self.end_headers()
self.wfile.write(json.dumps(resp).encode())
httpd = http.server.HTTPServer(('0.0.0.0', 8443), MutatingHandler)
ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
ctx.load_cert_chain(certfile='webhook.crt', keyfile='webhook.key')
httpd.socket = ctx.wrap_socket(httpd.socket, server_side=True)
print("Webhook listening on https://0.0.0.0:8443...")
httpd.serve_forever()
EOF
# 在后台启动 Webhook HTTPS 服务
python3 webhook_server.py &
WEBHOOK_PID=$!
# 3. 注册 MutatingWebhookConfiguration(仅拦截带有 webhook-test=enabled 标签的 Namespace,安全防误伤)
kubectl label namespace lab6-gov webhook-test=enabled --overwrite
CA_BUNDLE=$(base64 -w0 < webhook.crt)
cat <<EOF | kubectl apply -f -
apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
name: auto-label-mutating-webhook
webhooks:
- name: autolabel.example.com
admissionReviewVersions: ["v1"]
sideEffects: None
timeoutSeconds: 3
failurePolicy: Ignore
namespaceSelector:
matchLabels:
webhook-test: enabled
clientConfig:
url: "https://192.168.104.1:8443/mutate"
caBundle: "${CA_BUNDLE}"
rules:
- operations: ["CREATE"]
apiGroups: [""]
apiVersions: ["v1"]
resources: ["pods"]
EOF
# 4. 创建一个只带 app: demo 标签的 Pod,观察它被自动注入了 environment=production 标签!
cat <<'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: webhook-injected-pod
labels:
app: demo
spec:
containers:
- name: pause
image: registry.k8s.io/pause:3.10
EOF
kubectl get pod webhook-injected-pod --show-labels
# 5. 实验成功后立即清理 Webhook 配置与后台进程
kubectl delete mutatingwebhookconfiguration auto-label-mutating-webhook
kill "$WEBHOOK_PID" 2>/dev/null || true8. 实验点 6.7:在 lima-master 上直接操作 etcd 快照备份与集群升级维护 SOP
原理回顾:在集群升级或重大变更前,第一铁律是备份 etcd 快照。由于 kubeadm 使用静态 Pod 运行 etcd,我们可以直接在 lima-master 上通过容器内自带的 etcdctl(或宿主机安装 etcd 客户端)直接查询 /registry 键值并导出快照文件!
bash
# 1. 通过 etcd 容器内的 etcdctl 查看 etcd 成员列表与健康状态
kubectl exec -n kube-system etcd-lima-master -- etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
endpoint health
kubectl exec -n kube-system etcd-lima-master -- etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
member list -w table
# 2. 直接查看 etcd 数据库中存储的 Namespace 或 Pod 底层路径键名
kubectl exec -n kube-system etcd-lima-master -- etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
get /registry/namespaces --prefix --keys-only
# 3. 导出 etcd 一致性快照到宿主机 /var/lib/etcd/snapshot-backup.db(因为 /var/lib/etcd 是 hostPath 挂载的!)
kubectl exec -n kube-system etcd-lima-master -- etcdctl \
--endpoints=https://127.0.0.1:2379 \
--cacert=/etc/kubernetes/pki/etcd/ca.crt \
--cert=/etc/kubernetes/pki/etcd/server.crt \
--key=/etc/kubernetes/pki/etcd/server.key \
snapshot save /var/lib/etcd/snapshot-backup.db
sudo ls -lh /var/lib/etcd/snapshot-backup.db9. 实验六环境清理
bash
kubectl config use-context kubernetes-admin@kubernetes
kubectl config set-context --current --namespace=default
kubectl config delete-context alice-ctx 2>/dev/null || true
kubectl config delete-user dev-alice 2>/dev/null || true
kubectl delete crd redisclusters.database.example.com --ignore-not-found
kubectl delete namespace lab6-gov
sudo rm -f /var/lib/etcd/snapshot-backup.db
rm -rf /tmp/rbac-test /tmp/my-web-chart /tmp/webhook-lab