17 - 自定义CRD | K8s实战课程

一、CRD——扩展K8s API

CustomResourceDefinition（CRD）让你定义自己的资源类型，就像K8s内置的Deployment/Service一样使用kubectl操作。CRD + 自定义控制器 = Operator模式。

┌────────── K8s扩展模式 ──────────┐
│                                   │
│  K8s内置资源                      │
│  Deployment / Service / Pod / ... │
│  → API Server自动处理             │
│                                   │
│  CRD自定义资源                    │
│  WebApp / Database / Certificate  │
│  → 定义数据结构（Schema）         │
│  → 需要控制器实现业务逻辑         │
│                                   │
│  ┌─── CRD定义 ──────────────┐     │
│  │  API组：myapp.example.com│     │
│  │  版本：v1alpha1          │     │
│  │  类型：WebApp            │     │
│  │  Schema：                │     │
│  │    spec.image            │     │
│  │    spec.replicas         │     │
│  │    spec.config           │     │
│  └───────────────────────────┘    │
│           ↓                       │
│  ┌─── 控制器 ────────────────┐    │
│  │  Watch WebApp CR         │    │
│  │  → 创建Deployment        │    │
│  │  → 创建Service           │    │
│  │  → 创建ConfigMap         │    │
│  │  → 调谐到期望状态        │    │
│  └───────────────────────────┘    │
└───────────────────────────────────┘

二、定义CRD

2.1 基础CRD

# crd-webapp.yaml apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: name: webapps.myapp.example.com # 格式：<plural>.<group> spec: group: myapp.example.com # API组 scope: Namespaced # Namespaced | Cluster names: kind: WebApp # 资源类型名 listKind: WebAppList # 列表类型名 singular: webapp # 单数名 plural: webapps # 复数名 shortNames: [wa] # kubectl get wa categories: [all] # kubectl get all包含 versions: - name: v1 served: true # 是否通过API提供 storage: true # 只能有一个storage版本 subresources: status: {} # 启用/status子资源 scale: # 启用/scale子资源 specReplicasPath: .spec.replicas statusReplicasPath: .status.replicas labelSelectorPath: .status.labelSelector additionalPrinterColumns: # kubectl get显示的列 - name: Image type: string jsonPath: .spec.image - name: Replicas type: integer jsonPath: .spec.replicas - name: Ready type: integer jsonPath: .status.readyReplicas - name: Age type: date jsonPath: .metadata.creationTimestamp schema: openAPIV3Schema: type: object properties: spec: type: object required: [image] # 必填字段 properties: image: type: string description: "Container image" pattern: "^[a-zA-Z0-9._/-]+:[a-zA-Z0-9._-]+$" replicas: type: integer minimum: 1 maximum: 100 default: 1 port: type: integer minimum: 1 maximum: 65535 default: 8080 config: type: object additionalProperties: type: string description: "Application configuration key-value pairs" resources: type: object properties: requests: type: object properties: cpu: {type: string} memory: {type: string} limits: type: object properties: cpu: {type: string} memory: {type: string} ingress: type: object properties: enabled: type: boolean default: false host: type: string tls: type: boolean default: false status: type: object properties: replicas: type: integer readyReplicas: type: integer labelSelector: type: string conditions: type: array items: type: object properties: type: {type: string} status: {type: string} reason: {type: string} message: {type: string} lastTransitionTime: {type: string, format: date-time} # ✅ 验证通过 kubectl apply -f crd-webapp.yaml kubectl get crd webapps.myapp.example.com # NAME ESTABLISHED AGE # webapps.myapp.example.com True 5s

2.2 创建CR实例

# webapp-instance.yaml apiVersion: myapp.example.com/v1 kind: WebApp metadata: name: my-webapp namespace: default spec: image: nginx:1.25 replicas: 3 port: 8080 config: LOG_LEVEL: "info" APP_ENV: "production" resources: requests: cpu: "100m" memory: "128Mi" limits: cpu: "500m" memory: "512Mi" ingress: enabled: true host: myapp.example.com tls: true # ✅ 验证通过 kubectl apply -f webapp-instance.yaml kubectl get webapps # NAME IMAGE REPLICAS READY AGE # my-webapp nginx:1.25 3 0 5s kubectl describe webapp my-webapp # Spec: # Image: nginx:1.25 # Replicas: 3

三、CRD版本管理

# 多版本CRD spec: versions: - name: v1alpha1 served: true storage: false # 不再是存储版本 deprecated: true # 标记为废弃 deprecationWarning: "myapp.example.com/v1alpha1 WebApp is deprecated" schema: ... # v1alpha1 schema - name: v1beta1 served: true storage: false schema: ... # v1beta1 schema - name: v1 served: true storage: true # 当前存储版本 schema: ... # v1 schema # 版本转换Webhook（v1alpha1/v1beta1 → v1） # 需要实现Conversion Webhook服务 spec: conversion: strategy: Webhook webhook: clientConfig: service: name: crd-converter namespace: system path: /convert conversionReviewVersions: ["v1", "v1beta1"]

四、使用kubebuilder生成CRD

# 安装kubebuilder curl -L -o kubebuilder https://go.kubebuilder.io/dl/latest/linux/amd64 chmod +x kubebuilder mv kubebuilder /usr/local/bin/ # 初始化项目 mkdir my-operator && cd my-operator kubebuilder init --domain example.com --repo example.com/my-operator # 创建API kubebuilder create api --group myapp --version v1 --kind WebApp # 编辑类型定义：api/v1/webapp_types.go # +kubebuilder:object:root=true # +kubebuilder:subresource:status # +kubebuilder:subresource:scale:specReplicasPath=.spec.replicas,statusReplicasPath=.status.replicas # +kubebuilder:printcolumn:name="Image",type=string,JSONPath=`.spec.image` type WebApp struct { metav1.TypeMeta `json:",inline"` metav1.ObjectMeta `json:"metadata,omitempty"` Spec WebAppSpec `json:"spec,omitempty"` Status WebAppStatus `json:"status,omitempty"` } type WebAppSpec struct { // +kubebuilder:validation:Required // +kubebuilder:validation:Pattern=`^[a-zA-Z0-9._/-]+:[a-zA-Z0-9._-]+$` Image string `json:"image"` // +kubebuilder:default=1 // +kubebuilder:validation:Minimum=1 // +kubebuilder:validation:Maximum=100 Replicas *int32 `json:"replicas,omitempty"` // +kubebuilder:default=8080 Port int32 `json:"port,omitempty"` } type WebAppStatus struct { Replicas int32 `json:"replicas,omitempty"` ReadyReplicas int32 `json:"readyReplicas,omitempty"` LabelSelector string `json:"labelSelector,omitempty"` } # 生成CRD make manifests # 生成 config/crd/bases/myapp.example.com_webapps.yaml # 安装CRD到集群 make install # 运行控制器 make run

五、Finalizer——资源清理

# Finalizer确保CR删除前执行清理逻辑 # 在控制器中检查deletionTimestamp func (r *WebAppReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) { var webapp myappv1.WebApp if err := r.Get(ctx, req.NamespacedName, &webapp); err != nil { return ctrl.Result{}, client.IgnoreNotFound(err) } // 检查是否正在删除 if webapp.DeletionTimestamp != nil { if controllerutil.ContainsFinalizer(&webapp, "myapp.example.com/finalizer") { // 执行清理逻辑 if err := r.cleanupExternalResources(ctx, &webapp); err != nil { return ctrl.Result{}, err } // 移除Finalizer controllerutil.RemoveFinalizer(&webapp, "myapp.example.com/finalizer") if err := r.Update(ctx, &webapp); err != nil { return ctrl.Result{}, err } } return ctrl.Result{}, nil } // 添加Finalizer if !controllerutil.ContainsFinalizer(&webapp, "myapp.example.com/finalizer") { controllerutil.AddFinalizer(&webapp, "myapp.example.com/finalizer") if err := r.Update(ctx, &webapp); err != nil { return ctrl.Result{}, err } } // 正常调谐逻辑... return ctrl.Result{}, nil }

六、练习

🏆 第17课成就解锁

✅ 理解CRD扩展K8s API的原理与价值
✅ 掌握CRD定义：Schema/Version/Subresource
✅ 掌握kubebuilder项目创建与代码生成
✅ 理解Finalizer与资源清理机制
✅ 了解CRD版本管理与转换Webhook
✅ 能定义生产级CRD并创建自定义资源

下一课预告：第18课深入Operator开发——实现自定义控制器。

📌 补充知识

17-自定义crd补充要点：K8s生产实践扩展

🔹 资源配额(ResourceQuota)：限制命名空间总资源
  apiVersion: v1
  kind: ResourceQuota
  metadata:
    name: compute-quota
    namespace: production
  spec:
    hard:
      requests.cpu: "20"
      requests.memory: 40Gi
      limits.cpu: "40"
      limits.memory: 80Gi
      pods: "50"
      services: "10"

🔹 LimitRange：设置默认资源限制
  apiVersion: v1
  kind: LimitRange
  metadata:
    name: default-limits
  spec:
    limits:
    - type: Container
      default:
        cpu: "200m"
        memory: 256Mi
      defaultRequest:
        cpu: "100m"
        memory: 128Mi
      max:
        cpu: "2"
        memory: 4Gi

🔹 Pod优先级与抢占
  apiVersion: scheduling.k8s.io/v1
  kind: PriorityClass
  metadata:
    name: high-priority
  value: 1000000
  globalDefault: false
  ---
  spec:
    preemptionPolicy: PreemptLowerPriority

🔹 优雅处理Pod中断
  • PDB保证最小可用副本
  • preStop钩子处理连接排空
  • terminationGracePeriodSeconds充足
  • 应用必须处理SIGTERM信号

🔹 生产环境Checklist
  ✅ 设置resources requests/limits
  ✅ 配置liveness/readiness探针
  ✅ 使用PDB保护关键服务
  ✅ 实现优雅关闭（SIGTERM）
  ✅ 配置HPA自动伸缩
  ✅ 使用NetworkPolicy隔离
  ✅ 开启RBAC最小权限
  ✅ 日志结构化输出
  ✅ 指标暴露/metrics端点
  ✅ 配置PVC数据备份