Skip to content
第 74 / 250 章后端⏱ 10 分钟阅读

第 74 章:健康检查与 K8s 探针

学习目标

  • 理解三种探针:Liveness / Readiness / Startup
  • 自定义健康指示器
  • 实战 K8s 部署时的探针配置

一、为什么需要探针?

三大场景

  • 应用启动慢(缓存预热、连接池初始化)→ Startup Probe 防止被误杀
  • 应用暂时不可用(数据库连不上)→ Liveness Probe 重启
  • 应用暂时过载(线程池满)→ Readiness Probe 摘流量

二、Spring Boot 探针端点

yaml
management:
  endpoint:
    health:
      probes:
        enabled: true                          # ① 启用探针
      group:
        liveness:
          include: livenessState                # ② /actuator/health/liveness
        readiness:
          include: readinessState,db,redis      # ③ /actuator/health/readiness
        startup:
          include: startupState                 # ④ /actuator/health/startup
      show-details: when-authorized

启动后访问:

  • GET /actuator/health/liveness ← K8s liveness probe
  • GET /actuator/health/readiness ← K8s readiness probe
  • GET /actuator/health ← 综合健康状态

三、状态机

关键点:Readiness Probe 失败时,K8s 把 Pod 从 Service 端点列表中摘除(停止转发流量),但不重启容器。Liveness Probe 失败时,K8s 重启容器。

四、K8s 配置实战

yaml
# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: taskflow
spec:
  replicas: 3
  template:
    spec:
      containers:
        - name: app
          image: taskflow:1.0.0
          ports:
            - containerPort: 8080

          # ① Startup Probe:启动慢时给宽限时间
          startupProbe:
            httpGet:
              path: /actuator/health/startup
              port: 8080
            initialDelaySeconds: 0                # 立即开始
            periodSeconds: 10                     # 每 10 秒检查
            failureThreshold: 30                  # 30 次失败才算启动失败(5 分钟宽限期)
            successThreshold: 1

          # ② Liveness Probe:失败则重启
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: 8080
            initialDelaySeconds: 60               # 启动 60 秒后才检查
            periodSeconds: 10
            timeoutSeconds: 3
            failureThreshold: 3                   # 连续 3 次失败才重启
            successThreshold: 1

          # ③ Readiness Probe:失败则摘流量
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: 8080
            initialDelaySeconds: 30
            periodSeconds: 5
            timeoutSeconds: 3
            failureThreshold: 3                   # 连续 3 次失败才摘流量
            successThreshold: 1

          resources:
            requests:
              memory: "512Mi"
              cpu: "500m"
            limits:
              memory: "1Gi"
              cpu: "1000m"

五、自定义健康检查

健康指示器

java
@Component
@RequiredArgsConstructor
public class DatabaseHealthIndicator implements HealthIndicator {

    private final DataSource dataSource;

    @Override
    public Health health() {
        try (Connection conn = dataSource.getConnection()) {
            if (conn.isValid(2)) {                  // 2 秒超时
                return Health.up()
                        .withDetail("database", "MySQL")
                        .withDetail("validationQuery", "isValid()")
                        .build();
            }
            return Health.down()
                    .withDetail("error", "Connection invalid")
                    .build();
        } catch (SQLException e) {
            return Health.down(e)
                    .withDetail("error", e.getMessage())
                    .build();
        }
    }
}

@Component
public class RedisHealthIndicator implements HealthIndicator {

    private final RedisConnectionFactory factory;

    @Override
    public Health health() {
        try (RedisConnection conn = factory.getConnection()) {
            String pong = conn.ping();
            if ("PONG".equalsIgnoreCase(pong)) {
                return Health.up().build();
            }
            return Health.down().withDetail("ping", pong).build();
        } catch (Exception e) {
            return Health.down(e).build();
        }
    }
}

带参数的健康检查

java
// 通过 application.yml 启用
management.endpoint.health.db.show-details: always

// 自定义 Health 名称
@Component
public class CustomHealthIndicator implements HealthIndicator {
    // Bean 名去掉 "HealthIndicator" 后缀就是健康检查 key
    // 如 "CustomHealthIndicator" → key="custom"
}

@Component
public class PaymentHealthIndicator implements HealthIndicator {
    // key="payment"
}
json
// /actuator/health 响应
{
  "status": "UP",
  "components": {
    "db": { "status": "UP", "details": { "database": "MySQL" } },
    "redis": { "status": "UP" },
    "payment": { "status": "DOWN", "details": { "error": "..." } },
    "diskSpace": { "status": "UP" }
  }
}

六、Liveness vs Readiness 怎么设计?

检查项LivenessReadiness
JVM 是否存活(线程死锁、OOM)
应用进程是否响应
数据库连接⚠️ 慎用
外部依赖(Redis、MQ)
磁盘空间

关键原则

  • Liveness 只检查进程本身(不要检查外部依赖!)
    • 数据库挂了,重启应用没用 → 不应重启
    • 外部依赖挂了就重启 → 雪崩(重启完还是连不上)
  • Readiness 检查所有依赖(数据库、缓存、下游服务)
    • 依赖不可用时摘流量,恢复了再加回来

实战配置

java
// Liveness 只检查 JVM
@Component
public class LivenessHealthIndicator implements HealthIndicator {

    @Override
    public Health health() {
        // ① 检查线程是否都活着
        ThreadMXBean threadBean = ManagementFactory.getThreadMXBean();
        long deadlocked = threadBean.findDeadlockedThreads()?.length ?: 0;
        if (deadlocked > 0) {
            return Health.down().withDetail("deadlockedThreads", deadlocked).build();
        }

        // ② 检查堆内存是否足够
        MemoryMXBean memBean = ManagementFactory.getMemoryMXBean();
        long used = memBean.getHeapMemoryUsage().getUsed();
        long max = memBean.getHeapMemoryUsage().getMax();
        if (max > 0 && (double) used / max > 0.95) {
            return Health.down()
                    .withDetail("heapUsage", String.format("%.2f%%", 100.0 * used / max))
                    .build();
        }

        return Health.up().build();
    }
}

// Readiness 检查所有依赖
@Component
public class ReadinessHealthIndicator implements HealthIndicator {

    private final DataSource dataSource;
    private final RedisConnectionFactory redisFactory;

    @Override
    public Health health() {
        Health.Builder builder = Health.up();

        // ① 数据库
        try (Connection conn = dataSource.getConnection()) {
            if (!conn.isValid(2)) {
                return Health.down().withDetail("db", "invalid").build();
            }
        } catch (Exception e) {
            return Health.down(e).withDetail("db", "down").build();
        }

        // ② Redis
        try (RedisConnection conn = redisFactory.getConnection()) {
            if (!"PONG".equalsIgnoreCase(conn.ping())) {
                return Health.down().withDetail("redis", "no ping").build();
            }
        } catch (Exception e) {
            return Health.down(e).withDetail("redis", "down").build();
        }

        return builder.build();
    }
}

七、Startup Probe 实战

场景:Spring Boot 应用启动需要 60 秒(连接池初始化、缓存预热、二级索引构建)。普通的 Liveness 探针 initialDelaySeconds: 60 太粗。

yaml
# Startup Probe:给充足时间启动
startupProbe:
  httpGet:
    path: /actuator/health/startup
    port: 8080
  periodSeconds: 5
  failureThreshold: 60                  # 60 * 5 = 300 秒(5 分钟)内必须启动完成
  successThreshold: 1

# Liveness Probe:启动后开始检查
livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  periodSeconds: 10
  failureThreshold: 3
  # 注意:没有 initialDelaySeconds,因为 Startup 成功后才进入 Liveness

自定义 Startup 健康检查

java
// 启动期间的状态由 startupState 自动管理
// 启动完成(ApplicationReadyEvent 触发)后变 UP

@Component
@RequiredArgsConstructor
public class CacheWarmerHealthIndicator implements HealthIndicator {

    private volatile boolean cacheWarmed = false;

    @EventListener(ApplicationReadyEvent.class)
    public void onReady() {
        log.info("应用启动完成,开始预热缓存");
        try {
            // 模拟加载热点数据
            cacheService.warmUp();
            cacheWarmed = true;
            log.info("缓存预热完成");
        } catch (Exception e) {
            log.error("缓存预热失败", e);
        }
    }

    @Override
    public Health health() {
        if (cacheWarmed) {
            return Health.up().build();
        }
        return Health.down().withDetail("reason", "缓存预热未完成").build();
    }
}

八、优雅停机

yaml
server:
  shutdown: graceful                      # ① 启用优雅停机

spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s       # ② 等待 30 秒完成收尾
yaml
# K8s 配置:先发 SIGTERM,等服务优雅关闭后再发 SIGKILL
terminationGracePeriodSeconds: 60        # K8s 等待 60 秒后强杀
java
@Component
public class GracefulShutdown implements DisposableBean {

    @PreDestroy
    public void onShutdown() {
        log.info("开始关闭...");
        // ① 停止接收新请求(Readiness 探针失败 → K8s 摘流量)
        // ② 等待正在处理的请求完成(最多 30 秒)
        // ③ 关闭连接池
        // ④ 退出
    }
}

九、Nginx 健康检查

nginx
upstream taskflow {
    server 192.168.1.10:8080 max_fails=3 fail_timeout=30s;
    server 192.168.1.11:8080 max_fails=3 fail_timeout=30s;

    # 主动健康检查(需要 nginx_upstream_check_module)
    check interval=3000 rise=2 fall=3 timeout=1000 type=http;
    check_http_send "GET /actuator/health HTTP/1.0\r\n\r\n";
    check_http_expect_alive http_2xx http_3xx;
}

十、健康检查最佳实践

yaml
# application.yml 综合配置
management:
  endpoint:
    health:
      probes:
        enabled: true
      show-details: when-authorized
      group:
        liveness:
          include: livenessState              # 仅进程级检查
        readiness:
          include: readinessState,db,redis,diskSpace
  endpoints:
    web:
      exposure:
        include: health,info,prometheus
  health:
    livenessstate:
      enabled: true
    readinessstate:
      enabled: true
    diskspace:
      enabled: true
      threshold: 1GB                        # 磁盘剩余 < 1GB 视为 DOWN
    db:
      enabled: true
      show-details: when-authorized
实践说明
Liveness 只检查进程不要检查外部依赖,否则雪崩
Readiness 检查依赖DB/Redis 挂了摘流量
Startup 给足时间慢启动场景必备
优雅停机 30 秒配合 K8s terminationGracePeriodSeconds
超时分清楚探针 timeoutSeconds < periodSeconds
失败阈值 ≥ 3避免抖动误判

十一、本章小结

要点关键
Liveness进程是否活着,只检查 JVM
Readiness是否能接收流量,检查所有依赖
Startup应用启动是否完成,给慢启动应用宽限期
自定义指示器实现 HealthIndicator 接口
K8s 配置initialDelaySeconds / periodSeconds / failureThreshold
优雅停机server.shutdown: graceful + K8s terminationGracePeriodSeconds

动手练习

练习 1:基础题

给你的应用配置 K8s 三种探针,部署到 K8s 集群并验证:

  • 启动慢时 Startup Probe 给宽限
  • 杀进程时 Liveness Probe 重启
  • 数据库挂时 Readiness Probe 摘流量

练习 2:进阶题

实现一个支付服务的健康检查:

  • Liveness:检查进程和支付 SDK 是否正常
  • Readiness:检查下游支付通道、银行接口
  • 自定义 Health Indicator:在支付通道不可用时返回 DOWN

下一章第 75 章:单元测试与 Mockito

本站基于 VitePress 构建 · 由 Codebook 团队维护