📊 监控：日志 / 指标 / 追踪 / 告警

可观测性三支柱

OpenTelemetry：统一标准

OpenTelemetry (OTel) 是 CNCF 的可观测性标准，被 90+ 厂商支持。它解决的核心问题是：不要被任何一家监控厂商锁定。

来源：opentelemetry.io（2026-05）。OTel 支持 Traces、Metrics、Logs 三种信号，覆盖 11+ 语言的 SDK。

Next.js 接入 OTel 示例

// next.config.js
module.exports = {
  experimental: {
    instrumentationHook: true,
  },
}

// instrumentation.ts (Next.js 14+)
export async function register() {
  if (process.env.NEXT_RUNTIME === 'nodejs') {
    const { NodeSDK } = await import('@opentelemetry/sdk-node')
    const { OTLPTraceExporter } = await import(
      '@opentelemetry/exporter-trace-otlp-http'
    )
    const { Resource } = await import('@opentelemetry/resources')
    const { ATTR_SERVICE_NAME } = await import(
      '@opentelemetry/semantic-conventions'
    )

    const sdk = new NodeSDK({
      resource: new Resource({
        [ATTR_SERVICE_NAME]: 'my-saas-web',
      }),
      traceExporter: new OTLPTraceExporter({
        url: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
      }),
    })
    sdk.start()
  }
}

OTel Collector 配置

# otel-collector-config.yaml
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318
      grpc:
        endpoint: 0.0.0.0:4317

processors:
  batch:
    timeout: 5s
    send_batch_size: 1024
  filter:
    error_mode: ignore
    traces:
      span:
        - 'attributes["http.route"] == "/health"'

exporters:
  # 导出到 Grafana Tempo (追踪)
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true
  # 导出到 Prometheus (指标)
  prometheus:
    endpoint: 0.0.0.0:8889
  # 导出到 Loki (日志)
  loki:
    endpoint: http://loki:3100/loki/api/v1/push

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch, filter]
      exporters: [otlp/tempo]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [prometheus]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [loki]

平台对比：Grafana / Datadog / Sentry / 自建

来源：grafana.com/pricing（2026-05）、datadoghq.com/pricing（2026-05）、sentry.io/pricing（2026-05）

日志：结构化日志与聚合

为什么必须结构化日志

2026-05-18 10:23:45 ERROR - Payment failed for user 123
2026-05-18 10:23:45 INFO - Retrying payment...
2026-05-18 10:23:46 ERROR - Payment failed again

{
  "ts": "2026-05-18T10:23:45.123Z",
  "level": "error",
  "msg": "Payment failed",
  "userId": "123",
  "paymentId": "pay_abc",
  "error": "Connection timeout",
  "duration_ms": 5000,
  "traceId": "abc123def456",
  "spanId": "789ghi"
}

结构化日志实现（Next.js + pino）

// lib/logger.ts
import pino from 'pino'

export const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
  formatters: {
    level: (label) => ({ level: label }),
  },
  // 生产环境用 JSON，开发环境用 pretty
  transport: process.env.NODE_ENV !== 'production'
    ? { target: 'pino-pretty' }
    : undefined,
})

// 使用
logger.info({ userId: '123', action: 'checkout' }, 'Checkout started')
logger.error({ err, userId: '123', paymentId: 'pay_abc' }, 'Payment failed')

// 关联 Trace ID（OTel 集成）
import { trace, context } from '@opentelemetry/api'

function getLoggerWithTrace() {
  const span = trace.getSpan(context.active())
  const spanContext = span?.spanContext()
  return logger.child({
    traceId: spanContext?.traceId,
    spanId: spanContext?.spanId,
  })
}

日志级别策略

指标：RED / USE 方法论

RED 方法（适用于请求驱动型服务/API）

USE 方法（适用于基础设施/资源）

Prometheus 指标类型

// 使用 prom-client (Node.js)
import client from 'prom-client'

// Counter：只增不减（请求数、错误数）
const httpRequestsTotal = new client.Counter({
  name: 'http_requests_total',
  help: 'Total HTTP requests',
  labelNames: ['method', 'route', 'status'],
})

// Histogram：分布（延迟、大小）
const httpRequestDuration = new client.Histogram({
  name: 'http_request_duration_seconds',
  help: 'HTTP request duration',
  labelNames: ['method', 'route'],
  buckets: [0.01, 0.05, 0.1, 0.5, 1, 2, 5, 10],
})

// Gauge：可增可减（当前连接数、队列长度）
const activeConnections = new client.Gauge({
  name: 'active_connections',
  help: 'Current active connections',
})

// 使用
httpRequestsTotal.inc({ method: 'GET', route: '/api/users', status: 200 })
httpRequestDuration.observe({ method: 'GET', route: '/api/users' }, 0.234)
activeConnections.inc()

分布式追踪

当请求穿过多个服务时，追踪告诉你"时间都花在哪了"。

手动埋点示例

import { trace } from '@opentelemetry/api'

const tracer = trace.getTracer('my-saas')

async function checkout(userId: string, cartId: string) {
  return tracer.startActiveSpan('checkout', async (span) => {
    span.setAttribute('userId', userId)
    span.setAttribute('cartId', cartId)

    try {
      const cart = await getCart(cartId)  // 自动传播 trace context
      const charge = await chargePayment(userId, cart.total)
      await createOrder(userId, cart, charge)
      span.setStatus({ code: 1 }) // OK
      return { success: true }
    } catch (err) {
      span.setStatus({ code: 2, message: err.message }) // ERROR
      span.recordException(err)
      throw err
    } finally {
      span.end()
    }
  })
}

告警策略

告警分级

好的告警 vs 坏的告警

Grafana 告警规则示例

# Prometheus/Grafana Alert Rule
# 基于 SLO 的错误率告警
- alert: HighErrorRate
  expr: |
    (
      sum(rate(http_requests_total{status=~"5.."}[5m]))
      /
      sum(rate(http_requests_total[5m]))
    ) > 0.05
  for: 3m
  labels:
    severity: warning
  annotations:
    summary: "High error rate on {{ $labels.service }}"
    description: |
      Error rate is {{ $value | humanizePercentage }}
      for service {{ $labels.service }}.
      SLO target: < 1% errors.
      Runbook: https://wiki/internal/runbooks/high-error-rate
    dashboard: "https://grafana.example.com/d/api-overview"

SaaS 业务监控

技术监控之外，SaaS 还需要关注业务指标——它们直接反映产品健康度。

技术 + 业务指标仪表板布局

实战：从零搭建监控

阶段 1：Day 1 基础（$0）

# compose.yaml — 添加监控栈
services:
  # ... 你的应用服务 ...

  # OTel Collector
  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    ports:
      - "4317:4317"  # gRPC
      - "4318:4318"  # HTTP
    volumes:
      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml

  # Prometheus
  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml

  # Grafana
  grafana:
    image: grafana/grafana:latest
    ports:
      - "3001:3000"
    environment:
      GF_SECURITY_ADMIN_PASSWORD: admin
    volumes:
      - grafanadata:/var/lib/grafana

  # Loki (日志)
  loki:
    image: grafana/loki:latest
    ports:
      - "3100:3100"

volumes:
  grafanadata:

阶段 2：成长期（$0-50/月）

阶段 3：规模化（$200+/月）

反模式

🔗 相关内容