RocketMQ 云原生部署与 Kubernetes 实战

RocketMQ 云原生部署是现代化部署的趋势。本文将深入探讨 Docker、Kubernetes、Helm、Operator 等云原生部署方案。

一、Docker 部署

1.1 官方镜像

# 拉取官方镜像
docker pull apache/rocketmq:5.0.0

# 查看镜像
docker images | grep rocketmq

1.2 Docker Compose 部署

# docker-compose.yml
version: '3'
services:
  nameserver:
    image: apache/rocketmq:5.0.0
    command: sh mqnamesrv
    ports:
      - "9876:9876"
    environment:
      - JAVA_OPT_EXT=-server -Xms1g -Xmx1g
    volumes:
      - nameserver-data:/home/rocketmq/logs
      - nameserver-store:/home/rocketmq/store
  
  broker:
    image: apache/rocketmq:5.0.0
    command: sh mqbroker -n nameserver:9876 -c /home/rocketmq/conf/broker.conf
    ports:
      - "10911:10911"
      - "10909:10909"
    environment:
      - JAVA_OPT_EXT=-server -Xms2g -Xmx2g
      - NAMESRV_ADDR=nameserver:9876
    volumes:
      - broker-data:/home/rocketmq/logs
      - broker-store:/home/rocketmq/store
      - ./broker.conf:/home/rocketmq/conf/broker.conf
    depends_on:
      - nameserver
  
  console:
    image: apacherocketmq/rocketmq-console:latest
    ports:
      - "8080:8080"
    environment:
      - JAVA_OPTS=-Drocketmq.namesrv.addr=nameserver:9876
    depends_on:
      - nameserver

volumes:
  nameserver-data:
  nameserver-store:
  broker-data:
  broker-store:

1.3 自定义镜像

# Dockerfile
FROM apache/rocketmq:5.0.0

# 复制配置文件
COPY broker.conf /home/rocketmq/conf/broker.conf
COPY plain_acl.yml /home/rocketmq/conf/plain_acl.yml

# 设置环境变量
ENV JAVA_OPT_EXT="-server -Xms4g -Xmx4g -XX:+UseG1GC"

# 暴露端口
EXPOSE 9876 10911 10909

# 启动脚本
COPY docker-entrypoint.sh /docker-entrypoint.sh
RUN chmod +x /docker-entrypoint.sh

ENTRYPOINT ["/docker-entrypoint.sh"]

二、Kubernetes 部署

2.1 NameServer StatefulSet

# nameserver-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rocketmq-nameserver
  labels:
    app: rocketmq-nameserver
spec:
  serviceName: rocketmq-nameserver
  replicas: 2
  selector:
    matchLabels:
      app: rocketmq-nameserver
  template:
    metadata:
      labels:
        app: rocketmq-nameserver
    spec:
      containers:
      - name: nameserver
        image: apache/rocketmq:5.0.0
        command: ["sh", "-c", "mqnamesrv"]
        ports:
        - containerPort: 9876
        env:
        - name: JAVA_OPTS
          value: "-server -Xms2g -Xmx2g -XX:+UseG1GC"
        readinessProbe:
          tcpSocket:
            port: 9876
          initialDelaySeconds: 30
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 9876
          initialDelaySeconds: 60
          periodSeconds: 30
        volumeMounts:
        - name: data
          mountPath: /home/rocketmq/logs
        - name: store
          mountPath: /home/rocketmq/store
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 10Gi
  - metadata:
      name: store
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 50Gi
---
apiVersion: v1
kind: Service
metadata:
  name: rocketmq-nameserver
spec:
  clusterIP: None
  selector:
    app: rocketmq-nameserver
  ports:
  - port: 9876
    targetPort: 9876

2.2 Broker StatefulSet

# broker-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: rocketmq-broker
  labels:
    app: rocketmq-broker
spec:
  serviceName: rocketmq-broker
  replicas: 2
  selector:
    matchLabels:
      app: rocketmq-broker
  template:
    metadata:
      labels:
        app: rocketmq-broker
    spec:
      initContainers:
      - name: wait-for-nameserver
        image: busybox
        command: ['sh', '-c', 'until nc -z rocketmq-nameserver-0.rocketmq-nameserver 9876; do echo waiting for nameserver; sleep 2; done']
      containers:
      - name: broker
        image: apache/rocketmq:5.0.0
        command: ["sh", "-c", "mqbroker -c /home/rocketmq/conf/broker.conf"]
        ports:
        - containerPort: 10911
        - containerPort: 10909
        env:
        - name: JAVA_OPTS
          value: "-server -Xms4g -Xmx4g -XX:+UseG1GC"
        - name: NAMESRV_ADDR
          value: "rocketmq-nameserver-0.rocketmq-nameserver:9876;rocketmq-nameserver-1.rocketmq-nameserver:9876"
        readinessProbe:
          tcpSocket:
            port: 10911
          initialDelaySeconds: 30
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 10911
          initialDelaySeconds: 60
          periodSeconds: 30
        volumeMounts:
        - name: config
          mountPath: /home/rocketmq/conf
        - name: data
          mountPath: /home/rocketmq/logs
        - name: store
          mountPath: /home/rocketmq/store
      volumes:
      - name: config
        configMap:
          name: rocketmq-broker-config
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 20Gi
  - metadata:
      name: store
    spec:
      accessModes: ["ReadWriteOnce"]
      resources:
        requests:
          storage: 100Gi

2.3 Broker ConfigMap

# broker-config.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: rocketmq-broker-config
data:
  broker.conf: |
    brokerClusterName=DefaultCluster
    brokerName=broker-a
    brokerId=0
    namesrvAddr=rocketmq-nameserver-0.rocketmq-nameserver:9876;rocketmq-nameserver-1.rocketmq-nameserver:9876
    listenPort=10911
    storePathRootDir=/home/rocketmq/store
    storePathCommitLog=/home/rocketmq/store/commitlog
    autoCreateTopicEnable=true
    autoCreateSubscriptionGroup=true

三、Helm Chart 部署

3.1 Chart 结构

rocketmq/
├── Chart.yaml
├── values.yaml
├── templates/
│   ├── _helpers.tpl
│   ├── nameserver-statefulset.yaml
│   ├── nameserver-service.yaml
│   ├── broker-statefulset.yaml
│   ├── broker-service.yaml
│   ├── broker-configmap.yaml
│   └── console-deployment.yaml
└── charts/

3.2 Chart.yaml

apiVersion: v2
name: rocketmq
description: A Helm chart for RocketMQ
type: application
version: 1.0.0
appVersion: "5.0.0"
keywords:
  - rocketmq
  - messaging
  - queue
home: https://rocketmq.apache.org/
sources:
  - https://github.com/apache/rocketmq

3.3 values.yaml

# values.yaml
replicaCount:
  nameserver: 2
  broker: 2

image:
  repository: apache/rocketmq
  tag: "5.0.0"
  pullPolicy: IfNotPresent

resources:
  nameserver:
    requests:
      cpu: 1
      memory: 2Gi
    limits:
      cpu: 2
      memory: 4Gi
  broker:
    requests:
      cpu: 2
      memory: 4Gi
    limits:
      cpu: 4
      memory: 8Gi

storage:
  nameserver:
    data: 10Gi
    store: 50Gi
  broker:
    data: 20Gi
    store: 100Gi

console:
  enabled: true
  image: apacherocketmq/rocketmq-console:latest
  port: 8080

3.4 安装命令

# 添加 Helm 仓库
helm repo add rocketmq https://apache.github.io/rocketmq-helm/
helm repo update

# 安装 RocketMQ
helm install my-rocketmq rocketmq/rocketmq \
  --namespace rocketmq \
  --create-namespace \
  -f values.yaml

# 查看状态
helm status my-rocketmq -n rocketmq

# 升级
helm upgrade my-rocketmq rocketmq/rocketmq \
  --namespace rocketmq \
  -f values.yaml

# 卸载
helm uninstall my-rocketmq -n rocketmq

四、Operator 部署

4.1 RocketMQ Operator

# rocketmq-operator.yaml
apiVersion: rocketmq.apache.org/v1alpha1
kind: Broker
metadata:
  name: rocketmq-broker
spec:
  size: 2
  nameServiceNumber: 2
  brokerImage: apache/rocketmq:5.0.0
  storageSize: 100Gi
  resources:
    requests:
      cpu: 2
      memory: 4Gi
    limits:
      cpu: 4
      memory: 8Gi

4.2 安装 Operator

# 安装 Operator
kubectl apply -f https://raw.githubusercontent.com/apache/rocketmq-operator/master/deploy/operator.yaml

# 创建 RocketMQ 集群
kubectl apply -f rocketmq-cluster.yaml

# 查看状态
kubectl get broker -n rocketmq
kubectl get nameservice -n rocketmq

五、监控告警

5.1 Prometheus Operator

# servicemonitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: rocketmq-monitor
  labels:
    app: rocketmq
spec:
  selector:
    matchLabels:
      app: rocketmq-broker
  endpoints:
  - port: metrics
    interval: 15s
    path: /metrics
  namespaceSelector:
    matchNames:
    - rocketmq

5.2 Grafana Dashboard

# grafana-dashboard.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana-dashboard-rocketmq
  labels:
    grafana_dashboard: "1"
data:
  rocketmq.json: |
    {
      "dashboard": {
        "title": "RocketMQ 监控",
        "panels": [...]
      }
    }

六、最佳实践

6.1 资源配置

资源配置建议：
- NameServer：2 节点，2 核 4GB
- Broker：根据业务规模配置
- 存储：SSD，100GB+
- 网络：千兆以上

6.2 高可用配置

高可用配置：
1. NameServer 多副本
2. Broker 主从部署
3. 跨可用区部署
4. 持久化存储
5. 自动故障切换

6.3 运维检查清单

运维检查：
- [ ] Pod 状态正常
- [ ] 存储卷正常
- [ ] 网络连通正常
- [ ] 监控告警正常
- [ ] 日志收集正常
- [ ] 备份恢复正常

总结

RocketMQ 云原生部署的核心要点：

1. Docker 部署：官方镜像、Docker Compose、自定义镜像
2. Kubernetes 部署：StatefulSet、ConfigMap、Service
3. Helm Chart：Chart 结构、values.yaml、安装命令
4. Operator：CRD、自动化运维
5. 监控告警：Prometheus Operator、Grafana Dashboard

核心要点：

• 理解云原生部署优势
• 掌握 Kubernetes 部署方法
• 使用 Helm 简化部署
• 建立完善的监控体系
• 实现自动化运维

参考资料

• RocketMQ 官方文档
• RocketMQ Docker
• RocketMQ Operator
• 《Kubernetes 权威指南》