K8s 部署实战
Kubernetes 简介
核心概念
Pod：
- K8s 最小调度单元
- 一个或多个容器
- 共享网络和存储
Deployment：
- 管理 Pod 副本
- 支持滚动更新
- 支持回滚
Service：
- 服务抽象
- 负载均衡
- 服务发现
ConfigMap：
- 配置文件
- 环境变量
- 与应用解耦
Secret：
- 敏感信息
- 加密存储
- 安全分发
架构设计
┌─────────────────────────────────────────────┐
│ Control Plane │
│ ┌─────────┐ ┌─────────┐ ┌─────────┐ │
│ │ API │ │Scheduler│ │Controller │ │
│ │ Server │ │ │ │ Manager │ │
│ └─────────┘ └─────────┘ └─────────┘ │
│ │
│ ┌─────────────────────────────────────┐ │
│ │ etcd (数据存储) │ │
│ └─────────────────────────────────────┘ │
└─────────────────────────────────────────────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
┌───────────┐ ┌───────────┐ ┌───────────┐
│ Node 1 │ │ Node 2 │ │ Node 3 │
│ ┌───────┐ │ │ ┌───────┐ │ │ ┌───────┐ │
│ │ Kubelet│ │ │ │Kubelet│ │ │ │Kubelet│ │
│ └───────┘ │ │ └───────┘ │ │ └───────┘ │
│ ┌───────┐ │ │ ┌───────┐ │ │ ┌───────┐ │
│ │ Pod 1 │ │ │ │ Pod 2 │ │ │ │ Pod 3 │ │
│ └───────┘ │ │ └───────┘ │ │ └───────┘ │
└───────────┘ └───────────┘ └───────────┘快速开始
1. 安装 K8s
Minikube（本地开发）：
# 安装 Minikube
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
# 启动集群
minikube start
# 查看状态
minikube status
# 停止集群
minikube stopKind（本地开发）：
# 安装 Kind
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind
# 创建集群
kind create cluster --name my-cluster
# 删除集群
kind delete cluster --name my-cluster生产环境：
- kubeadm
- kops
- EKS（AWS）
- GKE（GCP）
- AKS（Azure）
2. kubectl 基础
# 查看节点
kubectl get nodes
# 查看 Pod
kubectl get pods
# 查看 Deployment
kubectl get deployments
# 查看 Service
kubectl get services
# 查看日志
kubectl logs <pod-name>
# 进入容器
kubectl exec -it <pod-name> -- bash
# 端口转发
kubectl port-forward <pod-name> 8080:8080
# 应用配置
kubectl apply -f deployment.yaml
# 删除资源
kubectl delete -f deployment.yaml微服务部署
1. Deployment 配置
用户服务：
apiVersion: apps/v1
kind: Deployment
metadata:
name: user-service
labels:
app: user-service
spec:
replicas: 3
selector:
matchLabels:
app: user-service
template:
metadata:
labels:
app: user-service
spec:
containers:
- name: user-service
image: registry.example.com/user-service:1.0.0
ports:
- containerPort: 8080
env:
- name: SPRING_PROFILES_ACTIVE
value: "prod"
- name: NACOS_SERVER_ADDR
value: "nacos:8848"
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "1000m"
memory: "1Gi"
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 30
periodSeconds: 5
volumeMounts:
- name: logs
mountPath: /app/logs
volumes:
- name: logs
emptyDir: {}2. Service 配置
ClusterIP（内部访问）：
apiVersion: v1
kind: Service
metadata:
name: user-service
spec:
selector:
app: user-service
ports:
- protocol: TCP
port: 80
targetPort: 8080
type: ClusterIPLoadBalancer（外部访问）：
apiVersion: v1
kind: Service
metadata:
name: gateway
spec:
selector:
app: gateway
ports:
- protocol: TCP
port: 80
targetPort: 8080
type: LoadBalancerNodePort（节点访问）：
apiVersion: v1
kind: Service
metadata:
name: gateway
spec:
selector:
app: gateway
ports:
- protocol: TCP
port: 80
targetPort: 8080
nodePort: 30080
type: NodePort3. ConfigMap 配置
apiVersion: v1
kind: ConfigMap
metadata:
name: user-service-config
data:
application.yml: |
spring:
datasource:
url: jdbc:mysql://mysql:3306/user_db
username: ${DB_USERNAME}
password: ${DB_PASSWORD}
redis:
host: redis
port: 6379
cloud:
nacos:
discovery:
server-addr: nacos:8848
sentinel:
transport:
dashboard: sentinel:8080
logging:
level:
root: INFO
com.example: DEBUG使用 ConfigMap：
apiVersion: apps/v1
kind: Deployment
metadata:
name: user-service
spec:
template:
spec:
containers:
- name: user-service
image: user-service:1.0.0
envFrom:
- configMapRef:
name: user-service-config
volumeMounts:
- name: config
mountPath: /app/config
volumes:
- name: config
configMap:
name: user-service-config4. Secret 配置
apiVersion: v1
kind: Secret
metadata:
name: db-secret
type: Opaque
stringData:
DB_USERNAME: app_user
DB_PASSWORD: S3cr3tP@ssw0rd使用 Secret：
apiVersion: apps/v1
kind: Deployment
metadata:
name: user-service
spec:
template:
spec:
containers:
- name: user-service
image: user-service:1.0.0
envFrom:
- secretRef:
name: db-secret服务网格
1. Nacos 部署
apiVersion: apps/v1
kind: Deployment
metadata:
name: nacos
spec:
replicas: 1
selector:
matchLabels:
app: nacos
template:
metadata:
labels:
app: nacos
spec:
containers:
- name: nacos
image: nacos/nacos-server:2.2.0
ports:
- containerPort: 8848
- containerPort: 9848
env:
- name: MODE
value: "standalone"
- name: SPRING_DATASOURCE_PLATFORM
value: "mysql"
- name: MYSQL_SERVICE_HOST
value: "mysql"
- name: MYSQL_SERVICE_DB_NAME
value: "nacos_config"
---
apiVersion: v1
kind: Service
metadata:
name: nacos
spec:
selector:
app: nacos
ports:
- port: 8848
targetPort: 8848
name: http
- port: 9848
targetPort: 9848
name: grpc
type: ClusterIP2. Sentinel 部署
apiVersion: apps/v1
kind: Deployment
metadata:
name: sentinel
spec:
replicas: 1
selector:
matchLabels:
app: sentinel
template:
metadata:
labels:
app: sentinel
spec:
containers:
- name: sentinel
image: bladex/sentinel-dashboard:1.8.6
ports:
- containerPort: 8080
env:
- name: SENTINEL_USERNAME
value: "sentinel"
- name: SENTINEL_PASSWORD
value: "sentinel"
---
apiVersion: v1
kind: Service
metadata:
name: sentinel
spec:
selector:
app: sentinel
ports:
- port: 8080
targetPort: 8080
type: ClusterIP自动扩缩容
1. HPA（水平 Pod 扩缩容）
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: user-service-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: user-service
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 802. VPA（垂直 Pod 扩缩容）
apiVersion: autoscaling/v1
kind: VerticalPodAutoscaler
metadata:
name: user-service-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: user-service
updatePolicy:
updateMode: "Auto"3. Cluster Autoscaler
apiVersion: v1
kind: ServiceAccount
metadata:
name: cluster-autoscaler
namespace: kube-system
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: cluster-autoscaler
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: cluster-autoscaler
template:
metadata:
labels:
app: cluster-autoscaler
spec:
serviceAccountName: cluster-autoscaler
containers:
- name: cluster-autoscaler
image: registry.k8s.io/autoscaling/cluster-autoscaler:v1.25.0
command:
- ./cluster-autoscaler
- --cloud-provider=aws
- --namespace=kube-system
- --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
- --balance-similar-node-groups
- --skip-nodes-with-system-pods=false监控日志
1. Prometheus 部署
apiVersion: v1
kind: Namespace
metadata:
name: monitoring
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: user-service
namespace: monitoring
labels:
app: user-service
spec:
selector:
matchLabels:
app: user-service
endpoints:
- port: http
path: /actuator/prometheus
interval: 15s
namespaceSelector:
matchNames:
- default2. Grafana 配置
apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-datasources
namespace: monitoring
data:
datasources.yaml: |
apiVersion: 1
datasources:
- name: Prometheus
type: prometheus
url: http://prometheus:9090
access: proxy
isDefault: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: grafana
namespace: monitoring
spec:
replicas: 1
selector:
matchLabels:
app: grafana
template:
metadata:
labels:
app: grafana
spec:
containers:
- name: grafana
image: grafana/grafana:10.0.0
ports:
- containerPort: 3000
env:
- name: GF_SECURITY_ADMIN_PASSWORD
value: "admin"
volumeMounts:
- name: datasources
mountPath: /etc/grafana/provisioning/datasources
volumes:
- name: datasources
configMap:
name: grafana-datasources3. ELK 日志收集
apiVersion: v1
kind: DaemonSet
metadata:
name: filebeat
namespace: kube-system
spec:
selector:
matchLabels:
app: filebeat
template:
metadata:
labels:
app: filebeat
spec:
serviceAccountName: filebeat
containers:
- name: filebeat
image: elastic/filebeat:8.8.0
args:
- "-c"
- "/etc/filebeat.yml"
- "-e"
env:
- name: ELASTICSEARCH_HOST
value: elasticsearch
- name: ELASTICSEARCH_PORT
value: "9200"
volumeMounts:
- name: config
mountPath: /etc/filebeat.yml
readOnly: true
subPath: filebeat.yml
- name: varlog
mountPath: /var/log
readOnly: true
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
volumes:
- name: config
configMap:
name: filebeat-config
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containersCI/CD 集成
1. GitOps 工作流
# .github/workflows/deploy.yml
name: Deploy to K8s
on:
push:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up kubectl
uses: azure/setup-kubectl@v3
with:
version: 'v1.25.0'
- name: Configure kubectl
run: |
echo "${{ secrets.KUBE_CONFIG }}" | base64 -d > kubeconfig
export KUBECONFIG=kubeconfig
- name: Build and push
run: |
docker build -t registry.example.com/user-service:${{ github.sha }} .
docker push registry.example.com/user-service:${{ github.sha }}
- name: Deploy to K8s
run: |
kubectl set image deployment/user-service \
user-service=registry.example.com/user-service:${{ github.sha }}
- name: Verify deployment
run: |
kubectl rollout status deployment/user-service2. Helm 部署
# .github/workflows/helm-deploy.yml
name: Helm Deploy
on:
push:
tags:
- 'v*'
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Helm
uses: azure/setup-helm@v3
with:
version: 'v3.12.0'
- name: Configure kubectl
run: |
echo "${{ secrets.KUBE_CONFIG }}" | base64 -d > kubeconfig
export KUBECONFIG=kubeconfig
- name: Deploy with Helm
run: |
helm upgrade --install user-service ./charts/user-service \
--namespace production \
--set image.tag=${{ github.ref_name }} \
--wait最佳实践
1. 资源管理
resources:
requests:
cpu: "500m"
memory: "512Mi"
limits:
cpu: "1000m"
memory: "1Gi"2. 健康检查
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
readinessProbe:
httpGet:
path: /actuator/health/readiness
port: 8080
initialDelaySeconds: 30
periodSeconds: 53. 滚动更新
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 04. Pod 安全
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
capabilities:
drop:
- ALL5. 网络策略
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: user-service-network-policy
spec:
podSelector:
matchLabels:
app: user-service
policyTypes:
- Ingress
- Egress
ingress:
- from:
- podSelector:
matchLabels:
app: gateway
ports:
- protocol: TCP
port: 8080
egress:
- to:
- podSelector:
matchLabels:
app: mysql
ports:
- protocol: TCP
port: 3306
- to:
- podSelector:
matchLabels:
app: redis
ports:
- protocol: TCP
port: 6379总结
Kubernetes 是容器编排的事实标准，提供自动部署、扩缩容、服务发现、负载均衡等强大功能。
通过合理的资源配置、健康检查和监控告警，可以构建高可用、可扩展的微服务架构。
在生产环境中，建议结合 GitOps、Helm 等工具，建立完善的 CI/CD 和运维体系。