flink-on-k8s-operator部署指南：从Helm Chart到生产环境的最佳实践

flink-on-k8s-operator部署指南从Helm Chart到生产环境的最佳实践【免费下载链接】flink-on-k8s-operator[DEPRECATED] Kubernetes operator for managing the lifecycle of Apache Flink and Beam applications.项目地址: https://gitcode.com/gh_mirrors/fli/flink-on-k8s-operator什么是flink-on-k8s-operatorflink-on-k8s-operator是一个Kubernetes operator用于管理Apache Flink和Beam应用程序的生命周期。它允许用户通过Kubernetes自定义资源(CR)来定义和管理Flink集群和作业极大简化了Flink在Kubernetes上的部署和运维流程。准备工作环境要求一个运行中的Kubernetes集群建议1.14版本kubectl v1.14Helm推荐v3版本go v1.12如果从源码构建验证集群信息kubectl cluster-info克隆代码仓库git clone https://gitcode.com/gh_mirrors/fli/flink-on-k8s-operator cd flink-on-k8s-operator使用Helm Chart部署Flink Operator准备Flink Operator镜像您可以使用官方发布的镜像export IMGgcr.io/flink-operator/flink-operator:latest或者根据开发者指南从源码构建自己的镜像。更新模板文件本地安装时需要cd helm-chart/flink-operator ./update_template.sh您可以设置以下环境变量来自定义脚本行为export IMGimage-name- 操作器镜像默认为flink-operator:latestexport NSnamespace-name- 安装操作器的命名空间默认为flink-operator-system添加Helm仓库helm repo add flink-operator-repo https://googlecloudplatform.github.io/flink-on-k8s-operator/安装Helm Charthelm install --name flink-operator flink-operator-repo/flink-operator --set operatorImage.name$IMG如果要安装到已存在的命名空间helm install --name flink-operator flink-operator-repo/flink-operator \ --set operatorImage.name$IMG,flinkOperatorNamespace.namemy-namespace,flinkOperatorNamespace.createfalse验证部署检查CRD是否创建kubectl get crds | grep flinkclusters.flinkoperator.k8s.io查看CRD详情kubectl describe crds/flinkclusters.flinkoperator.k8s.io检查Operator部署状态kubectl get deployments -n flink-operator-system验证Operator Pod是否运行kubectl get pods -n flink-operator-system查看Operator日志kubectl logs -n flink-operator-system -l appflink-operator --all-containers成功部署后您应该能看到类似以下的日志INFO setup Starting manager INFO controller-runtime.certwatcher Starting certificate watcher INFO controller-runtime.controller Starting workers {controller: flinkcluster, worker count: 1}创建示例Flink集群创建Flink会话集群kubectl apply -f config/samples/flinkoperator_v1beta1_flinksessioncluster.yaml创建Flink作业集群kubectl apply -f config/samples/flinkoperator_v1beta1_flinkjobcluster.yaml验证集群状态kubectl get flinkclusters查看集群详细信息kubectl describe flinkclusters CLUSTER-NAME提交作业到Flink集群通过Flink Web UI提交首先创建到JobManager的端口转发kubectl port-forward svc/[FLINK_CLUSTER_NAME]-jobmanager 8081:8081然后通过浏览器访问 http://localhost:8081 提交作业。通过命令行提交flink run -m localhost:8081 examples/batch/WordCount.jar --input ./README.txt监控Flink集群和作业监控Operatorkubectl logs -n flink-operator-system -l appflink-operator --all-containers -f --tail1000监控Flink作业查看作业提交状态和日志kubectl describe jobs CLUSTER-NAME-job-submitter kubectl logs jobs/CLUSTER-NAME-job-submitter -f使用Prometheus监控创建启用Prometheus指标导出器的Flink集群和PodMonitorkubectl apply -f examples/prometheus/flink_metric_cluster.yaml kubectl apply -f examples/prometheus/pod-monitor.yaml生产环境最佳实践部署多个Operator实例在多租户集群中可以为每个命名空间部署专用的Operatormake deploy \ IMGoperator-image \ FLINK_OPERATOR_NAMESPACEnamespace-to-deploy-operator \ RESOURCE_PREFIXkuberntes-resource-name-prefix \ WATCH_NAMESPACEnamespace-to-watch管理Savepoints请参考Savepoints指南了解如何使用Operator管理Savepoints。更新Flink集群和作业可以通过更新FlinkCluster自定义资源来更新Flink作业和集群。例如kubectl apply -f examples/update/wordcount-1.9.2.yaml kubectl apply -f examples/update/wordcount-1.9.3.yaml检查更新历史kubectl get controllerrevision kubectl get controllerrevision REVISION-NAME -o yaml控制日志行为可以使用spec.logConfig字段完全控制log4j和logback配置示例配置见log_config.yaml。控制Pod安全性和权限可以在FlinkCluster规范中设置PodSecurityContexttaskManager: ... securityContext: runAsUser: 9999 runAsGroup: 1000 fsGroup: 2000挂载外部卷如果您的部署需要更大的存储容量或更快的状态后端访问可以使用TaskManager配置中的volumeClaimTemplates选项taskManager: ... volumeClaimTemplates: - metadata: name: flink-storage spec: accessModes: [ ReadWriteOnce ] resources: requests: storage: 10Gi volumeMounts: - name: flink-storage mountPath: /opt/flink/storage卸载Operator卸载Helm Charthelm delete flink-operator删除CRD谨慎操作kubectl delete crd flinkclusters.flinkoperator.k8s.io注意删除CRD将导致所有FlinkCluster CR和相关的Kubernetes资源被终止因此应谨慎删除。总结通过Helm Chart部署flink-on-k8s-operator是在Kubernetes上管理Flink集群的便捷方式。本文介绍了从环境准备到生产环境最佳实践的完整流程帮助您快速上手并正确配置Flink Operator。如需更多详细信息请参考项目的官方文档。【免费下载链接】flink-on-k8s-operator[DEPRECATED] Kubernetes operator for managing the lifecycle of Apache Flink and Beam applications.项目地址: https://gitcode.com/gh_mirrors/fli/flink-on-k8s-operator创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考