一、背景与目标在个人学习Kubernetes的过程中我希望搭建一套完整的入口流量管理方案。本文将详细记录在一个3节点的裸金属K8s集群ubuntu-111/112/113中如何部署Traefik作为Ingress Controller同时运行在ubuntu-111、ubuntu-112节点并通过一个Python HTTP服务验证整个链路的可用性。二、环境规划K8s集群3节点无MetalLB使用HAProxy作为外部负载均衡Traefik版本Helm Chart 39.1.0-ea.2Traefik 3.x核心目标Traefik仅调度到ubuntu-111、ubuntu-112节点通过节点亲和性实现启用hostNetwork模式直接占用节点80/443端口HAProxy代理两个Traefik节点增加后端健康检查部署Python HTTP服务并通过Traefik Ingress暴露三、Traefik部署实践1. 节点标签与亲和性配置1.1 为多节点打标签给ubuntu-111、ubuntu-112都打上traefik-worktrue标签作为Traefik调度的目标节点# 给ubuntu-111打标签kubectl label nodes ubuntu-111 traefik-worktrue# 给ubuntu-112打标签kubectl label nodes ubuntu-112 traefik-worktrue# 验证标签是否添加成功kubectl get nodes ubuntu-111 ubuntu-112 --show-labels|greptraefik-work1.2 多节点亲和性配置在Traefik的values.yaml中配置节点亲和性匹配两个打标签的节点同时设置Pod反亲和性避免同一节点运行多个Traefik实例affinity:# 节点亲和性匹配traefik-worktrue的节点ubuntu-111/112nodeAffinity:requiredDuringSchedulingIgnoredDuringExecution:nodeSelectorTerms:-matchExpressions:-key:traefik-workoperator:Invalues:-true# 必须是字符串类型避免JSON解析错误# Pod反亲和性避免同一节点运行多个Traefik PodpodAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:-labelSelector:matchLabels:app.kubernetes.io/name:traefiktopologyKey:kubernetes.io/hostname# hostNetwork配置直接使用节点网络hostNetwork:truedeployment:replicas:2# 副本数与目标节点数一致2个pod:securityContext:capabilities:add:-NET_BIND_SERVICE# 允许绑定80/443端口runAsUser:0# 端口配置绑定节点80/443ports:web:port:80hostPort:80websecure:port:443hostPort:4432. 使用Helm部署Traefik将上述配置写入traefik-values.yaml执行以下命令完成部署# 添加Traefik Helm仓库helm repoaddtraefik https://traefik.github.io/charts helm repo update# 部署Traefik指定Chart版本使用自定义配置helminstalltraefik traefik/traefik\--namespace kube-system\--create-namespace\--version39.1.0-ea.2\-f traefik-values.yaml# 验证Traefik Pod调度结果应分别运行在ubuntu-111/112kubectl get pods -n kube-system -l app.kubernetes.io/nametraefik -o wide3. HAProxy配置多节点代理健康检查3.1 完整HAProxy配置文件修改HAProxy配置代理两个Traefik节点的80/443端口并增加TCP层健康检查验证Traefik服务可用性#!/bin/bashset-euo pipefail# 基础配置HAPROXY_NODE_IP192.168.56.102# HAProxy所在节点IPMASTER_NODES(192.168.56.111192.168.56.112192.168.56.113)K8S_API_PORT6443HAPROXY_PORT6443# Traefik配置代理ubuntu-111/112的80/443端口TRAEFIK_NODES(192.168.56.111192.168.56.112)# Traefik运行节点TRAEFIK_HTTP_PORT80TRAEFIK_HTTPS_PORT443# 安装haproxyaptupdate -yaptinstall-y haproxy net-tools||truemv/etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.bak$(date%Y%m%d%H%M%S)||true# 生成haproxy.cfgcat/etc/haproxy/haproxy.cfgEOF global log /dev/log local0 log /dev/log local1 notice chroot /var/lib/haproxy stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners stats timeout 30s user haproxy group haproxy daemon maxconn 2000 defaults log global mode tcp option tcplog option dontlognull timeout connect 5000 timeout client 50000 timeout server 50000 errorfile 400 /etc/haproxy/errors/400.http errorfile 403 /etc/haproxy/errors/403.http errorfile 408 /etc/haproxy/errors/408.http errorfile 500 /etc/haproxy/errors/500.http errorfile 502 /etc/haproxy/errors/502.http errorfile 503 /etc/haproxy/errors/503.http errorfile 504 /etc/haproxy/errors/504.http # K8s apiserver代理原有配置 frontend k8s-api-frontend bind${HAPROXY_NODE_IP}:${HAPROXY_PORT}default_backend k8s-api-backend backend k8s-api-backend mode tcp balance roundrobin option tcp-check EOF# 添加master节点到apiserver backendformasterin${MASTER_NODES[]};docat/etc/haproxy/haproxy.cfgEOF server master-${master//./-}${master}:${K8S_API_PORT}check inter 2000 fall 3 rise 2 EOFdone# Traefik HTTP代理多节点健康检查cat/etc/haproxy/haproxy.cfgEOF # Traefik HTTP代理hostNetwork模式 frontend traefik-http-frontend bind${HAPROXY_NODE_IP}:${TRAEFIK_HTTP_PORT}default_backend traefik-http-backend backend traefik-http-backend mode tcp balance roundrobin # TCP健康检查验证后端80端口是否可连接 option tcp-check tcp-check connect port${TRAEFIK_HTTP_PORT}tcp-check send GET /ping HTTP/1.1\r\nHost: localhost\r\n\r\ntcp-check expect string 200 OK EOF# 添加两个Traefik节点到HTTP backend带健康检查fornodein${TRAEFIK_NODES[]};docat/etc/haproxy/haproxy.cfgEOF server traefik-node-${node//./-}-http${node}:${TRAEFIK_HTTP_PORT}check inter 2000 fall 3 rise 2 timeout connect 3s EOFdone# Traefik HTTPS代理多节点健康检查cat/etc/haproxy/haproxy.cfgEOF # Traefik HTTPS代理hostNetwork模式 frontend traefik-https-frontend bind${HAPROXY_NODE_IP}:${TRAEFIK_HTTPS_PORT}default_backend traefik-https-backend backend traefik-https-backend mode tcp balance roundrobin # TCP健康检查验证后端443端口是否可连接 option tcp-check tcp-check connect port${TRAEFIK_HTTPS_PORT}EOF# 添加两个Traefik节点到HTTPS backend带健康检查fornodein${TRAEFIK_NODES[]};docat/etc/haproxy/haproxy.cfgEOF server traefik-node-${node//./-}-https${node}:${TRAEFIK_HTTPS_PORT}check inter 2000 fall 3 rise 2 timeout connect 3s EOFdone# 重启haproxy并验证systemctl daemon-reload systemctlenable--now haproxy systemctl restart haproxy# 验证HAProxy状态ifsystemctl is-active --quiet haproxy;thenecho-e\033[32mHAProxy部署成功echo-e - K8s API:${HAPROXY_NODE_IP}:${HAPROXY_PORT}echo-e - Traefik HTTP:${HAPROXY_NODE_IP}:${TRAEFIK_HTTP_PORT}(代理到${TRAEFIK_NODES[*]})echo-e - Traefik HTTPS:${HAPROXY_NODE_IP}:${TRAEFIK_HTTPS_PORT}(代理到${TRAEFIK_NODES[*]})\033[0melseecho-e\033[31mHAProxy启动失败\033[0msystemctl status haproxy --no-pagerexit1fi3.2 健康检查关键配置说明option tcp-check启用TCP层健康检查验证后端端口是否存活tcp-check send GET /ping HTTP/1.1\r\nHost: localhost\r\n\r\n向Traefik的/ping接口发送请求Traefik内置健康检查接口tcp-check expect string 200 OK期望返回200状态码确认Traefik服务正常check inter 2000 fall 3 rise 2每2秒检查一次连续3次失败标记为不可用连续2次成功恢复可用timeout connect 3s连接超时时间3秒避免健康检查阻塞。3.3 验证HAProxy健康检查# 查看HAProxy后端状态echoshow stat|socat /run/haproxy/admin.sock stdio|greptraefik# 查看HAProxy日志验证健康检查结果tail-f /var/log/haproxy.log|greptraefik四、hostNetwork vs NodePort差异与优劣对比特性hostNetwork模式NodePort模式网络模型Pod直接使用节点的网络命名空间监听节点IP和端口。Service在节点上打开一个端口NodePort流量通过kube-proxy转发到Pod。端口占用直接占用节点的80/443等标准端口同一节点上的多个Pod不能监听同一端口。使用30000-32767范围内的端口避免与主机服务冲突。性能性能最优无额外的NAT和转发开销。存在kube-proxy的转发开销性能略低。隔离性差Pod可以看到节点上的所有网络接口和连接。好Pod的网络栈与节点隔离。高可用需要通过Pod反亲和性确保每个节点只运行一个Pod配合HAProxy健康检查实现高可用。自动负载均衡到所有节点的NodePort高可用更简单但依赖kube-proxy。适用场景对性能要求极高的Ingress Controller、网络插件等。通用的服务暴露方式适合大多数应用。五、Python HTTP服务部署1. 服务代码新增容器信息获取fromhttp.serverimportBaseHTTPRequestHandler,HTTPServerimportos,socket,jsonclassHandler(BaseHTTPRequestHandler):defdo_GET(self):self.send_response(200)self.send_header(Content-type,text/html; charsetutf-8)self.end_headers()# 获取容器与Pod元数据pod_nameos.getenv(POD_NAME,socket.gethostname())node_nameos.getenv(NODE_NAME,unknown-node)app_info{app:python-http-server,container:os.getenv(CONTAINER_NAME,python-http-server)}responsef h1✅ Python HTTP Server/h1 pPod:{pod_name}/p pNode:{node_name}/p pApp Info:{json.dumps(app_info,indent2)}/p self.wfile.write(response.encode(utf-8))defrun(server_classHTTPServer,handler_classHandler,port8080):server_address(0.0.0.0,port)httpdserver_class(server_address,handler_class)print(f Python HTTP服务器启动: 0.0.0.0:{port})try:httpd.serve_forever()exceptKeyboardInterrupt:httpd.server_close()print( 服务器已停止)if__name____main__:run()2. K8s资源配置apiVersion:apps/v1kind:Deploymentmetadata:name:python-http-servernamespace:defaultspec:replicas:2selector:matchLabels:app:python-http-servertemplate:metadata:labels:app:python-http-serverspec:containers:-name:python-http-serverimage:192.168.56.102/library/python-http-server:v1ports:-containerPort:8080name:httpenv:-name:NODE_NAMEvalueFrom:fieldRef:fieldPath:spec.nodeName-name:POD_NAMEvalueFrom:fieldRef:fieldPath:metadata.name-name:CONTAINER_NAMEvalue:python-http-server---apiVersion:v1kind:Servicemetadata:name:python-http-servernamespace:defaultspec:selector:app:python-http-serverports:-port:80targetPort:8080protocol:TCPname:httptype:ClusterIP---apiVersion:networking.k8s.io/v1kind:Ingressmetadata:name:python-http-ingressnamespace:defaultannotations:kubernetes.io/ingress.class:traefikspec:rules:-host:python-http.example.comhttp:paths:-path:/pathType:Prefixbackend:service:name:python-http-serverport:number:80六、整体验证流程1. 验证Traefik调度kubectl get pods -n kube-system -l app.kubernetes.io/nametraefik -o wide# 预期结果两个Traefik Pod分别运行在ubuntu-111、ubuntu-1122. 验证HAProxy健康检查# 查看Traefik后端状态STATUS列为UP表示正常echoshow stat|socat /run/haproxy/admin.sock stdio|greptraefik|awk{print $1,$2,$18}3. 验证Python服务访问# 配置本地hostsecho192.168.56.102 python-http.example.com/etc/hosts# 多次访问验证负载均衡会轮询到两个Traefik节点curlhttp://python-http.example.comcurlhttp://python-http.example.com4. 故障模拟验证# 停止ubuntu-111上的Traefik Podkubectl delete pod -n kube-system$(kubectl get pods -n kube-system -l app.kubernetes.io/nametraefik -o name|grepubuntu-111|cut-d/ -f2)# 查看HAProxy后端状态ubuntu-111节点会被标记为DOWN流量自动切换到ubuntu-112echoshow stat|socat /run/haproxy/admin.sock stdio|greptraefik七、总结本次实践完成了Traefik在多节点ubuntu-111/112的部署通过节点标签和亲和性实现精准调度配合HAProxy的多节点代理和健康检查保障了入口流量的高可用。对比hostNetwork和NodePort模式清晰体现了不同网络模式的优劣也验证了Python HTTP服务通过Traefik Ingress暴露的完整链路为K8s流量管理的学习打下了坚实基础。