智能礼让系统 - 让车学会有温度的礼让一、实际应用场景描述想象一个真实场景在北京的胡同里一位老人推着自行车慢慢过马路系统应该检测到这是需要特别礼让的情况而在上海陆家嘴的早高峰系统识别出外卖小哥焦急的表情和急促的车铃声应该做出适度让行但不完全停下的智能决策到了成都的老街区遇到打麻将的老人搬桌椅占道车辆应该耐心等待并播放亲切的方言提示音。这就是我们要解决的痛点传统自动驾驶的固定规则无法适应中国复杂的文化语境。固定的红灯停绿灯行、斑马线必须停车在面对中国式人情世故时显得冰冷且低效。二、痛点分析1. 文化差异难以量化北方人的豪爽礼让 vs 南方人的细腻体贴 vs 西部人的热情周到2. 场景复杂性学校门口、菜市场、老街区、商务区各有不同的礼让逻辑3. 动态适应性同一地点不同时段早晨送孩子vs晚上跳广场舞礼让策略完全不同4. 人性化交互如何让机器理解挥手示意、点头致谢等非语言信号三、核心逻辑讲解我们的解决方案基于三层自适应架构┌─────────────────────────────────────────────────────────────┐│ 感知层 (Perception) ││ 摄像头 麦克风 雷达 → 识别行人特征、行为、表情、语音 │├─────────────────────────────────────────────────────────────┤│ 文化推理层 (Cultural Reasoning) ││ 地域识别 → 场景分类 → 人群画像 → 礼让等级计算 │├─────────────────────────────────────────────────────────────┤│ 决策执行层 (Decision Action) ││ 生成个性化礼让策略 → 执行动作 → 反馈学习 │└─────────────────────────────────────────────────────────────┘核心算法思想- 使用贝叶斯网络处理不确定性推理- 深度学习模型识别文化和行为特征- 强化学习持续优化礼让策略- 知识图谱存储各地礼让文化规则四、项目结构smart_yielding_system/├── README.md # 项目说明文档├── requirements.txt # 依赖包列表├── config/│ ├── regional_rules.yaml # 各地区礼让规则配置│ └── cultural_weights.json # 文化权重参数├── src/│ ├── perception/ # 感知模块│ │ ├── __init__.py│ │ ├── pedestrian_detector.py # 行人检测器│ │ ├── behavior_analyzer.py # 行为分析器│ │ └── emotion_recognizer.py # 表情识别器│ ││ ├── reasoning/ # 推理模块│ │ ├── __init__.py│ │ ├── region_classifier.py # 地域分类器│ │ ├── scene_analyzer.py # 场景分析器│ │ └── yielding_strategy.py # 礼让策略生成器│ ││ ├── decision/ # 决策模块│ │ ├── __init__.py│ │ ├── action_planner.py # 动作规划器│ │ └── feedback_learner.py # 反馈学习器│ ││ └── main.py # 主程序入口│├── models/ # 预训练模型├── data/ # 训练数据├── tests/ # 单元测试└── docs/ # 文档五、核心代码实现1. requirements.txtnumpy1.21.0opencv-python4.5.0torch1.10.0torchvision0.11.0scikit-learn1.0.0pandas1.3.0yaml0.2.5playsound1.2.2paddlepaddle2.3.0 # 百度飞桨用于中文语音识别2. config/regional_rules.yaml# 各地区礼让文化规则配置regions:beijing:name: 北京characteristics: [豪爽, 守规矩, 重视长辈]yielding_rules:elderly_crossing:priority: 10 # 最高优先级stop_distance: 2.0 # 停车距离(米)wait_time: 15 # 等待时间(秒)greeting: 您慢走注意安全school_zone:priority: 9speed_limit: 20 # 限速(km/h)announcement: 前方学校区域请减速慢行normal_pedestrian:priority: 5stop_distance: 1.5wait_time: 8shanghai:name: 上海characteristics: [细腻, 高效, 注重秩序]yielding_rules:elderly_crossing:priority: 9stop_distance: 1.8wait_time: 12greeting: 老先生/老太太您请先过。delivery_worker:priority: 7 # 对外卖小哥的特殊礼让stop_distance: 1.2wait_time: 5announcement: 辛苦了注意安全normal_pedestrian:priority: 6stop_distance: 1.2wait_time: 6chengdu:name: 成都characteristics: [悠闲, 热情, 有人情味]yielding_rules:mahjong_elderly: # 打麻将老人的特殊场景priority: 10stop_distance: 2.5wait_time: 30 # 可能需要等待收拾桌椅greeting: 慢慢来不着急哈tea_house_crowd:priority: 8stop_distance: 2.0wait_time: 20announcement: 各位茶客车辆礼让请慢行normal_pedestrian:priority: 5stop_distance: 1.5wait_time: 10guangzhou:name: 广州characteristics: [务实, 包容, 重商]yielding_rules:market_vendors:priority: 8stop_distance: 1.5wait_time: 15greeting: 老板您先忙我等您business_rush:priority: 6stop_distance: 1.0wait_time: 3announcement: 赶时间的朋友请小心通行normal_pedestrian:priority: 5stop_distance: 1.2wait_time: 5scenes:school_entrance:time_sensitive: truemorning_peak: 07:00-08:30evening_peak: 16:00-18:00special_rules:student_with_backpack: priority_boost: 2parent_waiting: patience_required: highresidential_alley:cultural_sensitivity: very_highelderly_density: highconversation_priority: trueflexible_timing: truecommercial_district:efficiency_weight: 0.8pedestrian_flow: dynamicadaptive_speed: truebehavior_indicators:urgency_levels:very_urgent: [running, waving_anxiously, carrying_heavy_load]urgent: [walking_fast, looking_at_phone_stressed, pushing_cart]normal: [walking, standing, chatting]leisurely: [strolling, playing_with_phone, group_chatting]social_signals:gratitude: [nodding, waving_hand, thumbs_up, smiling]impatience: [foot_tapping, checking_watch, sighing]confusion: [looking_around, scratching_head]3. src/perception/pedestrian_detector.py行人检测器模块负责检测并识别行人提取关键特征用于后续分析和礼让决策import cv2import numpy as npfrom typing import List, Dict, Tuple, Optionalfrom dataclasses import dataclassfrom enum import Enumclass PedestrianType(Enum):行人类型枚举ELDERLY elderly # 老年人CHILD child # 儿童ADULT_NORMAL adult_normal # 普通成年人DELIVERY_WORKER delivery # 外卖配送员VENDOR vendor # 摊贩STUDENT student # 学生dataclassclass PedestrianInfo:行人信息数据结构bbox: Tuple[int, int, int, int] # 边界框 (x, y, w, h)pedestrian_type: PedestrianType # 行人类型estimated_age: int # 估计年龄gender: str # 性别has_burden: bool # 是否携带重物is_in_group: bool # 是否在群体中behavior_state: str # 行为状态confidence: float # 检测置信度position: Tuple[float, float] # 位置坐标 (x, y)velocity: Tuple[float, float] (0, 0) # 速度向量class PedestrianDetector:智能行人检测器结合YOLOv5和特征分析不仅检测行人还分析其社会属性def __init__(self, model_path: str models/yolov5s.pt):初始化检测器Args:model_path: YOLOv5模型路径self.model_path model_pathself.net Noneself.classes [person, bicycle, car, motorcycle, bus, truck]self.confidence_threshold 0.5self.nms_threshold 0.4# 加载YOLOv5模型self._load_model()# 特征分析器self.age_estimator AgeEstimator()self.gender_estimator GenderEstimator()self.behavior_analyzer BehaviorAnalyzer()def _load_model(self):加载YOLOv5模型try:# 使用OpenCV的DNN模块加载YOLOself.net cv2.dnn.readNetFromDarknet(f{self.model_path[:-3]}.cfg,self.model_path)self.net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)self.net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)print(✅ YOLOv5模型加载成功使用GPU加速)except Exception as e:print(f⚠️ GPU模型加载失败使用CPU: {e})self.net cv2.dnn.readNetFromDarknet(f{self.model_path[:-3]}.cfg,self.model_path)def detect(self, frame: np.ndarray) - List[PedestrianInfo]:检测帧中的行人Args:frame: 输入图像帧 (BGR格式)Returns:检测到的行人信息列表height, width frame.shape[:2]# 预处理创建blobblob cv2.dnn.blobFromImage(frame, 1/255.0, (416, 416), swapRBTrue, cropFalse)self.net.setInput(blob)# 前向传播layer_names self.net.getLayerNames()output_layers [layer_names[i[0] - 1] for i in self.net.getUnconnectedOutLayers()]outputs self.net.forward(output_layers)# 解析检测结果pedestrians []boxes []confidences []class_ids []for output in outputs:for detection in output:scores detection[5:]class_id np.argmax(scores)confidence scores[class_id]if class_id 0 and confidence self.confidence_threshold: # person classcenter_x int(detection[0] * width)center_y int(detection[1] * height)w int(detection[2] * width)h int(detection[3] * height)x int(center_x - w / 2)y int(center_y - h / 2)boxes.append([x, y, w, h])confidences.append(float(confidence))class_ids.append(class_id)# 非极大值抑制indices cv2.dnn.NMSBoxes(boxes, confidences, self.confidence_threshold, self.nms_threshold)if len(indices) 0:for i in indices.flatten():x, y, w, h boxes[i]# 裁剪行人区域进行深度分析pedestrian_roi frame[y:yh, x:xw]# 分析行人属性ped_type self._analyze_pedestrian_type(pedestrian_roi)age self.age_estimator.estimate(pedestrian_roi)gender self.gender_estimator.estimate(pedestrian_roi)behavior self.behavior_analyzer.analyze(pedestrian_roi)# 判断是否携带重物或有群体行为has_burden self._detect_burden(pedestrian_roi)is_in_group False # 需要在多帧分析中确定pedestrian_info PedestrianInfo(bbox(x, y, w, h),pedestrian_typeped_type,estimated_ageage,gendergender,has_burdenhas_burden,is_in_groupis_in_group,behavior_statebehavior,confidenceconfidences[i],position(center_x, center_y))pedestrians.append(pedestrian_info)return pedestriansdef _analyze_pedestrian_type(self, roi: np.ndarray) - PedestrianType:分析行人类型通过多种特征判断体型、姿态、装备、行为等# 计算宽高比老年人通常步幅较小h, w roi.shape[:2]aspect_ratio h / w# 分析姿态特征pose_score self._analyze_pose(roi)# 检测是否携带外卖箱has_delivery_box self._detect_delivery_box(roi)# 检测是否推着货物has_cart self._detect_shopping_cart(roi)# 综合判断if has_delivery_box or (pose_score[rushing] and has_burden_like_object(roi)):return PedestrianType.DELIVERY_WORKERelif has_cart or self._detect_vendor_behavior(roi):return PedestrianType.VENDORelif aspect_ratio 2.5 or pose_score[slow_moving] or self._detect_white_cane(roi):return PedestrianType.ELDERLYelif pose_score[backpack] and aspect_ratio 2.0:return PedestrianType.STUDENTelif pose_score[small_body]:return PedestrianType.CHILDelse:return PedestrianType.ADULT_NORMALdef _analyze_pose(self, roi: np.ndarray) - Dict[str, bool]:分析行人姿态# 简化的姿态分析实际应用中应使用专门的姿态估计模型gray cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY)# 边缘检测找轮廓edges cv2.Canny(gray, 50, 150)contours, _ cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)score {slow_moving: False,rushing: False,backpack: False,small_body: False}if contours:largest_contour max(contours, keycv2.contourArea)area cv2.contourArea(largest_contour)# 根据轮廓特征判断if area 5000: # 面积较小可能是儿童score[small_body] Trueelif area 20000: # 面积较大可能是成年人负重score[backpack] Truereturn scoredef _detect_delivery_box(self, roi: np.ndarray) - bool:检测外卖箱特征# 在外卖箱中寻找蓝色/黄色矩形物体hsv cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)# 蓝色外卖箱颜色范围blue_lower np.array([100, 50, 50])blue_upper np.array([130, 255, 255])blue_mask cv2.inRange(hsv, blue_lower, blue_upper)# 黄色配送服/箱子yellow_lower np.array([20, 100, 100])yellow_upper np.array([30, 255, 255])yellow_mask cv2.inRange(hsv, yellow_lower, yellow_upper)return cv2.countNonZero(blue_mask) 1000 or cv2.countNonZero(yellow_mask) 1000def _detect_shopping_cart(self, roi: np.ndarray) - bool:检测购物车gray cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY)edges cv2.Canny(gray, 50, 150)# 寻找轮子形状圆形circles cv2.HoughCircles(gray, cv2.HOUGH_GRADIENT, 1, 20,param150, param230, minRadius5, maxRadius30)return circles is not None and len(circles[0]) 2def _detect_vendor_behavior(self, roi: np.ndarray) - bool:检测摊贩行为特征# 检测手持物品的特征秤、篮子、包装袋等hsv cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)# 塑料袋的颜色特征plastic_lower np.array([0, 0, 200])plastic_upper np.array([180, 30, 255])plastic_mask cv2.inRange(hsv, plastic_lower, plastic_upper)return cv2.countNonZero(plastic_mask) 800def _detect_burden(self, roi: np.ndarray) - bool:检测是否携带重物gray cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY)# 检测不对称负重背包在一侧left_half gray[:, :roi.shape[1]//2]right_half gray[:, roi.shape[1]//2:]left_std np.std(left_half)right_std np.std(right_half)# 一侧有明显重物会导致标准差差异asymmetry abs(left_std - right_std) / max(left_std, right_std)return asymmetry 0.3def _detect_white_cane(self, roi: np.ndarray) - bool:检测盲杖gray cv2.cvtColor(roi, cv2.COLOR_BGR2GRAY)# 霍夫直线检测找长条状物体edges cv2.Canny(gray, 50, 150)lines cv2.HoughLinesP(edges, 1, np.pi/180, threshold50, minLineLength50, maxLineGap10)if lines is not None:vertical_lines [l for l in lines if abs(l[0][3] - l[0][1]) abs(l[0][2] - l[0][0])]return len(vertical_lines) 1return Falseclass AgeEstimator:年龄估计器def estimate(self, roi: np.ndarray) - int:估计行人年龄简化版本基于面部特征和体型比例h, w roi.shape[:2]# 计算各种比例特征face_ratio h / w # 假设ROI中主要是上半身# 基于特征的经验估算if face_ratio 2.8:base_age 70elif face_ratio 2.3:base_age 55elif face_ratio 1.8:base_age 35elif face_ratio 1.4:base_age 20else:base_age 10# 添加随机扰动模拟真实情况noise np.random.normal(0, 5)return max(1, min(100, int(base_age noise)))class GenderEstimator:性别估计器def estimate(self, roi: np.ndarray) - str:估计性别基于肩宽、体型等特征h, w roi.shape[:2]# 简化版基于宽高比ratio w / hif ratio 0.6:return maleelse:return femaleclass BehaviorAnalyzer:行为分析器def analyze(self, roi: np.ndarray) - str:分析行为状态# 简化分析return normal4. src/reasoning/region_classifier.py地域分类器模块通过多模态数据识别当前所在地区为礼让策略提供文化背景import jsonimport requestsfrom typing import Dict, List, Tuple, Optionalfrom dataclasses import dataclassfrom enum import Enumimport numpy as npfrom geopy.geocoders import Nominatimfrom geopy.distance import geodesicclass RegionType(Enum):地域类型枚举BEIJING beijingSHANGHAI shanghaiGUANGZHOU guangzhouCHENGDU chengduOTHER otherdataclassclass RegionContext:地域上下文信息region_type: RegionTypecity_name: strdistrict: strcultural_traits: List[str]local_customs: Dict[str, any]confidence: floatcoordinates: Tuple[float, float]class RegionClassifier:智能地域分类器结合GPS、视觉地标、语音方言等多模态信息进行地域识别def __init__(self, config_path: str ../config/regional_rules.yaml):初始化地域分类器Args:config_path: 地域规则配置文件路径self.config_path config_pathself.regional_rules self._load_regional_rules()self.geolocator Nominatim(user_agentsmart_yielding_system)# 地域特征数据库self.landmark_database self._build_landmark_database()self.dialect_keywords self._build_dialect_database()# 当前识别结果缓存self.current_region: Optional[RegionContext] Noneself.recognition_history: List[RegionContext] []def _load_regional_rules(self) - Dict:加载地域规则配置import yamlwith open(self.config_path, r, encodingutf-8) as f:return yaml.safe_load(f)def _build_landmark_database(self) - Dict[str, Dict]:构建地标数据库return {RegionType.BEIJING.value: {landmarks: [{name: 天安门, keywords: [天安门, 城楼, 国旗], weight: 1.0},{name: 故宫, keywords: [故宫, 紫禁城, 角楼], weight: 0.9},{name: 胡同, keywords: [胡同, 四合院, 老北京], weight: 0.8},{name: 鸟巢, keywords: [鸟巢, 水立方, 奥体], weight: 0.7},{name: 王府井, keywords: [王府井, 步行街, 商业街], weight: 0.6}],architecture_style: [红墙, 琉璃瓦, 牌楼, 灰砖],vehicle_patterns: [出租车多, 公交专用道, 自行车道明显]},RegionType.SHANGHAI.value: {landmarks: [{name: 外滩, keywords: [外滩, 黄浦江, 万国建筑], weight: 1.0},{name: 东方明珠, keywords: [东方明珠, 陆家嘴, 三件套], weight: 0.95},{name: 豫园, keywords: [豫园, 城隍庙, 九曲桥], weight: 0.8},{name: 南京路, keywords: [南京路, 步行街, 世纪广场], weight: 0.7}],architecture_style: [石库门, 现代摩天楼, 法式梧桐, 精装修],vehicle_patterns: [网约车多, 高架密集, 电动车规范]},RegionType.CHENGDU.value: {landmarks: [{name: 宽窄巷子, keywords: [宽窄巷子, 青砖墙, 茶馆], weight: 1.0},{name: 锦里, keywords: [锦里, 武侯祠, 红灯笼], weight: 0.9},{name: 熊猫基地, keywords: [熊猫, 竹子, 保护区], weight: 0.85},{name: 春熙路, keywords: [春熙路, 太古里, IFS], weight: 0.7}],architecture_style: [川西民居, 竹林, 茶馆招牌, 麻将元素],vehicle_patterns: [电瓶车灵活, 路边喝茶多, 节奏较慢]},RegionType.GUANGZHOU.value: {landmarks: [{name: 广州塔, keywords: [小蛮腰, 珠江新城, 广州塔], weigh利用AI解决实际问题如果你觉得这个工具好用欢迎关注长安牧笛