w wyl123ly 的博客
MD

01-05-自动驾驶人工智能的可解释性

25 KB 2026.10.13 原文件

System helps humans predict when self-driving cars will make mistakes

本篇导读

自动驾驶汽车由深度神经网络控制,而这类模型是典型的"黑箱":车突然刹住了,没人说得清它为什么刹。MIT 与自动驾驶公司 Motional 合作的 CW-Net 想解决这件事——把模型内部的推理"翻译"成人能看懂的概念("正在接近停着的车""离骑车人太近"),并保证这些解释是"因果忠实"的,不是事后编出来的理由。

本文是 MIT News 的科研报道,写作套路很典型:先抛问题场景,再介绍方法、给数据、举一个真实测试中的小故事。句子长度中等,专业名词不少但都自带解释,适合练习"读人物引语抓观点"。

一、英文正文

Self-driving cars are often controlled by deep learning models that sometimes fail in unexpected situations. For instance, the car might inexplicably brake and block the path of an oncoming emergency vehicle. A human driver or passenger may need to react rapidly to prevent a collision.

To help humans better anticipate a vehicle's mistakes, researchers from MIT and autonomous vehicle technology company Motional developed a new method that provides clear explanations of the underlying model's decisions.

Usually, the internal reasoning process of a deep learning model is opaque and difficult to understand. But the new method, called the Concept-Wrapper Network (CW-Net), translates that reasoning process into concepts that faithfully describe the autonomous vehicle's decisions without altering its driving performance.

CW-Net explains the decisions of machine learning-based planners using understandable concepts, like "approaching stopped vehicle" or "close to cyclist." These explanations can correct misconceptions drivers and passengers have about vehicle behavior and improve their situational awareness.

In road tests on a private track, CW-Net explanations helped safety drivers more accurately predict vehicle behavior; a larger simulation study with nonexpert users yielded similar results. These experiments show how CW-Net can provide important feedback for engineers as they troubleshoot in-vehicle artificial intelligence systems. In the longer term, this technique could boost the safety and transparency of autonomous vehicles, while building appropriate trust in drivers and passengers.

"This work shows how explanations are supportive to the human's mental model and understanding of the behavior of a system, and how it could be used in engineering and development to improve the technology," says Julie Shah, an MIT professor of aeronautics and astronautics, director of the Interactive Robotics Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL), and co-senior author of the paper on CW-Net. "Unless we are building these technologies in a way that we can rely on and predict their behavior, then it is a shaky and unsafe foundation for their use."

She is joined on the paper by lead author Eoin Kenny, a former MIT postdoc who is now a senior AI researcher at J.P. Morgan Chase; co-senior author Momchil Tomov, a staff research scientist at Motional; as well as Motional team members Akshay Dharmavaram, Sang Uk Lee, Tung Phan-Minh, Shreyas Rajesh, Yunqing Hu, and Laura Major, president and CEO of Motional. The research appears today in Nature.

Faithful explanations

Machine-learning-based planners act as the "brain" of a self-driving car. These powerful deep-learning architectures process data from the vehicle's cameras and lidar sensors, generate a high-level summary of the vehicle's environment, decide what the car should do next, and output a trajectory for it to follow.

The planners are usually black-box models, which means their internal decision-making process is so complex it is difficult to understand. This can leave scientists and safety drivers in the dark about why an autonomous vehicle made an unexpected decision, like phantom braking.

The researchers designed CW-Net to explain a vehicle's decisions using understandable concepts, while ensuring those explanations accurately reflect the true reasons behind its behavior.

"Especially in high-stakes settings like self-driving cars, it's important that the explanations are not potentially misleading. Because CW-Net is causally faithful in how it makes decisions, that provides certain guarantees around the explanations," Kenny says.

CW-Net is a "concept classifier," an AI algorithm that has been trained to predict the high-level concepts that exist within input data. The researchers plug the CW-Net module into the middle of an autonomous vehicle's existing machine-learning planner architecture.

It translates the model's internal reasoning process into understandable concepts, like "approaching stopped vehicle" or "close to cyclist." Then it forces the final piece of the planning model architecture to use those concepts when it decides what the vehicle should do next. In this way, CW-Net ensures the concepts faithfully explain the vehicle's actions.

At the same time, CW-Net uses the concepts it classified to generate clear explanations that are output along with the vehicle trajectory, in real-time.

"Instead of just wondering why the car stopped, having real-time data provides feedback that lets you test the system during deployment. You could also give that data to an engineer to potentially improve the system," Kenny says.

The researchers trained CW-Net to predict concepts using a dataset of 130 million examples of scenes from self-driving cars, with multiple labeled concepts in each scene. Using such a large, labeled dataset enables it to identify concepts accurately in a wide range of settings.

They also designed CW-Net to mimic the driving decisions of machine-learning-based planners, so the module would not negatively impact vehicle performance.

In the end, CW-Net generates accurate, understandable explanations without altering the original deep learning model.

Improving situational awareness

To test CW-Net, the researchers deployed the module on a real autonomous driving test vehicle (a Motional robotaxi) on a private track with a safety driver. They found that CW-Net helped the safety driver better predict how the vehicle would behave in surprising situations.

For instance, the vehicle consistently stopped when it approached a cyclist, and the safety driver assumed it did so because it detected that cyclist. But CW-Net explanations revealed that the model wasn't properly configured to detect the cyclist and chose a trajectory that would have caused a collision. Instead, it stopped because its emergency braking procedure kicked in when it got too close.

Armed with this information about the model's mistake, the safety driver could reduce speed or engage manual driving mode sooner in similar situations. This could also help engineers fix the model to avoid this failure in the future.

In larger online simulation studies using real driving situations captured on the roads of Las Vegas, the researchers saw similar results. CW-Net explanations significantly improved participants' abilities to predict how an autonomous vehicle will behave.

In the future, the researchers could extend CW-Net so the module can cover more concepts and explore different training and design techniques that could boost performance and improve interpretability.

"Our study shows how crucial interpretability can be to these high-stakes environments, and how it should be on the mind of people as they are making AI in the future, for self-driving cars or other safety-critical environments," Kenny says.


二、生词与短语精讲

用法

先通读正文,再回来核对。建议先遮住中文释义自测。

单词/短语音标词性英文释义中文释义文中例句
inexplicably/ˌɪnɪkˈsplɪkəbli/adv.in a way that cannot be explained or understood莫名其妙地、无法解释地For instance, the car might inexplicably brake and block the path of an oncoming emergency vehicle.
anticipate/ænˈtɪsɪpeɪt/v.to expect something and be ready for it预判、预料To help humans better anticipate a vehicle's mistakes, researchers from MIT and autonomous vehicle technology company Motional developed a new method that provides clear explanations of the underlying model's decisions.
underlying/ˌʌndəˈlaɪɪŋ/adj.existing beneath the surface as the real cause底层的、根本的To help humans better anticipate a vehicle's mistakes, researchers from MIT and autonomous vehicle technology company Motional developed a new method that provides clear explanations of the underlying model's decisions.
opaque/əʊˈpeɪk/adj.difficult or impossible to understand不透明的、难以理解的Usually, the internal reasoning process of a deep learning model is opaque and difficult to understand.
faithfully/ˈfeɪθfəli/adv.accurately and exactly, without distortion忠实地、如实反映地In this way, CW-Net ensures the concepts faithfully explain the vehicle's actions.
alter/ˈɔːltə(r)/v.to change something改变、改动But the new method, called the Concept-Wrapper Network (CW-Net), translates that reasoning process into concepts that faithfully describe the autonomous vehicle's decisions without altering its driving performance.
misconception/ˌmɪskənˈsepʃn/n.a wrong idea or belief about something误解、错误看法These explanations can correct misconceptions drivers and passengers have about vehicle behavior and improve their situational awareness. ...
situational awareness/ˌsɪtʃuˈeɪʃənl əˈweənəs/n.knowing what is happening around you right now情境意识、态势感知These explanations can correct misconceptions drivers and passengers have about vehicle behavior and improve their situational awareness. ...
nonexpert/ˌnɒnˈekspɜːt/n.a person without special knowledge in a field非专业人士A larger simulation study with nonexpert users yielded similar results. ...
yield/jiːld/v.to produce a result or an amount产生、得出(结果)A larger simulation study with nonexpert users yielded similar results. ...
troubleshoot/ˈtrʌblʃuːt/v.to find and fix faults in a system排查故障、检修CW-Net can provide important feedback for engineers as they troubleshoot in-vehicle artificial intelligence systems. ...
transparency/trænsˈpærənsi/n.the quality of being open and easy to inspect透明度In the longer term, this technique could boost the safety and transparency of autonomous vehicles, while building appropriate trust in drivers and passengers.
shaky/ˈʃeɪki/adj.weak and not to be relied on不牢靠的、摇晃的Unless we are building these technologies in a way that we can rely on and predict their behavior, then it is a shaky and unsafe foundation for their use. ...
postdoc/ˌpəʊstˈdɒk/n.a researcher working temporarily after a PhD博士后She is joined on the paper by lead author Eoin Kenny, a former MIT postdoc who is now a senior AI researcher at J.P. Morgan Chase ...
lidar/ˈlaɪdɑː(r)/n.a sensor that measures distance using laser light激光雷达These powerful deep-learning architectures process data from the vehicle's cameras and lidar sensors, generate a high-level summary of the vehicle's environment, decide what the car should do next, and output a trajectory for it to follow.
trajectory/trəˈdʒektəri/n.the path that a moving object follows轨迹、行进路线These powerful deep-learning architectures process data from the vehicle's cameras and lidar sensors, generate a high-level summary of the vehicle's environment, decide what the car should do next, and output a trajectory for it to follow.
black-box/ˌblæk ˈbɒks/adj.whose internal workings cannot be inspected黑箱的The planners are usually black-box models, which means their internal decision-making process is so complex it is difficult to understand.
phantom braking/ˈfæntəm ˈbreɪkɪŋ/n.braking with no visible obstacle in the way幽灵刹车、莫名刹车This can leave scientists and safety drivers in the dark about why an autonomous vehicle made an unexpected decision, like phantom braking.
high-stakes/ˌhaɪ ˈsteɪks/adj.involving serious risk of loss or harm高风险的、利害重大的Especially in high-stakes settings like self-driving cars, it's important that the explanations are not potentially misleading.
misleading/ˌmɪsˈliːdɪŋ/adj.giving a wrong idea or impression误导性的It's important that the explanations are not potentially misleading. ...
causally faithful/ˈkɔːzəli ˈfeɪθfl/adj.reflecting the real causes of a decision因果忠实的Because CW-Net is causally faithful in how it makes decisions, that provides certain guarantees around the explanations," Kenny says.
classifier/ˈklæsɪfaɪə(r)/n.an algorithm that sorts data into categories分类器CW-Net is a "concept classifier," an AI algorithm that has been trained to predict the high-level concepts that exist within input data.
plug into/plʌɡ ˈɪntuː/phrasal v.to connect a part into a larger system接入、插入The researchers plug the CW-Net module into the middle of an autonomous vehicle's existing machine-learning planner architecture.
deployment/dɪˈplɔɪmənt/n.the stage of putting a system into real use部署、上线运行Having real-time data provides feedback that lets you test the system during deployment. ...
labeled/ˈleɪbld/adj.having tags that state what each item is已标注的Using such a large, labeled dataset enables it to identify concepts accurately in a wide range of settings.
mimic/ˈmɪmɪk/v.to copy the way something else behaves模仿、模拟They also designed CW-Net to mimic the driving decisions of machine-learning-based planners, so the module would not negatively impact vehicle performance.
emergency braking/ɪˈmɜːdʒənsi ˈbreɪkɪŋ/n.automatic hard braking to avoid a crash紧急制动It stopped because its emergency braking procedure kicked in when it got too close. ...
kick in/kɪk ˈɪn/phrasal v.to start to take effect开始起作用、启动Its emergency braking procedure kicked in when it got too close. ...
interpretability/ɪnˌtɜːprɪtəˈbɪləti/n.how far a model's behaviour can be understood by people可解释性"Our study shows how crucial interpretability can be to these high-stakes environments, and how it should be on the mind of people as they are making AI in the future, for self-driving cars or other safety-critical environments," Kenny says. ...
safety-critical/ˌseɪfti ˈkrɪtɪkl/adj.where a failure could cause injury or death安全攸关的"Our study shows how crucial interpretability can be to these high-stakes environments, and how it should be on the mind of people as they are making AI in the future, for self-driving cars or other safety-critical environments," Kenny says. ...

高频搭配与短语动词

搭配中文例句
for instance例如For instance, the car might inexplicably brake and block the path of an oncoming emergency vehicle.
react rapidly to对……迅速作出反应A human driver or passenger may need to react rapidly to prevent a collision.
act as充当、起……的作用Machine-learning-based planners act as the "brain" of a self-driving car.
be in the dark about对……一无所知This can leave scientists and safety drivers in the dark about why an autonomous vehicle made an unexpected decision, like phantom braking.
at the same time与此同时At the same time, CW-Net uses the concepts it classified to generate clear explanations that are output along with the vehicle trajectory, in real-time.
in the end最终、到头来In the end, CW-Net generates accurate, understandable explanations without altering the original deep learning model.
be armed with掌握了、带着(信息)Armed with this information about the model's mistake, the safety driver could reduce speed or engage manual driving mode sooner in similar situations.
be on the mind of被……放在心上It should be on the mind of people as they are making AI in the future, for self-driving cars or other safety-critical environments ...
plug sth into sth把……接入……The researchers plug the CW-Net module into the middle of an autonomous vehicle's existing machine-learning planner architecture.
without altering在不改变……的情况下But the new method, called the Concept-Wrapper Network (CW-Net), translates that reasoning process into concepts that faithfully describe the autonomous vehicle's decisions without altering its driving performance.

三、长难句解析

长难句 1

原句:These experiments show how CW-Net can provide important feedback for engineers as they troubleshoot in-vehicle artificial intelligence systems. In the longer term, this technique could boost the safety and transparency of autonomous vehicles, while building appropriate trust in drivers and passengers.

结构拆解:第一句主干是 These experiments show...,how CW-Net can provide important feedback 是 how 引导的宾语从句;for engineers 说明反馈给谁;as they troubleshoot in-vehicle artificial intelligence systems 是时间状语从句,as 相当于 when,表示"在……的过程中"。第二句主干是 this technique could boost the safety and transparency,of autonomous vehicles 作后置定语;句末 while building appropriate trust in drivers and passengers 是 while + 动名词短语作伴随状语,注意 trust 前的 appropriate 暗示信任要"恰当",并非越多越好。In the longer term 意为"从长远看"。

参考翻译:这些实验表明,CW-Net 能在工程师排查车载人工智能系统故障的过程中提供重要反馈。从长远看,这项技术可以提升自动驾驶汽车的安全性与透明度,同时帮助驾驶员和乘客建立起恰当的信任。

长难句 2

原句:"This work shows how explanations are supportive to the human's mental model and understanding of the behavior of a system, and how it could be used in engineering and development to improve the technology," says Julie Shah, an MIT professor of aeronautics and astronautics, director of the Interactive Robotics Group in the Computer Science and Artificial Intelligence Laboratory (CSAIL), and co-senior author of the paper on CW-Net.

结构拆解:引语部分的主干是 This work shows how... and how...,两个 how 引导的宾语从句并列作 shows 的宾语,这种"同一引导词并列"的结构在学术英语里很常见。第一个从句用 be supportive to 表示"对……有支持作用",宾语是 the human's mental model and understanding,其后的 of the behavior of a system 是连续两个 of 短语作后置定语(of 套 of),需要逐层对应;第二个从句是被动语态 it could be used in engineering and development to improve the technology,to improve 作目的状语。引语之后 says Julie Shah 采用"动词在前、人名在后"的引述结构,随后用三个并列的同位语(an MIT professor...、director of...、and co-senior author of...)交代身份,是新闻英语介绍信源的典型写法,读的时候可以先跳过同位语,抓住说话人姓名即可。

参考翻译:"这项工作说明,解释能够支持人的心智模型、帮助人理解系统的行为,也说明解释可以如何应用于工程与开发环节来改进技术。"说这话的是朱莉·沙阿——麻省理工学院航空与航天学教授、计算机科学与人工智能实验室(CSAIL)交互机器人组主任,也是这篇关于 CW-Net 的论文的共同资深作者。

长难句 3

原句:Usually, the internal reasoning process of a deep learning model is opaque and difficult to understand. But the new method, called the Concept-Wrapper Network (CW-Net), translates that reasoning process into concepts that faithfully describe the autonomous vehicle's decisions without altering its driving performance.

结构拆解:第一句是"主系表"结构,主干为 the internal reasoning process is opaque and difficult to understand,主语后接 of a deep learning model 作后置定语;Usually 是频度副词作状语。第二句用 But 转折,主干是 the new method translates that reasoning process into concepts,用的是 translate A into B 搭配;两个逗号之间的 called the Concept-Wrapper Network (CW-Net) 是过去分词短语作插入语,交代新方法的名称。句末 that faithfully describe... 是定语从句修饰 concepts,其中 without altering its driving performance 是介词短语作状语,强调"解释归解释,开车性能不变"这一前提。

参考翻译:通常,深度学习模型内部的推理过程是不透明的、难以理解的。但这种名为"概念封装网络"(CW-Net)的新方法,能把那段推理过程转换成一些概念,而这些概念忠实地描述了自动驾驶汽车的决定,同时不改变它的驾驶表现。

四、专业词汇标注

专业词汇

这些词只需"认识",不必会拼写。

专业词汇中文解释所属领域
self-driving car / autonomous vehicle自动驾驶汽车智能交通
deep learning model深度学习模型:由多层神经网络从数据中学习规律机器学习
black-box model黑箱模型:内部决策过程难以被人理解人工智能
lidar激光雷达:用激光测距来感知周围环境的传感器自动驾驶感知
planner规划器:决定车辆下一步动作与路线的模块自动驾驶
trajectory轨迹:规划器输出的车辆未来行驶路径自动驾驶
interpretability / explainability可解释性:模型行为能被人类理解的程度人工智能
causally faithful explanation因果忠实的解释:解释反映的是真正影响决策的原因可解释 AI
concept classifier概念分类器:把输入数据映射到高层语义概念的算法机器学习
phantom braking幽灵刹车:前方无障碍却突然制动自动驾驶安全
situational awareness情境意识:对当前周围状况的把握人机交互
safety driver安全员:测试自动驾驶时坐在驾驶位随时接管的人自动驾驶测试
robotaxi无人驾驶出租车智能交通
emergency braking紧急制动:系统在危险临近时自动全力刹车车辆安全
training dataset训练数据集:用来训练模型的标注样本集合机器学习

五、阅读训练建议

  • [ ] 第一遍:不看注释通读全文,记录用时 ___ 分钟
  • [ ] 第二遍:查生词表,回读卡住的句子
  • [ ] 第三遍:只看英文,口头复述每段大意
  • [ ] 输出:用 5 句英文写出本文摘要

关联阅读

- 01-02-人工智能风险管理框架 —— 本文说的"透明度"与"可解释性",正是 NIST 框架里"可解释性与可说明性"这一类治理要求的工程落地。

- 01-04-大语言模型对医疗意味着什么 —— 同样的问题换到诊室:AI 给出建议时,医生和患者凭什么相信它。

- 01-03-人工智能规模扩张之路 —— 模型为何越来越"黑":参数与数据规模同时暴涨,是黑箱化的根源之一。

- 01-06-AI如何设计稳定新材料 —— 另一条思路:与其事后解释模型,不如一开始就把物理与化学规则塞进生成过程。

六、来源与版权说明

  • 原文标题:System helps humans predict when self-driving cars will make mistakes
  • 来源:MIT News(麻省理工学院新闻办公室),2026 年 9 月 2 日,作者 Adam Zewe
  • 原文链接:https://news.mit.edu/2026/system-helps-humans-predict-when-self-driving-cars-will-make-mistakes-0902
  • 说明:本篇英文正文为原文节选,逐字摘录自报道开头至结尾的连续英文段落(仅略去图片说明、联系方式、文末分享按钮与相关链接等非正文部分,故正文内未使用 [...] 标记),仅用于个人英语学习;版权归原作者与麻省理工学院所有。