AI systems with emotional intelligence could learn faster and be more helpful

拥有情感智能的AI系统可以学习得更快，也更有帮助

In the past year, have you found yourself under stress? Have you ever wished for help coping? Imagine if, throughout the pandemic, you’d had a virtual therapist powered by an artificial intelligence (AI) system, an entity that empathized with you and gradually got to know your moods and behaviors. Therapy is just one area where we think an AI system that can recognize and interpret emotions could offer great benefits to people.

在过去的一年里，你是否发现自己处于压力之下?你曾希望有人帮助你应对吗?想象一下，如果在流感大流行期间，你有一个由人工智能(AI)系统驱动的虚拟治疗师，这个实体与你感同身受，并逐渐了解你的情绪和行为。我们认为，能够识别和解释情绪的人工智能系统可以为人们带来巨大好处，治疗只是其中一个领域。

Our team hails from Microsoft’s Human Understanding and Empathy group, where our mission is to imbue technology with emotional intelligence. Why? With that quality, AI can better understand its users, more effectively communicate with them, and improve their interactions with technology. The effort to produce emotionally intelligent AI builds on work in psychology, neuroscience, human-computer interaction, linguistics, electrical engineering, and machine learning.

我们的团队来自微软的人类理解和同理心小组，我们的使命是为技术注入情商。为什么?有了这种品质，人工智能可以更好地理解用户，更有效地与他们交流，并改善他们与技术的互动。在心理学、神经科学、人机交互、语言学、电气工程和机器学习等领域的工作基础上，研发具有情感智能的人工智能。

Lately, we’ve been considering how we could improve AI voice assistants such as Alexa and Siri, which many people now use as everyday aides. We anticipate that they’ll soon be deployed in cars, hospitals, stores, schools, and more, where they’ll enable more personalized and meaningful interactions with technology. But to achieve their potential, such voice assistants will require a major boost from the field of affective computing. That term, coined by MIT professor Rosalind W. Picard in a 1997 book by the same name, refers to technology that can sense, understand, and even simulate human emotions. Voice assistants that feature emotional intelligence should be more natural and efficient than those that do not.

最近，我们一直在考虑如何改进人工智能语音助手，比如Alexa和Siri，它们现在被很多人用作日常助手。我们预计，它们很快就会被部署在汽车、医院、商店、学校和更多地方，它们将使与技术进行更个性化和更有意义的互动。但要实现它们的潜力，这种语音助手将需要情感计算领域的大力推动。这个术语是由麻省理工学院教授罗莎琳德·w·皮卡德在1997年的一本同名书中创造的，指的是能够感知、理解甚至模拟人类情感的技术。具有情商的语音助手应该比那些没有情商的语音助手更加自然和高效。

Consider how such an AI agent could help a person who’s feeling overwhelmed by stress. Currently, the best option might be to see a real human psychologist who, over a series of costly consultations, would discuss the situation and teach relevant stress-management skills. During the sessions, the therapist would continually evaluate the person’s responses and use that information to shape what’s discussed, adapting both content and presentation in an effort to ensure the best outcome.

想想这样一个人工智能代理是如何帮助一个感到压力过大的人的。目前，最好的选择可能是去见一位真正的人类心理学家，通过一系列昂贵的咨询，他将讨论情况并教授相关的压力管理技能。在治疗过程中，治疗师会不断评估患者的反应，并利用这些信息来塑造讨论的内容，调整内容和表现形式，以确保最佳结果。

While this treatment is arguably the best existing therapy, and while technology is still far from being able to replicate that experience, it’s not ideal for some. For example, certain people feel uncomfortable discussing their feelings with therapists, and some find the process stigmatizing or time-consuming. An AI therapist could provide them with an alternative avenue for support, while also conducting more frequent and personalized assessments. One recent review article found that 1 billion people globally are affected by mental and addictive disorders; a scalable solution such as a virtual counselor could be a huge boon.

虽然这种治疗可以说是现有的最好的治疗方法，虽然技术还远不能复制这种体验，但它对一些人来说并不理想。例如，有些人会觉得与治疗师讨论自己的感受很不舒服，有些人觉得这个过程让人感到耻辱或很耗时。人工智能治疗师可以为他们提供另一种支持途径，同时还可以进行更频繁、更个性化的评估。最近的一篇综述文章发现，全球有10亿人受到精神和成瘾障碍的影响;一个可扩展的解决方案，比如一个虚拟顾问可能是一个巨大的恩惠。

There’s some evidence that people can feel more engaged and are more willing to disclose sensitive information when they’re talking to a machine. Other research, however, has found that people seeking emotional support from an online platform prefer responses coming from humans to those from a machine, even when the content is the same. Clearly, we need more research in this area.

有证据表明，当人们与机器交谈时，他们会感觉更投入，更愿意透露敏感信息。然而，另一项研究发现，从在线平台上寻求情感支持的人更喜欢来自人类的回应，而不是来自机器的回应，即使内容是相同的。显然，我们需要在这一领域进行更多的研究。

About 1 billion people globally are affected by mental disorders; a scalable solution such as an AI therapist could be a huge boon.

全球约有10亿人受到精神障碍的影响;一个可扩展的解决方案，如人工智能治疗师可能是一个巨大的福音。

In any case, an AI therapist offers a key advantage: It would always be available. So it could provide crucial support at unexpected moments of crisis or take advantage of those times when a person is in the mood for more analytical talk. It could potentially gather much more information about the person’s behavior than a human therapist could through sporadic sessions, and it could provide reminders to keep the person on track. And as the pandemic has greatly increased the adoption of telehealth methods, people may soon find it quite normal to get guidance from an agent on a computer or phone display.

在任何情况下，人工智能治疗师提供了一个关键优势:它总是可用的。因此，它可以在意想不到的危机时刻提供关键支持，或者利用人们想要进行更多分析性谈话的时机。它可能比人类治疗师通过零星的治疗收集更多关于患者行为的信息，它可以提供提醒，让患者保持在正轨上。随着大流行大大增加了远程医疗方法的采用，人们可能很快就会发现，在电脑或手机显示器上获得代理人的指导是很正常的。

For this kind of virtual therapist to be effective, though, it would require significant emotional intelligence. It would need to sense and understand the user’s preferences and fluctuating emotional states so it could optimize its communication. Ideally, it would also simulate certain emotional responses to promote empathy and better motivate the person.

然而，要想让这种虚拟治疗师发挥作用，就需要很高的情商。它需要感知和理解用户的喜好和波动的情绪状态，这样才能优化沟通。理想情况下，它还会模拟某些情绪反应，以促进同理心，更好地激励人。

The virtual therapist is not a new invention. The very first example came about in the 1960s, when Joseph Weizenbaum of MIT wrote scripts for his ELIZA natural-language-processing program, which often repeated users’ words back to them in a vastly simplified simulation of psychotherapy. A more serious effort in the 2000s at the University of Southern California’s Institute for Creative Technologies produced SimSensei, a virtual human initially designed to counsel military personnel. Today, the most well-known example may be Woebot, a free chatbot that offers conversations based on cognitive behavioral therapy. But there’s still a long way to go before we’ll see AI systems that truly understand the complexities of human emotion.

虚拟治疗师并不是一个新发明。第一个例子出现在20世纪60年代，当时麻省理工学院的Joseph Weizenbaum为他的ELIZA自然语言处理程序编写了脚本，这个程序经常在极大简化的心理治疗模拟中重复用户的话给他们听。2000年代，南加州大学(University of Southern California)创新技术研究所(Institute for Creative Technologies)做出了一项更为严肃的努力，生产出了SimSensei，这是一种最初设计用于为军事人员提供咨询的虚拟人。今天，最著名的例子可能是Woebot，一个基于认知行为疗法提供对话的免费聊天机器人。但在我们看到人工智能系统真正理解人类情感的复杂性之前，还有很长的路要走。

Our group is doing foundational work that will lead to such sophisticated machines. We’re also exploring what might happen if we build AI systems that are motivated by something approximating human emotions. We argue that such a shift would take modern AI’s already impressive capabilities to the next level.

我们的小组正在做基础性工作，以研制出这种精密的机器。我们还在探索，如果我们构建的AI系统是由类似人类情感的东西驱动的，会发生什么。我们认为，这种转变将把现代人工智能已经令人印象深刻的能力提升到新的水平。

Only a decade ago, affective computing required custom-made hardware and software, which in turn demanded someone with an advanced technical degree to operate. Those early systems usually involved awkwardly large sensors and cumbersome wires, which could easily affect the emotional experience of wearers.

仅仅在十年前，情感计算还需要定制的硬件和软件，而这反过来又需要有高级技术学位的人来操作。这些早期的系统通常涉及笨重的传感器和电线，很容易影响佩戴者的情感体验

Today, high-quality sensors are tiny and wireless, enabling unobtrusive estimates of a person’s emotional state. We can also use mobile phones and wearable devices to study visceral human experiences in real-life settings, where emotions really matter. And instead of short laboratory experiments with small groups of people, we can now study emotions over time and capture data from large populations “in the wild,” as it were.

如今，高质量的传感器小巧而无线，可以不引人注意地估计一个人的情绪状态。我们还可以使用手机和可穿戴设备来研究现实生活中的人类本能体验，在那里情感真的很重要。我们现在可以长期研究情绪，并从“野外”的大量人群中获取数据，而不是在实验室里用一小群人做简短的实验。

To predict someone’s emotional state, it’s best to combine readouts. In this example, software that analyzes facial expressions detects visual cues, tracking the subtle muscle movements that can indicate emotion (1). A physiological monitor detects heart rate (2), and speech-recognition software transcribes a person’s words and extracts features from the audio (3), such as the emotional tone of the speech.

要预测一个人的情绪状态，最好结合读数。在这个例子中,软件分析面部表情检测视觉线索,跟踪可以表达感情的微妙的肌肉运动(1)生理监测检测心率(2),和语音识别软件转录一个人的文字和音频(3)提取特征,如演讲的感情基调。

Earlier studies in affective computing usually measured emotional responses with a single parameter, like heart rate or tone of voice, and were conducted in contrived laboratory settings. Thanks to significant advances in AI—including automated speech recognition, scene and object recognition, and face and body tracking—researchers can do much better today. Using a combination of verbal, visual, and physiological cues, we can better capture subtleties that are indicative of certain emotional states.

早期的情感计算研究通常使用心率或声调等单一参数来测量情感反应，并在人工实验室环境中进行。得益于人工智能的显著进步——包括自动语音识别、场景和物体识别、面部和身体跟踪——研究人员现在可以做得更好。通过结合语言、视觉和生理线索，我们可以更好地捕捉到暗示某些情绪状态的微妙之处。

We’re also building on new psychological models that better explain how and why people express their emotions. For example, psychologists have critiqued the common notion that certain facial expressions always signal certain emotions, arguing that the meaning of expressions like smiles and frowns varies greatly according to context, and also reflects individual and cultural differences. As these models continue to evolve, affective computing must evolve too.

我们也在建立新的心理学模型，更好地解释人们如何以及为什么表达他们的情绪。例如，心理学家批评了“特定的面部表情总是预示着特定的情绪”这一普遍观念，他们认为微笑和皱眉等表情的含义会随着语境的变化而变化，也反映出个体和文化的差异。随着这些模型不断进化，情感计算也必须进化。

This technology raises a number of societal issues. First, we must think about the privacy implications of gathering and analyzing people’s visual, verbal, and physiological signals. One strategy for mitigating privacy concerns is to reduce the amount of data that needs to leave the sensing device, making it more difficult to identify a person by such data. We must also ensure that users always know whether they’re talking to an AI or a human. Additionally, users should clearly understand how their data is being used—and know how to opt out or to remain unobserved in a public space that might contain emotion-sensing agents.

这项技术引发了一系列社会问题。首先，我们必须考虑收集和分析人们的视觉、语言和生理信号对隐私的影响。减轻隐私担忧的一个策略是减少需要离开传感设备的数据量，使通过这些数据来识别一个人变得更加困难。我们还必须确保用户始终知道他们是在与AI还是人类对话。此外，用户应该清楚地了解他们的数据是如何被使用的，并知道如何选择退出或在可能包含情感感知代理的公共空间中不被观察。

As such agents become more realistic, we’ll also have to grapple with the “uncanny valley” phenomenon, in which people find that somewhat lifelike AI entities are creepier than more obviously synthetic creatures. But before we get to all those deployment challenges, we have to make the technology work.

随着这些代理变得越来越现实，我们也将不得不应对“恐怖谷”现象，即人们会发现一些栩栩如生的AI实体比更明显的合成生物更令人毛骨悚然。但在我们解决所有这些部署挑战之前，我们必须让技术发挥作用。

As a first step toward an AI system that can support people’s mental health and well-being, we created Emma, an emotionally aware phone app. In one 2019 experiment, Emma asked users how they were feeling at random times throughout the day. Half of them then got an empathetic response from Emma that was tailored to their emotional state, while the other half received a neutral response. The result: Those participants who interacted with the empathetic bot more frequently reported a positive mood.

作为实现支持人们心理健康和幸福的人工智能系统的第一步，我们创建了情感感知手机应用艾玛。在2019年的一次实验中，艾玛随机询问用户一天中的感觉。然后，其中一半人从艾玛那里得到了符合他们情绪状态的同情回应，而另一半人则得到了中立的回应。结果是:那些经常与移情机器人互动的参与者报告了积极的情绪。

In a second experiment with the same cohort, we tested whether we could infer people’s moods from basic mobile-phone data and whether suggesting appropriate wellness activities would boost the spirits of those feeling glum. Using just location (which gave us the user’s distance from home or work), time of day, and day of the week, we were able to predict reliably where the user’s mood fell within a simple quadrant model of emotions.

在同一批人的第二个实验中，我们测试了我们是否可以从基本的手机数据推断人们的情绪，以及建议适当的健康活动是否会让那些感到沮丧的人振作起来。仅仅使用位置(即用户离家或工作的距离)，一天中的时间，以及一周中的一天，我们就能够可靠地预测用户的情绪在简单象限模型中的位置。

Depending on whether the user was happy, calm, agitated, or sad, Emma responded in an appropriate tone and recommended simple activities such as taking a deep breath or talking with a friend. We found that users who received Emma’s empathetic urgings were more likely to take the recommended actions and reported greater happiness than users who received the same advice from a neutral bot.

根据用户是高兴、平静、激动还是悲伤，艾玛用适当的语气回应，并推荐简单的活动，如深呼吸或与朋友交谈。我们发现，收到艾玛的移情请求的用户更有可能采取推荐的行动，并报告更大的幸福感，而收到中性机器人同样的建议的用户。

In one early experiment with Emma, an emotionally aware phone app, users were asked to rate their emotional state several times throughout the day, using a quadrant model of emotions.

在艾玛的早期实验中，一款情感感知的手机应用程序要求用户在一天中多次对自己的情绪状态进行评级，使用的是情绪象限模型。

We collected other data, too, from the mobile phone: Its built-in accelerometer gave us information about the user’s movements, while metadata from phone calls, text messages, and calendar events told us about the frequency and duration of social contact. Some technical difficulties prevented us from using that data to predict emotion, but we expect that including such information will only make assessments more accurate.

我们还从手机上收集了其他数据:手机内置的加速度计给我们提供了用户活动的信息，而电话、短信和日历事件的元数据则告诉我们社交联系的频率和持续时间。一些技术上的困难使我们无法使用这些数据来预测情绪，但我们预计，包含这些信息只会使评估更加准确。

In another area of research, we’re trying to help information workers reduce stress and increase productivity. We’ve developed many iterations of productivity support tools, the most recent being our work on “focus agents.” These assistants schedule time on users’ calendars to focus on important tasks. Then they monitor the users’ adherence to their plans, intervene when distractions pop up, remind them to take breaks when appropriate, and help them reflect on their daily moods and goals. The agents access the users’ calendars and observe their computer activity to see if they’re using applications such as Word that aid their productivity or wandering off to check social media.

在另一个研究领域，我们试图帮助信息工作者减轻压力，提高工作效率。我们已经开发了许多生产力支持工具的迭代，最近的是我们在“焦点代理”上的工作。这些助手在用户的日历上安排时间，以专注于重要的任务。然后，他们会监控用户对计划的坚持情况，在出现干扰时进行干预，提醒他们在适当的时候休息，并帮助他们反思自己的日常情绪和目标。这些特工会访问用户的日历，观察他们的电脑活动，看看他们是否在使用Word等有助于提高工作效率的应用程序，还是在闲逛，查看社交媒体。

To see whether emotional intelligence would improve the user experience, we created one focus agent that appeared on the screen as a friendly avatar. This agent used facial-expression analysis to estimate users’ emotions, and relied on an AI-powered dialogue model to respond in appropriate tones.

为了看看情商是否会改善用户体验，我们创建了一个焦点代理，它以友好的化身出现在屏幕上。该代理使用面部表情分析来评估用户的情绪，并依赖人工智能驱动的对话模型以适当的语气做出回应。

We compared this avatar agent’s impact with that of an emotionless text-based agent and also with that of an existing Microsoft tool that simply allowed users to schedule time for focused work. We found that both kinds of agents helped information workers stay focused and that people used applications associated with productivity for a larger percentage of their time than did their colleagues using the standard scheduling tool. And overall, users reported feeling the most productive and satisfied with the avatar-based agent.

我们比较了这个虚拟角色代理与一个没有感情的基于文本的代理的影响，以及一个现有的微软工具的影响，它只允许用户为专注的工作安排时间。我们发现，这两种代理都有助于信息工作者保持专注，人们使用与生产力相关的应用程序的时间比使用标准调度工具的同事更多。总体而言，用户报告说，他们对基于虚拟形象的代理的工作效率最高，也最满意。

Our agent was adept at predicting a subset of emotions, but there’s still work to be done on recognizing more nuanced states such as focus, boredom, stress, and task fatigue. We’re also refining the timing of the interactions so that they’re seen as helpful and not irritating.

我们的代理擅长预测情绪的一个子集，但在识别更细微的状态，如专注、无聊、压力和任务疲劳等方面仍有工作要做。我们还改进了互动的时间安排，使它们看起来是有益的，而不是令人不快的。

If an AI agent was motivated by fear, curiosity, or delight, how would that change the technology and its capabilities?

如果一个人工智能的动机是恐惧、好奇或快乐，这将如何改变技术和它的能力?

We found it interesting that responses to our empathetic, embodied avatar were polarized. Some users felt comforted by the interactions, while others found the avatar to be a distraction from their work. People expressed a wide range of preferences for how such agents should behave. While we could theoretically design many different types of agents to satisfy many different users, that approach would be an inefficient way to scale up. It would be better to create a single agent that can adapt to a user’s communication preferences, just as humans do in their interactions.

我们发现有趣的是，人们对我们移情的、具身的化身的反应是两极化的。一些用户从互动中感到安慰，而另一些人则发现虚拟形象分散了他们的工作。人们对这些代理应该如何行为表达了广泛的偏好。虽然理论上我们可以设计许多不同类型的代理来满足许多不同的用户，但这种方法是一种效率低下的扩展方式。最好创建一个能够适应用户通信偏好的单一代理，就像人类在交互中所做的那样。

For example, many people instinctively match the conversational style of the person they’re speaking with; such “linguistic mimicry” has been shown to increase empathy, rapport, and prosocial behaviors. We developed the first example of an AI agent that performs this same trick, matching its conversational partner’s habits of speech, including pitch, loudness, speech rate, word choice, and statement length. We can imagine integrating such stylistic matching into a focus agent to create a more natural dialogue.

例如，许多人本能地与对方的谈话风格保持一致;这种“语言模仿”已被证明能增加同理心、融洽关系和亲社会行为。我们开发了第一个执行相同技巧的人工智能实例，匹配其会话伙伴的说话习惯，包括音调、音量、说话速度、词语选择和语句长度。我们可以想象将这种风格匹配集成到焦点代理中，从而创建更自然的对话。

We’re always talking with Microsoft’s product teams about our research. We don’t yet know which of our efforts will show up in office workers’ software within the next five years, but we’re confident that future Microsoft products will incorporate emotionally intelligent AI.

我们总是和微软的产品团队谈论我们的研究。我们还不知道我们的哪些努力将在未来五年内出现在办公软件上，但我们有信心，未来的微软产品将整合具有情感智能的人工智能。

AI systems that can predict and respond to human emotions are one thing, but what if an AI system could actually experience something akin to human emotions? If an agent was motivated by fear, curiosity, or delight, how would that change the technology and its capabilities? To explore this idea, we trained agents that had the basic emotional drives of fear and happy curiosity.

能够预测并对人类情绪做出反应的人工智能系统是一回事，但如果人工智能系统真的能体验到类似于人类情绪的东西呢?如果一个代理人的动机是恐惧、好奇或快乐，这将如何改变技术和它的能力?为了探索这一想法，我们训练了具有恐惧和快乐好奇心等基本情感驱动的特工。

With this work, we’re trying to address a few problems in a field of AI called reinforcement learning, in which an AI agent learns how to do a task by relentless trial and error. Over millions of attempts, the agent figures out the best actions and strategies to use, and if it successfully completes its mission, it earns a reward. Reinforcement learning has been used to train AI agents to beat humans at the board game Go, the video game StarCraft II, and a type of poker known as Texas Hold’em.

通过这项工作，我们试图解决人工智能领域中被称为强化学习的一些问题，在强化学习中，人工智能智能体通过不断的试错学会如何完成一项任务。通过数百万次的尝试，代理会找出最好的行动和策略，如果成功完成任务，就会获得奖励。强化学习已被用于训练人工智能在棋类游戏围棋、电子游戏星际争霸2和一种名为德州扑克(Texas Hold’em)的扑克游戏中击败人类。

Our “focus agent” aimed to boost productivity by helping users schedule time to work on important tasks and helping them adhere to their plans. A camera (1) and computer software (2) kept track of the user’s behavior. The sensing framework (3) detected the number of people in view and the user’s position in front of the computer screen, estimated the user’s emotional state, and also kept track of the user’s activity within various applications. The agent app (4) controlled the focus agent avatar that engaged the user in conversation, using an AI-powered conversation bot (5) that drew on a variety of dialogue models to respond to the situation as appropriate.

我们的“焦点代理”旨在通过帮助用户为重要任务安排时间并帮助他们坚持自己的计划来提高生产力。一个摄像头(1)和一个计算机软件(2)跟踪用户的行为。感知框架(3)检测视野中的人数和用户在电脑屏幕前的位置，估计用户的情绪状态，并跟踪用户在各种应用程序中的活动。代理应用程序(4)控制焦点代理化身，使用户参与对话，使用人工智能驱动的对话机器人(5)，利用各种对话模型对适当的情况作出反应。

While this type of machine learning works well with games, where winning offers a clear reward, it’s harder to apply in the real world. Consider the challenge of training a self-driving car, for example. If the reward is getting safely to the destination, the AI will spend a lot of time crashing into things as it tries different strategies, and will only rarely succeed. That’s the problem of sparse external rewards. It might also take a while for the AI to figure out which specific actions are most important—is it stopping for a red light or speeding up on an empty street? Because the reward comes only at the end of a long sequence of actions, researchers call this the credit-assignment problem.

虽然这种类型的机器学习很适合游戏，因为在游戏中获胜会获得明确的奖励，但在现实世界中却很难应用。例如，考虑一下培训自动驾驶汽车的挑战。如果奖励能够安全到达目的地，AI便会花大量时间去尝试不同的策略，并且很少能够成功。这就是外部奖励稀少的问题。人工智能也可能需要一段时间才能弄清楚哪些具体行动是最重要的——是停下来等红灯还是在空旷的街道上加速?因为奖励只出现在一长串行动的最后，研究人员称之为信用分配问题。

Now think about how a human behaves while driving. Reaching the destination safely is still the goal, but the person gets a lot of feedback along the way. In a stressful situation, such as speeding down the highway during a rainstorm, the person might feel his heart thumping faster in his chest as adrenaline and cortisol course through his body. These changes are part of the person’s fight-or-flight response, which influences decision making. The driver doesn’t have to actually crash into something to feel the difference between a safe maneuver and a risky move. And when he exits the highway and his pulse slows, there’s a clear correlation between the event and the response.

现在想想人类在开车时的行为。安全到达目的地仍然是目标，但是这个人在这个过程中会得到很多反馈。在紧张的情况下，比如在暴雨中高速公路上超速行驶，人们可能会感到心跳加速，因为肾上腺素和皮质醇在体内流动。这些变化是人的“战或逃反应”的一部分，影响决策。司机不必真的撞上什么东西，就能感受到安全机动和冒险行动之间的区别。当他离开高速公路脉搏减慢时，事件和反应之间有明显的关联。

We wanted to capture those correlations and create an AI agent that in some sense experiences fear. So we asked people to steer a car through a maze in a simulated environment, measured their physiological responses in both calm and stressful moments, then used that data to train an AI driving agent. We programmed the agent to receive an extrinsic reward for exploring a good percentage of the maze, and also an intrinsic reward for minimizing the emotional state associated with dangerous situations.

我们想要捕捉这些关联并创造出某种程度上体验恐惧的AI代理。所以我们让人们在模拟环境中驾驶汽车穿过迷宫，测量他们在平静和紧张时刻的生理反应，然后用这些数据来训练人工智能驾驶机器人。我们为代理设定了一个程序，让其在探索相当一部分迷宫时获得外部奖励，同时也为最小化与危险情境相关的情绪状态提供内在奖励。

We found that combining these two rewards created agents that learned much faster than one that received only the typical extrinsic reward. These agents also crashed less often. What we found particularly interesting, though, is that an agent motivated primarily by the intrinsic reward didn’t perform very well: If we dialed down the external reward, the agent became so risk averse that it didn’t try very hard to accomplish its objective.

我们发现，将这两种奖励结合起来，会让玩家比只接受典型外部奖励的玩家学习得更快。这些代理程序崩溃的频率也较低。然而，我们发现特别有趣的是，一个主要由内在奖励驱动的代理并没有表现得很好:如果我们降低外部奖励，代理就会变得非常厌恶风险，以至于它不会努力去完成自己的目标。

During another effort to build intrinsic motivation into an AI agent, we thought about human curiosity and how people are driven to explore because they think they may discover things that make them feel good. In related AI research, other groups have captured something akin to basic curiosity, rewarding agents for seeking novelty as they explore a simulated environment. But we wanted to create a choosier agent that sought out not just novelty but novelty that was likely to make it “happy.”

在为AI代理构建内在动机的另一项努力中，我们思考了人类的好奇心，以及人们是如何被驱使着去探索，因为他们认为自己可能会发现让自己感觉良好的东西。在相关的人工智能研究中，其他研究小组捕捉到了类似于基本好奇心的东西，奖励在模拟环境中探索新奇事物的代理。但我们想要创造一个更挑剔的代理，不仅追求新奇，而且追求可能让它“快乐”的新奇。

We recorded the blood volume per pulse of test subjects while they drove through a virtual maze. In this example, the subject’s blood volume decreases between seconds 285 and 300. During that period, the driver collided with a wall while turning sharply to avoid another obstacle. This data was used to train an AI agent, which was given the objective of minimizing such stressful situations.

我们记录了测试对象在通过虚拟迷宫时每一次脉搏的血容量。在这个例子中，受试者的血容量在285到300秒之间下降。在此期间，司机急转弯以避开另一个障碍物时撞上了一堵墙。这些数据被用来训练一个人工智能代理，其目标是将这种压力情况降至最低。

To gather training data for such an agent, we asked people to drive a virtual car within a simulated maze of streets, telling them to explore but giving them no other objectives. As they drove, we used facial-expression analysis to track smiles that flitted across their faces as they navigated successfully through tricky parts or unexpectedly found the exit of the maze. We used that data as the basis for the intrinsic reward function, meaning that the agent was taught to maximize situations that would make a human smile. The agent received the external reward by covering as much territory as possible.

为了收集这样一个代理人的训练数据，我们要求人们在模拟的迷宫街道中驾驶一辆虚拟汽车，告诉他们去探索，但不给他们其他目标。当他们开车时，我们用面部表情分析来追踪他们脸上掠过的微笑，因为他们成功地通过了棘手的部分，或者意外地找到了迷宫的出口。我们使用这些数据作为内在奖励功能的基础，这意味着代理被教导要最大化能让人类微笑的情况。特工通过覆盖尽可能多的区域而获得外部奖励。

Again, we found that agents that incorporated intrinsic drive did better than typically trained agents—they drove in the maze for a longer period before crashing into a wall, and they explored more territory. We also found that such agents performed better on related visual-processing tasks, such as estimating depth in a 3D image and segmenting a scene into component parts.

我们再次发现，包含内在驱动的药物比一般训练的药物表现得更好——它们在迷宫中行驶更长的时间才撞到墙，探索的领域也更多。我们还发现，这些智能体在相关的视觉处理任务上表现得更好，比如估算3D图像的深度和将场景分割成多个组成部分。

We’re at the very beginning of mimicking human emotions in silico, and there will doubtless be philosophical debate over what it means for a machine to be able to imitate the emotional states associated with happiness or fear. But we think such approaches may not only make for more efficient learning, they may also give AI systems the crucial ability to generalize.

我们刚刚开始在硅谷模仿人类的情感，毫无疑问，对于机器能够模仿与快乐或恐惧相关的情感状态意味着什么，将会有哲学上的争论。但我们认为，这些方法不仅有助于提高学习效率，还可能赋予人工智能系统重要的泛化能力。

Today’s AI systems are typically trained to carry out a single task, one that they might get very good at, yet they can’t transfer their painstakingly acquired skills to any other domain. But human beings use their emotions to help navigate new situations every day; that’s what people mean when they talk about using their gut instincts.

今天的人工智能系统通常被训练去执行单一的任务，一个它们可能非常擅长的任务，但是它们不能把它们辛苦获得的技能转移到任何其他领域。但人类每天都会利用情感来帮助应对新的情况;这就是人们所说的凭直觉做事。

We want to give AI systems similar abilities. If AI systems are driven by humanlike emotion, might they more closely approximate humanlike intelligence? Perhaps simulated emotions could spur AI systems to achieve much more than they would otherwise. We’re certainly curious to explore this question—in part because we know our discoveries will make us smile.

我们希望赋予AI系统类似的能力。如果人工智能系统是由类似人类的情感驱动的，它们是否更接近于人类的智能?也许模拟情绪可以刺激人工智能系统取得比不使用它更大的成就。我们当然很想探索这个问题，部分原因是我们知道我们的发现会让我们微笑。

About the Author

Mary Czerwinski research manager of Microsoft’s Human Understanding and Empathy group where she works with Daniel McDuff and Javier Hernandez.

关于作者

Mary Czerwinski是微软人类理解和同理心小组的研究经理，她与Daniel McDuff和Javier Hernandez一起工作。