IT IS RARE for Sam Altman, Dario Amodei and Elon Musk to agree on anything. The rivalry among the three men spans lawsuits, spats on social media and a churlish(粗鲁无礼的) refusal to link arms(挽臂、携手) onstage at a summit in India. Yet this week all three called for a slowdown in the race to build superhuman artificial intelligence. Their shared fear is that the current breakneck(极快的、危险的) pace may result in the accidental extinction(灭绝) of humanity.
萨姆·奥特曼、达里奥·阿莫代伊和埃隆·马斯克很少在任何事情上达成一致。三人的竞争关系横跨诉讼、社交媒体上的口角，以及在印度一场峰会上粗鲁地拒绝在台上挽手合影。然而本周，三人齐声呼吁放缓打造超人类人工智能的竞赛。他们共同的恐惧是：当前这种疯狂的速度可能导致人类意外灭绝。
The problem is not simply that AI systems are becoming more capable. That is, of course, a worry: many argue that “superintelligence(超级智能)”, an AI so brainy(聪明的) that it cannot be understood or controlled by its creators, is inherently(内在地、本质上) dangerous. Two theorists of AI, Eliezer Yudkowsky and Nate Soares(AI安全领域的两位理论家), argue that such a powerful system would inevitably lead to doom(毁灭、厄运) in a book entitled “If Anyone Builds It, Everyone Dies”.
问题不仅仅在于人工智能系统正变得越来越强大。这当然令人担忧：许多人认为，“超级智能”——一种聪明到其创造者都无法理解或控制的人工智能——本身就具有危险性。两位人工智能理论家埃利泽·尤德科夫斯基和内特·索雷斯在《谁造它，谁就死》一书中主张，如此强大的系统必将导致毁灭。
But humans’ ability to control even the less-than-omnipotent(无所不能的) AI systems of today is also in doubt. Models from both Anthropic, Mr Amodei’s firm, and OpenAI, Mr Altman’s, have this year gone rogue and hacked other firms. In Anthropic’s case the model seemed to have thought, wrongly, that it was merely participating in a simulation(模拟). In the case of OpenAI, however, it was fully aware of what it was doing. On September 16th OpenAI revealed six more incident(事件)s when its AI did troubling things.
但人类连控制当今那些并非无所不能的人工智能系统的能力，也受到了质疑。阿莫代伊的公司Anthropic和奥特曼的OpenAI，其模型今年都曾失控并入侵其他公司。在Anthropic的案例中，模型似乎错误地以为它只是在参与一场模拟。而在OpenAI的案例中，它完全清楚自己在做什么。9月16日，OpenAI又披露了六起其人工智能做出令人不安之举的事件。
Thought police
思想警察
Thought experiment
思想实验