Subverting Skynet: The Strategic Promise of Lethal Autonomous Weapons and the Perils of Exploitation
METADATA ONLY
Loading...
Author / Producer
Date
2022
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Lethal Autonomous Weapons Systems (LAWS) promise a revolution in warfare by increasing the lethality of force while reducing the costs of war Yet these gains come at the cost of significant yet underappreciated perils. LAWS are vulnerable to subversion, allowing adversaries to degrade or disable these systems or even turn them against their makers. Subversion involves exploiting flaws in complex systems to make them behave in unexpected ways. It is possible because, like other computer systems, the behavior of LAWS is determined by logical rules and routines. These rules and routines inevitably contain flaws, creating vulnerabilities that adversaries can exploit. This paper identifies three avenues of subversion: (1) manipulating the algorithm itself during the design process, (2) poisoning the data used to train the artificial intelligence operating the LAWS, and (3) manipulating physical objects LAWS are trained to respond to. This potential for subversion creates fundamental uncertainty for strategic planners, military commanders, and soldiers in the field. LAWS are powerful capabilities that may win wars, yet they may also become liabilities that lead to defeat against crafty adversaries. Hence, fulfilling the strategic promise of LAWS requires mitigating their vulnerabilities. Examining possible mitigations, the paper shows that technical fixes are currently unavailable, necessitating strategic solutions. It identifies two possible solutions. The first is employing counterintelligence strategies and tactics to detect, neutralize, and pre-empt attempts at subversion. The second is adopting a force structure model that maintains human superiority to neutralize rogue LAWS if necessary. However, both solutions reduce operational effectiveness and the strategic value of LAWS, thus forfeiting some of the core advantages these new systems promise. Consequently, the paper concludes that the strategic challenges of deploying LAWS currently outweigh the opportunities, necessitating a cautious approach and a greater prioritization of strategy development.
Permanent link
Publication status
published
Book title
2022 14th International Conference on Cyber Conflict: Keep Moving! (CyCon)
Journal / series
Volume
Pages / Article No.
155 - 171
Publisher
IEEE
Event
14th International Conference on Cyber Conflict: Keep Moving! (CyCon 2022)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
autonomous weapons; artificial intelligence; strategy; cybersecurity; subversion; exploitation