Toward Meaningful Human Control of Autonomous Weapons Systems Through Function Allocation

Transcription

1 1 Toward Meaningful Human Control of Autonomous Weapons Systems Through Function Allocation Marc Canellas* (Presenter) and Rachel Haga Cognitive Engineering Center, School of Aerospace Engineering, Georgia Institute of Technology *PhD Candidate, Research Engineer

2 Autonomous weapons systems (AWS) are already here 2 Systems with sensors, software, and hardware capable of making decisions autonomously (e.g. detect, path plan) Weapons capable of killing humans (e.g. M mm medium machine gun)

3 There have been two UN Meetings of Experts focused on one question 3 Can autonomous weapons systems satisfy international humanitarian law? Currently: The most referenced minimum standard would require meaningful human control those who plan or decide on an attack [must] have sufficient information and control over a weapon to be able to predict how the weapon will operate and what effects it will produce in the context of an individual attack, and thus, to make the required legal judgements (Article 36, 2013)

4 Meaningful human control (MHC) frames the discussions of AWS UN Institutes 1 Legal scholars of war 5 4 States-parties 2 Roboticists 3 Political military appointees 6 UK Parliamentary Under Secretary of State for Ministry of Defense: let us be absolutely clear that the operation of weapons systems will always be under human control Political scientists 4

5 Can autonomous weapons systems satisfy international humanitarian law? 5 Currently: The most referenced minimum standard would require meaningful human control How can a weapons system be certified for meaningful human control?? How can the discussions of meaningful human control be aligned? Legally and technically sufficient definition of meaningful human control

6 How can a weapons system be certified for meaningful human control? 6 What parameters can be identified that would provide assurance that MHC is indeed being exercised? (UN Institute for Disarmament Research, 2014) How can a weapons system be certified for meaningful human control? Certification requires the following: 1. What are the requirements or rules? 2. How is compliance measured? This work proposes the use of principles, models, and measures of function allocation to certify weapons systems: the allocation of work between human and automated agents

7 How can a weapons system be certified for meaningful human control? 7 What are the requirements or rules? Five principles of function allocation identify those requirements than any function allocation should meet (Feigh, Pritchett, & Kim, 2014) 1. Can each agent perform each function? 2. Can each agent perform functions together when necessary? 3. Can the overall work be performed with reasonable teamwork? 4. Can the overall work be performed in real, operational settings? 5. Was the function allocation the result of deliberate design decisions?

8 How can a weapons system be certified for meaningful human control? 8 How is compliance measured? Use modeling and measuring techniques to evaluate function allocation performance (Pritchett, Kim, & Feigh, 2014a; 2014b) Measure workload and taskload: Are there high workload spikes in off-nominal conditions and excessively low workload during normal operations? Identify mismatches between authority and responsibility: Is the automation is assigned to execute a function in an operational sense but the human is held accountable in an organizational and legal sense? Measure stability of the human s work environment How often is the human unable to predict when or if he/she will need to act?

9 Can autonomous weapons systems satisfy international humanitarian law? 9 Currently: The most referenced minimum standard would require meaningful human control How can a weapons system be certified for meaningful human control? Principles, measures, and models of function allocation How can the discussions of meaningful human control be aligned? Legally and technically sufficient definition of meaningful human control

10 How can the discussions of meaningful human control be aligned? Regulations: Non-governmental organizations proposed definitions of meaningful human control: Operators: U.S. Dept. of Defense defined levels of AWS (Directive , 2012) 10 (ICRAC, 2014) (Article 36, 2013) Prioritizing Selected Targets Reference frame: Function allocation ICRAC Article 36 DoD Engaging Targets

11 Example: DoD Human-Supervised AWS 11 Human-Supervised AWS (p. 14) An autonomous weapon system that is designed to provide human operators with the ability to intervene and terminate engagements, including in the event of a weapon system failure, before unacceptable levels of damage occur. Human-Supervised AWS (p. 13) are designed to allow human operators to override operation of the weapon system, but [the system] can select and engage targets without further human input after activation. Functions Acquiring and tracking targets Select targets HumSup AWS Information about target H* Information on mission objectives H* Information on weapon effects H* Cueing potential targets A* Prioritizing selected targets A* Timing when to fire A* Engage targets A Terminal guidance A* Intervening/terminating engagements H A* A

12 Proposed MHC Definitions and U.S. DoD Technical Definitions 12 Functions Regulators Developers Article 36 ICRAC AWS HumSup AWS Semi- AWS To regulators: Acquiring and tracking targets A* A* A* A* A Select targets H H A A H Is this the function allocation you intended? Information about target H H A* H* H* Information on mission objectives H H A* H* H* Information on weapon effects H H A* H* H* Cueing potential targets A* A* A* A* A Need to resolve the prioritizing Prioritizing selected targets A* H A* A* A Timing when to fire A* A* A* A* A selected Engage targets H H A A A targets allocation Terminal guidance A* A* A* A* A Intervening/terminating engagements H* H A* H H*

13 Proposed MHC Definitions and DoD Technical Definitions 13 Functions Regulators Article 36 ICRAC AWS Developers HumSup AWS Acquiring and tracking targets A* A* A* A* A Select targets H H A A H Semi- AWS Information about target H H A* H* H* Information on mission objectives H H A* H* H* Information on weapon effects H H A* H* H* Cueing potential targets A* A* A* A* A Prioritizing selected targets A* H A* A* A Timing when to fire A* A* A* A* A Engage targets H H A A A Terminal guidance A* A* A* A* A Intervening/terminating engagements H* H A* H H*

14 Can autonomous weapons systems satisfy international humanitarian law? 14 Currently: The most referenced minimum standard would require meaningful human control How can a weapons system be certified for meaningful human control? Principles, measures, and models of function allocation How can the discussions of meaningful human control be aligned? Function allocation form Legally and technically sufficient definition of meaningful human control

15 Future Work 15 Use function allocation form to examine discussions in other emerging human-automation domains: Autonomous vehicles Unmanned aerial vehicles (UAVs, drones) Include responsibility in function allocation form to measure authority-responsibility mismatches Develop formal methods for including function allocation as a certification technique for human-automated systems

16 Thank You Toward Meaningful Human Control of Autonomous Weapons Systems Through Function Allocation Marc Canellas (Presenter) and Rachel Haga Cognitive Engineering Center, School of Aerospace Engineering, Georgia Institute of Technology