Challenges and opportunities for humanmachine collaboration at beyond human scales

Size: px

Start display at page:

Download "Challenges and opportunities for humanmachine collaboration at beyond human scales"

Flora Greene
5 years ago
Views:

1 Challenges and opportunities for humanmachine collaboration at beyond human scales Alex Morison Dave Woods Intelligent Human-Machine Collaboration

2 ~ Synonyms or placeholders for intelligent human-machine collaboration pursuing shared goals negotiation teaming assisting task-specific collaboration open collaboration multi-agent coordination (optimal distribution of tasks) explicit tasks or goals, no guessing optimized coordination well defined or not well-defined world where does initiative reside shared perception? Some workshop notes

3 ~ Other thoughts from workshop What is collaboration?...problem-holders and stake-holders......shared resp., auth., and goals......must negotiate and balance difficult trade-offs What capability is required for collaboration? communication, coordination, initiative, perceiving an environment, sensitivity to attention reciprocity? negotiability? Characteristics of collaboration asymmetric information, goals, both humans and machines are problem-holders Where does initiative reside? in the machine? in the human? What is the world? is it open (the world we live in) or closed (toy worlds) ~ Key opportunity (we didn t focus on much) Sensing systems (machines) are expanding human perceptual and attentional range Motivation

4 Challenge: Expanding spatial and temporal scales

5 Challenging: Expanding stakeholders

6 Challenge: Expanding modalities

7 ~ Challenges for beyond human scale sensor systems: exploring over wider spatial & temporal scales, collaboration, coordination, distance, modalities, algorithms, stakeholders, data (overload), others? ~ How do these new beyond human scales change the roles of (or challenge) human problem holders and stakeholders? ~ Two fundamental human processes are overlooked in coordinating human-sensor systems: Human perception Human attention ~ These human processes do not come for free According to Rodney Brook, HRI we would be fools not to take advantage of these abilities... Transitioning from human to beyond human scale

8 Human perception

9 Human perception

10 Human perception

11 Human perception

12 Human attention

13 Perceptual shortfall example 1

14 Perceptual shortfall example 1

15 Perceptual shortfall example 2

16 Perceptual shortfall example 2

17 Perceptual success, why?

18 Perceptual success, why?

19 Evaluating the perceptual shortfall

20 Evaluating the perceptual shortfall

21 Evaluating the perceptual shortfall

22 Sensor network example: sampling shortfall

23 ~ Our sensor network Sensors with only a partial view of total visual field Hemispheric viewable field Representation based on viewable field and center-surround rel. ~ Analogous to human attention Operates at a point-of-observation Uses a center-surround relationship Balances trade-off between focusing and reorienting ~ Key aspects Attention is a sampling process well-paced to environment Overcomes the potential for data overload at any single point-ofobservation The human eye is not an optical solution, it is a sampling solution, i.e., the eye is not a wide FOV sensor The sampling problem

~ Computational models of attention exist The

selection Input image Inhibition of return

Multiscale low-level feature extraction Other

competition Feature maps Orientations 0, 45, 90,

training ~ Limits to extending these models to

Fixed image extent Single process Sample

24 ~ Computational models of attention exist The number is growing Feature extraction Probabilistic selection Input image Inhibition of return Saliency map Attended location Winner-take-all Multiscale low-level feature extraction Other Motion, junctions and terminators, stereo disparity, shape from shading, etc. Centre surround differences and spatial competition Feature maps Orientations 0, 45, 90, 135, etc. Intensity On, off, etc. Colours Red, green, blue, yellow, etc. Feature combinations Top-down attentional bias and training ~ Limits to extending these models to beyond human scales. Fixed image extent Single process Sample orientation changes what is sampled ~ Artificial Attention (A W ) model Two separate but interdependent active processes (center and surround) A dynamic panorama (neither fixed nor static, extent emerges from interaction of the center and surround) Computational models of attention

25 The test environment

26 forward looking backward looking A projection representation

27 Sampling process output

28 Sampling process output

29 ~ Needed breakthroughs for collaboration Machines that move viewpoint to maximize the performance of the human-machine perceptual system Computational Models of Attention (Agents) that can operate over data sets that are beyond human scale ~ Summary We need better understanding of how to support perception and attention systems through sensors at human scales. We need better understanding of how to support perception and attention systems at beyond-human scales. Provided two examples of how we might use sensor systems to expand human fluency to knew scales Summary

30 ~ Thank you The end

31 ~ Potentially, for any single or set of tasks, brute force algorithms can be developed. ~ Brute force will likely... Decrease observability and directability Increase brittleness (decrease resilience) Exacerbate the data overload problem, distractions, losing the ability to stay in control; to effectively manage. ~ Designing for extending human reach beyond human scales Observability and directability follow naturally Resilience emerges from human problem- and stake-holders Why not brute force?

32 ~ Notes from Dave convo ~ compact response to control/automate ~ 1) you can automate but brittle (brute force) ~ 2) humans are problem holders ~ 3) smooth transfer of control ~ 4) directabiliy, observability come almost naturally ~ 6) they don t with algorithm design ~ 7) automate will only exacerbate the data overload problem, distractions, losing the ability to stay in control; to effectively manage. ~ 8) something you can t understand ~ friendly stuff ~ 9) this is how you get to resilience Slide description

Object Substitution Masking: When does Mask Preview work?

Object Substitution Masking: When does Mask Preview work? Stephen W. H. Lim (psylwhs@nus.edu.sg) Department of Psychology, National University of Singapore, Block AS6, 11 Law Link, Singapore 117570 Chua