Building Ships and Building Software:

Transcription

1 Building Ships and Building Software: Practicing Virtue in Software Design Chuck Huff St. Olaf College Northfield, MN Collaborators and Supporters Student Collaborators Britta Anderson, St. Olaf Nathan DeWall, Florida State Erin Engelbart, St. Olaf Mike Knoll, St. Olaf Cassie Seningen, St. Olaf Faculty Collaborators Simon Rogerson, DeMontfort Univ. Leicester, UK Supporters: National Science Foundation Proof of Concept DUE Curriculum Based Resources DUE Life Stories of Moral Exemplars in Computing SES Centre for Computing and Social Responsibility, Demontfort University St. Olaf College Building Virtues to Navigate a Rough Organizational Sea The difficult organizational context of moral behavior. Moral exemplars in computing who rise above this context What can this say about the skills we need to practice virtue in software design?

2 Organizational Influence on Moral Decisions Organizations control scope and complexity by using goal based rationality (Max Weber) This concentrates attention and energy on the efficient achievement of organizational goals As the organization concentrates on efficiency, it ignores other values, to its own detriment. This irrationality increases as narrowness of focus increases Individual Response to Organizational Influence Milgram s obedience study is a microcosm of the individual s response to organizational pressure The experimenter requires efficient performance of duty The individual must decide to obey, to disobey or to take some middle road. The punch line: most people obeyed The Obedience Paradigm The experimenter instructs a teacher to shock a student every time the student makes a mistake. They were required to choose between efficiently serving a role in an important experiment and protecting the student/victim. Milgram ran many variations to determine what influenced the teacher s willingness to continue. He also observed the teacher s attempts to cope with the conflict.

3 Coping with Immoral Orders Two thirds of the teachers chose to continue shocking the victim while the victim: Protested the pain Withdrew permission to continue Mentioned a heart condition Screamed to be let out Finally, fell completely silent All the teachers: Protested the victim s treatment Tried to call off the study Pressed the shock buttons so as to minimize shock Showed empathy for the victim Were worried about the victim s health Showed clear signs of distress Many of the teachers: Minimized harm to the victim Distanced themselves from the pain Narrowed their focus to efficient performance of duty Blamed the victim Organizational Processes of Moral Compromise Organizations narrow their focus as they rationalize their operations, and begin (irrationally) ignoring other values. Ordinarily moral employees narrow their focus in response to this rationalization, and begin (immorally) ignoring other values. Selective recruitment, socialization, and groupthink perpetuate this cycle. The technology employees design has its effects at a social distance, isolating designers and decision makers from the effects of their decisions. The Bad News (in the worst possible light) Highly committed, well-meaning, narrowly focused, competent computer professionals, guided by organizational imperatives and isolated from the effects of their design decisions can be induced to commit negligent, immoral, unethical, possibly even evil acts.

4 The Good News There are computer professionals who operate with moral integrity in this environment. We call them moral exemplars. Criteria for Selection by Panel Either a) a sustained commitment to moral ideals or ethical principles in computing, or b) sustained evidence of moral virtue in the practice of computing. A disposition to make computing decisions in accord with one's moral ideals or ethical principles. A willingness to risk one's self-interest for the sake of one's moral values. A tendency to be inspiring to other computing professionals and thereby to move them to moral action. The Interviews Life story interviews (McAdams, 1995) lasting 3 hours in which they told a series of stories (17 or more) about their professional life. Life chapters (and overview) Peak experience, Nadir experience, Turning point, Earliest memory, Childhood scene, Early career scene, Recent career scene, One other scene Positive and negative influences Influential stories about others (four of these) Positive and negative Futures Value questions (e.g. Most important value in system design) The Exemplars 24 total, 12 from the UK, 12 from Scandinavia 9 female, 15 male 4 in government, 11 in academia, 9 in industry (these categories overlapped) 3-5 early career, 5 retired, others ranged between Common Themes Positive focus Cultivated a network of social support Did not see themselves as morally extraordinary Practiced moral imagination in solutions to problems Achieved more freedom to act as they advanced in their career. Early and later career exemplars were aware of these changes.

5 Helpers vs. Reformers Not a true type system, but a useful tool for illuminating the way they navigated their careers. Helpers Focused on users Viewed users as having a need Viewed barriers as inert obstacles Took the role of providers of service/product Believed in the efficacy of helping users Were more positive in emotional tone Reformers Focused on the system Viewed individuals as victims of injustice Viewed barriers as active opposition Took the role of moral crusader Believed in the necessity of systemic reform Were more negative in emotional tone How can this inform software design? Many of our helper exemplars values are already a part of the values of software design: user focus, customer need, software quality. We need not become reformers (though we may) to have ethical integrity in software design. It may be as simple (and as hard) as doing software design well. The Engineering Virtues (Pritchard, 1998) that support our exemplars actions are learnable skills that can be used in service of these values Creativity under multiple constraints Recognizing and foreseeing difficulties Constructing, testing, and articulating solutions How to teach these Skills/Virtues? Narvaez & Lapsley (2004) draw on psychological work in expertise to suggest that moral skills should be learned in structured environments that Reward correct solutions Match explicit theory and strategy with practice and coaching Provide extensive, focused practice We learn a craft by producing [its] product; we become builders by building. Aristotle, Nicomachaen Ethics, Book II

6 Supporting Ethical Design Recognize that ethical action is based in shared values and learnable skills as much as in character traits. Make explicit the connections among these skills and values and current software design best practices. Provide designers explicit practice in these skills, embedded in real design problems in service of shared values. Provide networks of support for designers that maintain commitment to values and share problem solving and skill maintenance. We may do good in software design by doing software design well.