Responsible Crowdsourcing A short set of slides towards operationalizing ethical practices in online microtask marketplaces

Transcription

1 Responsible Crowdsourcing A short set of slides towards operationalizing ethical practices in online microtask marketplaces Martha Larson Delft University of Technology 1

2 What is ethics? According to the Internet Encyclopedia of Philosophy The field of ethics (or moral philosophy) involves systematizing, defending, and recommending concepts of right and wrong behavior. Normative ethics takes on a more practical task, which is to arrive at moral standards that regulate right and wrong conduct. This may involve articulating the good habits that we should acquire, the duties that we should follow, or the consequences of our behavior on others. 2

3 Normative Principles in Applied Ethics According to the Internet Encyclopedia of Philosophy Personal benefit: acknowledge the extent to which an action produces beneficial consequences for the individual in question. Social benefit: acknowledge the extent to which an action produces beneficial consequences for society. Principle of benevolence: help those in need. Principle of paternalism: assist others in pursuing their best interests when they cannot do so themselves. Principle of harm: do not harm others. 3

4 Normative Principles (continued) According to the Internet Encyclopedia of Philosophy Principle of honesty: do not deceive others. Principle of lawfulness: do not violate the law. Principle of autonomy: acknowledge a person s freedom over his/her actions or physical body. Principle of justice: acknowledge a person s right to due process, fair compensation for harm done, and fair distribution of benefits. Rights: acknowledge a person s rights to life, information, privacy, free expression, and safety. 4

5 Defining Crowdsourcing A general definition: Crowdsourcing is the process of producing value or services in a manner that takes advantage of both human intelligence and human multiplicity. What needs to be done and who does it come into connection outside of the locus of a conventional workplace, where specific tasks are delegated to specific agents at specific locations. A specific definition: Here, we focus on crowdsourcing platforms, i.e., online microtask markets where crowdworkers can carry out microtasks for taskaskers in exchange for micropayments. 5

6 Why should we care about ethics? As taskaskers, we ourselves are actors in crowdsourcing ecosystems and stand to directly benefit from an ethical environment. (Benefits could be: sustainability, reliability, greater sense of satisfaction with our work.) If crowdsourcing can remain mutually beneficial for taskaskers and crowdworkers it will co-evolve with positive public perception and enabling (rather than restrictive) legislation. The work of Robert Axelrod (The Evolution of Cooperation 1984) gives cause to question assumption that greed pays in the long run

7 When should we care about ethics? (Always) In cases where crowdwork income is not essential for the crowdworkers, we can ask the questions: What are people not doing when they are crowdsourcing? Does crowdsourcing have an addictive side? In cases where people are subsisting on crowdwork income, we can ask the questions: Is the crowdwork paying only money, without building a longer term future? Is the money enough? Is the timing of the payment ok? 7

8 Suggestions for taskaskers striving to be ethical individuals Understand all roles in the community. Try some crowdwork. Do not reveal identity of the crowdworkers. Address crowdworkers politely (and apologize if you goof up). Don t engage in the race to the bottom, offering less and less reward for your tasks. Review work promptly. Create well designed microtasks that are engaging and ergonomic. Realize that asking crowdworkers to carry out tasks on a microtask platform requires an investment of your time not only money. Respond to inquiries (even the ones with horrible grammar). Use contracts to clearly define your relationships (LbyL 1 ) Be open and honest (LbyL 1 ) 1 Stephen M. Wolfson and Matthew Lease. Look Before You Leap: Legal Pitfalls of Crowdsourcing. ASIST

9 Suggestions for taskaskers striving to be ethical as a group When you think or talk about crowdworkers behavior, realize that there is more than one system involved in determining what is ethical and non-ethical behavior on the platform. (Reflect that there is widespread consensus that it is ethical to steal a loaf of bread if your child would otherwise starve.) Give people the information that they need to make informed decisions about the use of their work and their information. (Don t ask crowdworkers to carry out tasks that are used for questionable ends.) Understand the past. Protect health-state, competence, and personality and any other personal information of crowdworkers like it was priceless art. 9

10 More challenging ways towards responsible crowdsourcing Understand the past (see slide 11). Think beyond instant gratification (see slide 12). Engage in ethics by design (see slide 13). Spend lots of time following guidelines for human subject research. Spend lots of time doing non-science activities, like joint task forces to improve and innovate crowdsourcing practices. Be close to workers, where close can be 1 st, 2 nd, 3 rd degree relationships. Mirror the ethics mechanism of the real world on the crowdsourcing platform. 10

11 Learning from past lessons Employers generally have more social and economic power than the people they employ. This asymmetry opens opportunities for abuse. Assembly-lines offered a breakthrough in production at the end of the 19th Century, but were also associated with sociological problems. Human-being-as-machine is historically a problematic metaphor. Societies have fought hard in the past, and are still fighting to eliminate child labor. Taskaskers should be alert for evidence that their tasks are being carried out by children

12 Beyond instant gratification Ethics with a very broad horizon No one is complaining doesn t necessarily mean we have achieved an ethical environment: What about five to ten years from now? In the moment a cigarette is nice and we are oblivious to consequences. (Think about crowdworkers developing repetitive stress disorder or becoming stuck in a selfreinforcing hand-to-mouth lifestyle.) It took years for Flickr: Patrin legislation to start protecting consumers from lack of awareness of the long term consequences of smoking. 12

13 Ethics by design Architectures where the right information flows The value of a fair-trade product is not only its inherent worth, but also how it is produced. In order to support fair-trade crowdwork, each microtask unit should be associated with certification Flickr: jetalone information. Crowdworkers can feedback information and self organize if they can communicate among themselves. Every party on a crowdsourcing platform should be able to reach all other parties. Enforcing isolation is not the answer to ensuring that crowdworkers work independently on tasks where independent opinions are critical. 13

14 References J. Zittrain Minds for Sales v=dw3h-rae3uo Ross, J. Et al. Who are the crowdworkers? CHI 2010 Felsteiner, Alek. Sweatshop or Paper Route?: Child Labor Laws and In-Game Work. CrowdConf Stephen M. Wolfson and Matthew Lease. Look Before You Leap: Legal Pitfalls of Crowdsourcing. ASIST Kittur, Aniket et al. The future of crowdwork. CSCW See also: M. Lease Ethics and Crowd Computing