CS 5306 INFO 5306: Crowdsourcing and. Human Computation. Lecture 10. 9/26/17 Haym Hirsh

Transcription

1 CS 5306 INFO 5306: Crowdsourcing and Human Computation Lecture 10 9/26/17 Haym Hirsh

2 Infotopia, Chapter 3 Four Big Problems for Deliberating Groups

3 Four Big Problems for Deliberating Groups Amplifying ( architectural ) errors Hidden profiles and (favoring) common knowledge Cascades and polarization Group polarization

4

5

6 Architectural Errors We use heuristics, or rules of thumb, that lead us to make predictable errors. Availability heuristic: Familiarity: biases perceptions in terms of what we know ( If you can think of it, it must be important Wikipedia) Salience: television vs print news, recency

7 Architectural Errors Framing effects: Of those who have this procedure, 90 percent are alive after five years vs Of those who have this procedure, 10 percent are dead after five years Write down the last two digits of your SSN. How much would you pay for X?

8 Architectural Errors Representativeness heuristic: Estimate quantities based on how representative it is in your own experience If you ve only met people from CT who were rich, when asked if CT people are rich you might say yes because of your experience, not because of the true numbers

9 Architectural Errors Conjunction errors: Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? Linda is a bank teller. Linda is a bank teller and is active in the feminist movement.

10 Architectural Errors Egocentric bias: We believe others are like us What percentage of people like computers? Hindsight bias: Our estimation of what we would have done knowing the outcome doesn t match what we would have done Would the Broncos win the Superbowl?

11 Amplifying Architectural Errors

12 Amplifying Architectural Errors Groups that deliberate amplify the effects of these architectural errors Amplify reliance on the representativeness heuristic Increase framing effects More conjunction errors

13 Amplifying Architectural Errors Groups are more likely than individuals to escalate commitment to a wrong path Increases with great identification with the group

14 Amplifying Architectural Errors Groups lead to decreased use of the availability heuristic Groups lead to decreased use of the egocentric bias Groups are less susceptible to hindsight bias

15 Hidden Profiles and Common Knowledge Hidden profiles: Information that is present across group members but that they do not find Common knowledge: Groups focus on the information that they share rather than the information that they don t People with the most common knowledge usually have a disproportionate influence in discussion

16 Hidden Profiles and Common Knowledge when key information is unshared, groups are more likely to select a bad option after discussion than would their individual members before discussion Increases with group size

17 Hidden Profiles and Common Knowledge Low-status individuals are less likely to provide hidden information Group diversity impacts group deliberation

18 Cascades and Polarization Informational cascades: Imagine polling people one by one Person A: Answers X Person B: Answers X Everyone thereafter now faces an X bias, even if the two X s were random chance.

19 Cascades and Polarization Reputational cascades: Imagine polling people one by one Person A: Answers X Person B: Answers X You might not want to risk your reputation to disagree with them. This cascades onward

20 Group Polarization Members of a deliberating group typically end up in a more extreme position in line with their tendencies before deliberation began Heightened with a sense of shared group identity In-group members have more force than out-group members

21 Wikipedia: List of cognitive biases

22

23 Wikipedia: List of cognitive biases 108 Decision-making, belief, and behavioral biases 27 Social biases 49 Memory errors and biases Common theoretical causes of some cognitive biases

24 "Embracing Error to Enable Rapid Crowdsourcing Ranjay Krishna, Kenji Hata, Stephanie Chen, Joshua Kravitz, David A. Shamma, Li FeiFei, Michael Bernstein CHI 2016.

25 Slides from: 6D-RTo9AhQYhtTsFo

26 26

27 27

28 28

29 29

30 30

31 31

32 32

33 33

34 34

35 35

36 36

37 37

38 38

39 39

40 40

41 41

42 42

43 43

44 44

45 45

46 46

47 47

48 48

49 49

50

51 51

52 52

53 53

54 54

55 55

56 56

57 57

58 58

59 59

60 60

61 61

62 62

63 63

64 64

65 65

66 66

67 67

68 68

69 69

70 70

71 71

72 72

73 73

74 74

75

76 Number of reactions time 76

77 Number of reactions time 77

78 Number of reactions time 78

79 Number of reactions time 79

80 Number of reactions time 80

81 Number of reactions time 81

82 Number of reactions time 82

83 Number of reactions time 83

84 Number of reactions σ=92ms μ=379ms time 84

85 Readings for Next Time Infotopia, Chapter 4 The power of prediction markets, Mann, A., Nature, 538(7625), p