Working with people. African Regional Training Workshop on Indoor Air Pollution and Household Energy Monitoring Kampala, Uganda, June 2005

Transcription

1 Module 1: Working with people African Regional Training Workshop on Indoor Air Pollution and Household Energy Monitoring Kampala, Uganda, June 2005 Acknowledgement: This presentation was prepared by Jonathan Rouse (WEDC/WHO) and Eva Rehfuess, WHO.

2 Working with people People Ethical considerations The basics Quantitative evaluation Qualitative evaluation

3 People

4 Why are people important? Why do we evaluate? Part of the project plan Necessary for future funding Contribute to knowledge and understanding To ensure we have served beneficiaries well, and to ensure we will serve beneficiaries better in the future Evaluation implicitly involves interaction with people: Module 2: indoor air pollution monitoring Module 3: health and well-being Module 4: Stove performance Module 5: Socio-economic impacts

5 Which people? Evaluation involves: Groups/individuals unaccustomed to being asked their opinions Not necessarily the heads of households Particular focus on women and children The poor Marginalised

6 How do we involve people? Monitoring & observation Visiting villages/ homes Gaining access to kitchens and living areas Installing equipment Questionnaires & discussions Asking for information Sometimes personal/sensitive/embarrassing Asking for time

7 Intrusive How would you feel? How would your study participant feel?

8 Why this session? Ethical considerations as integral part of any research on human subjects Responsibility to participants Sensitive Recognise and respect their part in evaluation Representative Ensure we accurately capture their views Evaluation methods are only tools Their effective use requires skills Evaluators can have a profound effect on quality of data

9 Ethical considerations

10 Need for Ethics Scientific imperative to advance knowledge versus Moral obligation not to harm human beings Ethical code of conduct for research on human volunteers

11 Three Basic Principles Beneficence: "the duty to do good" No harm to participants, whether intentionally or by failing to anticipate and avoid harm Research design should maximize benefits and minimize harm Respect for persons Autonomy or self-determination Voluntariness including choice to opt out of activities at a later stage Duty to protect persons with limited autonomy e.g. children, refugees, women Confidentiality/anonymity/privacy /privacy Justice use numeric codes (instead of address/name) on all forms/databases all records (health status, socioeconomic status etc.) should be stored in locked rooms with only study staff having access Research should not create injustices, e.g. in relation to risks and discomforts or benefits

12 Potential Harms Direct harm e.g. switching to kerosene may incur a risk of child poisoning through t kerosene ingestion e.g. income reduction due to extensive interviews Psychological harm e.g. emotional distress or embarrassment in relating experience with disease or death "Power relationships" with research participants or communities e.g. a community interviewer may abuse his role and knowledge Accidental or non-accidental disclosure of confidentiality Inappropriate communication of results e.g. stigmatization of tuberculosis-infected infected individual

13 Informed Consent Process Research as collaborative effort between the researcher and the research participants versus Research as activity carried out by the researcher on a research subject "Obtaining informed consent is a process that is begun when initial ial contact is made with a prospective subject and continues throughout the course of the study " Council for International Organizations of Medical Sciences (CIOMS) guidelines

14 Informed Consent Process Key Steps Community consent through informal and formal discussions with community leaders and members Clear understanding of each others' roles (including role of control group where applicable), expectations and concerns Appreciation that research has positive and negative impacts Careful analysis of all risks and benefits of research to individuals and community Selection of participants, based on justice principle Individual consent through informed consent form Source: Saxena, Capron, Peeling, WHO, Unpublished

15 Designing an Informed Consent Form Two parts: Information sheet Certificate of consent Important criteria: Avoid the use of technical jargon Use simple and un-ambiguous language Avoid a hierarchical approach Use local language Provide a reasonable amount of information Pilot information sheet (not consent) Source: Saxena, Capron, Peeling, WHO, Unpublished

16 How to Obtain Informed Consent from Illiterate participants Use of graphics One-on on-one one information sessions Use literate witness Use of audio or video tapes Head of household consent Respect cultural norm that women may wish to consult their husband or family members Children Obtain agreement from mother or other legally responsible person Source: Saxena, Capron, Peeling, WHO, Unpublished

17 Institutional Review Board (IRB) Approval for Studies Involving Human Subjects Many organizations will require formal IRB approval especially relevant if goal is to publish results in a scientific journal Relationship between indoor air pollution monitoring and formal IRB approval is a little unclear: area monitoring: less related to human subjects concerns personal exposure monitoring: more related to human subject concerns WHO view: All evaluation studies including questionnaires or indoor air pollution monitoring should have IRB approval.

18 Further Information on Ethical Considerations WHO's Research Ethics Review Committee Guidelines and Templates for Writing a research proposal Preparing informed consent documents are available at

19 The basics

20 Reversing roles For people-centred evaluation: The marginalised become the teachers Professionals become the learners Recognising the knowledge of participants: they are the experts on themselves Leave preconceptions behind Participation: The act of taking part or sharing in something 1970s: Survey overload Emergence from the South of participatory methods and approaches Established in international development vocabulary Rhetoric versus reality Common sense Natural outcome of genuine inquisitiveness

21 Participant choice What do you think of the workshop? Now what if - the workshop organisers weren t here? - questions were being asked by a fellow participant? - questions were being asked in a private room? Consider carefully Who is listening Where you are Who you are in relation to the participant Are you the most appropriate person for this aspect of the evaluation?

22

23

24 Getting information about people who are really relevant to the evaluation is often a question of listening to quieter voices The poorest and most vulnerable and marginalised are often the least articulate and least willing to speak.

25 The evaluator Who are you? Tell them who you are, and why you are there Hold eye contact * where appropriate * Don t intimidate e.g. with children crouch down Many poor fear authority They may not like the attention you attract Look interested Or better still, be interested! Chat! In some cultures you just can t / don t launch into personal questions. Feel your way

26 Gender The gender of evaluators and translators can significantly affect data. Household energy-related evaluation often needs to be undertaken with women, sometimes alone In Bangladesh not appropriate for two male evaluators to speak to female cooks in homes

27 When and where? Seasonal variation Avoid snapshot research e.g. visits restricted to dry season Seasonal charts Where? Privacy issues Can a large crowd gather? Ensure people feel comfortable Consider who is listening

28 Culture Taboos Men speaking to women Personal issues Embarrassing people How much do you earn? How often do you visit the latrine every day? Appreciating people's time pressures Are they missing work/earnings? Are they missing lunch hour to speak to you?

29 Perspectives Backgrounds and perspectives Participants versus evaluator Illiterate highly literate Uneducated highly educated Speak different languages From different countries and cultures How familiar are concepts in the research? Trends, interpreting diagrams, reading maps etc.

30 Expectations What are people expecting? Free stove? Money? Can impact how people answer May say they are healthier/less healthy than reality May say a stove is better/worse than reality What is in it for them? Often, directly, very little Honest and open Research fatigue

31 Quantitative evaluation

32 Contribution of Quantitative Methods Quantitative methods: can measure "quantifiables" quantifiables" track changes in e.g. emissions, pollution levels, exposures, time Important for "objectively" comparing one intervention against another Methodological approaches include: performance testing pollution monitoring questionnaires

33 The Need to Adapt Questionnaires to Local Conditions Local language and terminology e.g. specific words for illnesses/symptoms Climatic and geographic conditions e.g. need for heating Cultural taboos e.g. TB stigma in India Cultural practices e.g. temescal use in Guatemala Locally specific cooking devices and practices e.g. beer-brewing brewing in Nepal

34 The Importance of Pilot Testing Some Key Questions Does the flow of the questions work? e.g. heating questions in World Health Survey

35 Example: World Health Survey Questionnaire Do you heat your house when it is cold? No Yes What type of fuel do you mainly use for heating? Gas Electricity Kerosene Coal Charcoal Wood Agriculture/crop Animal dung Shrubs/grass Other What type of heating stove is used in your house? Open fire/stove without chimney/hood Open fire/stove with chimney/hood Closed stove with chimney Other Next question

36 The Importance of Pilot Testing Some Key Questions Does the flow of the questions work? e.g. heating questions in World Health Survey Are the words understood? Are they too difficult, too simple, ambiguous? e.g. wheezing, stove designs

37 Example: World Health Survey Questionnaire Q4048: What type of cooking stove is used in your house? Q4052: What type of heating stove is used in your house? Open fire or stove WITHOUT chimney or hood Closed stove with chimney Open fire or stove WITH chimney or hood

38 The Importance of Pilot Testing Some Key Questions Does the flow of the questions work? e.g. heating questions in World Health Survey Are the words understood? Are they too difficult, too simple, ambiguous? e.g. wheezing, stove designs Do the response categories capture all options? e.g. plastic used as fuel in South African slums Are there any cultural sensitivities in relation to specific questions? e.g. asking about cough symptoms in India Are the questions interpreted in the same way by different respondents? Do they measure what they are supposed to measure? (validity) Are the questions answered in the same way if repeated with the same respondent? (reproducibility)

39 Qualitative evaluation

40 Contribution of Qualitative Methods Qualitative evaluation Best suited to understanding users perspectives on needs and impact of interventions Useful for planning quantitative evaluation Methodological approaches include: Questionnaires In-depth, semi/un-structured discussions with groups or individuals Direct observation of behaviours and interactions Written and documentary information Participatory methods, e.g. diaries, art and drawing

41 Planning qualitative evaluation Easily generates large volumes of data Requires skilled planning, recording and analysis Avoid full-life stories: balance openness with direction Consider levels of detail necessary Do you really need to know their shoe size? Time consuming Chatting over tea Not the quick and easy option!

42 Interviews & discussions Semi-structured interviews: Clear set of issues to address Focussed but flexible Scope for interviewee to digress/ elaborate Un-structured Emphasis on interviewees thoughts Interviewer just introduces themes

43 Observation How do you cook on this stove? How do you light this fire? You know something best when you ve watched it or better still when you ve had a go!

44

45

46

47

48 Focus Group Discussions and participatory methods Another perspective: Collective versus aggregate views Detailed justifications Useful open-ended tool Can make some feel more comfortable But others, possibly less Participatory group methods Non verbal ways of unlocking people s views Particularly appropriate for illiterate populations e.g. Seasonal Charts, Ranking exercises

49 Further Reading UN/WHO Focus Group Manual Institute of Development Studies Participation Group Robert Chambers Whose Reality Counts? Putting the last first

50 What challenges have you encountered when working with people in your evaluations, and how have you addressed these challenges?