Pre-test of a lighting questionnaire for post-occupancy evaluation through cognitive interviewing

Transcription

1 CIBSE Technical Symposium April 2018, London, UK Pre-test of a lighting questionnaire for post-occupancy evaluation through cognitive interviewing User Lighting questionnaire Luminous environment Arch Seda Kaçel MArch MSc BArch AMSLL Prof Alpin Köknel Yener PhD MSc BSc Department of Architecture, Faculty of Architecture, Istanbul Technical University

2 Contents 1. Introduction 2. Cognitive interviewing 3. The cognitive interview for the assessment of lighting systems 3.1 Methodology 3.2 Results and analysis 3.3 The revision of the probed questions 4. Conclusion 2

3 Introduction Post-occupancy evaluation (POE) As part of POE studies, survey research and user questionnaires have been conducted widely. In questionnaire design, it is significant that the question is comprehensible and easy to understand. 3

4 Introduction Pre-testing surveys: Cognitive interviewing Pre-test methodologies are utilized in order to test and revise the questionnaires before the real study. As one of the pre-test methodologies, cognitive interviewing has been widely used in various disciplines. 4

5 Introduction Research aim This research aims to integrate the cognitive interviewing into POE studies in the field of lighting. Research methodology Step-1: Literature was reviewed in order to identify items and scales related to lighting. Step-2: Cognitive interviewing procedure was carried out with five undergraduate architecture students. Step-3: The interviews were analyzed, the questionnaire items and scales were revised according to this analysis. 5

6 Cognitive interviewing Survey research Some errors affect the survey results. Measurement error The measurement error occurs when respondents misunderstand the survey questions (Campanelli, 1997). A methodology, which helps to produce more comprehensible questions, can be useful. Thus, the understanding of the respondents can be enhanced. 6

7 Cognitive interviewing Cognitive interviewing is a pre-test methodology to assess the insight of respondents in a questionnaire study. The cognitive interviewing methodology is assessed as a qualitative instrument, which evaluates questions and considers cognitive actions of respondents (Miller, 2014). In a cognitive interviewing study of different survey modes, the respondents had different understandings of terms or they paid attention to the diverse parts in a question (Mohorko and Hlebec, 2016). In the questionnaire with 50 items, the problems were determined in eight items and improvements were done by modifications such as changing the way the question is expressed (through adding or removing words), removing the question and dividing the question divided into two (Hilton, 2017). One subscale in a questionnaire on sport psychology was expressed in an alternative way as a result of the cognitive interviewing study (Dietrich and Ehrlenspiel, 2010). In a study, how cognitive interviewing and eye-tracking supplemented each other was showed. Thus, more difficulties and more complicated questions were revealed (Neuert and Lenzner, 2016). 7

8 Step-2: Analysis Step-1: In-depth interviews Cognitive interviewing Step-1a: The respondent answers a question in the questionnaire. Step-1b: Think-aloud The respondent expresses ideas in words without intervention of the interviewer. And / Or Step-1b: Verbal probing The mental procedures of the respondent is assessed. The interviewer is more actively involved during the interview. Step-2: The interviewer describes and analyses the responses. 8

9 The cognitive interview for the assessment of lighting systems Methodology Sampling: Purposive sampling technique Participants: Five final year (Year 4) undergraduate architecture students, who took the Construction Project course (out of nine students) Location: The studio on the 2 nd floor, orientated towards southwest Interview time: During the course time (afternoon) in early December 2017 Istanbul Technical University (ITU) Faculty of Architecture Exterior (Photo by the author) 9

10 The cognitive interview for the assessment of lighting systems Methodology courtyard N ITU Faculty of Architecture Courtyard (Photo by the author) ITU Faculty of Architecture 2 nd Floor Plan (Ref 10

11 The cognitive interview for the assessment of lighting systems Methodology N The studio, where the cognitive interviews were conducted (Photo by the author) Exterior sky condition: Overcast sky with only minor occasions of direct sunlight Daylighting system: Daylight penetrated through the floorheight glazed facade. The manual blinds were fully open. Artificial lighting system: The suspended linear luminaires in two rows with fluorescent lamps were operated manually and switched on. 11

12 The cognitive interview for the assessment of lighting systems Methodology Table 1 The questionnaire items, scales and the probe questions with their references 12

13 The cognitive interview for the assessment of lighting systems Results & Analysis F: Female Table 2 The responses of the respondents for the questionnaire item Q1 and the probe question P1 13

14 The cognitive interview for the assessment of lighting systems Results & Analysis F: Female Table 3 The responses of the respondents for the questionnaire item Q2 and the probe question P2 14

15 The cognitive interview for the assessment of lighting systems Results & Analysis F: Female Table 4 The responses of the respondents for the questionnaire item Q3 and the probe question P3 15

16 The cognitive interview for the assessment of lighting systems Results & Analysis F: Female Table 5 The responses of the respondents for the questionnaire item Q4 and the probe question P4 16

17 The cognitive interview for the assessment of lighting systems Results & Analysis F: Female Table 6 The responses of the respondents for the questionnaire item Q5 and the probe question P5 17

18 The cognitive interview for the assessment of lighting systems Results & Analysis Table 7 The revisions on the questions based on the analysis of the think alouds and verbal probing 18

19 Conclusion Through the cognitive interviews in this study, some of the lighting questions were improved. As part of the on-going PhD research, a questionnaire on adaptive behaviour will be pre-tested. The adaptive behaviour of occupants, is believed to be assessed skilfully through the guidance of the cognitive interviewing. This is due to the fact that feedback of the occupants is received and real experiences can be learned through the pre-testing. This method can be beneficial to POE studies in the field of lighting by creating valid questions, which are insightful for the respondent as the occupant of the luminous environment. 19

20 Pre-test of a lighting questionnaire for post-occupancy evaluation through cognitive interviewing CIBSE Technical Symposium April 2018, London, UK Thank you for listening. Seda Kaçel, Alpin Köknel Yener Department of Architecture, Faculty of Architecture, Istanbul Technical University kacel@itu.edu.tr, yener@itu.edu.tr