REMOTE PERFORMANCE MEASUREMENT (RPM) A NEW, INTERNET-BASED METHOD FOR THE MEASUREMENT OF OCCUPANT PERFORMANCE IN OFFICE BUILDINGS

Transcription

1 REMOTE PERFORMANCE MEASUREMENT (RPM) A NEW, INTERNET-BASED METHOD FOR THE MEASUREMENT OF OCCUPANT PERFORMANCE IN OFFICE BUILDINGS J Toftum *, DP Wyon, H Svanekjær, A Lantner International Centre for Indoor Environment and Energy Department of Mechanical Engineering, Technical University of Denmark ABSTRACT A new software tool for measuring the performance of typical office tasks over the Internet (RPM = Remote Performance Measurement) was developed and tested in a crossover temperature intervention study in an office building. It provides easy collection and automatic scoring of the kind of data that it is otherwise both expensive and very difficult to obtain in practice. A significant negative effect of raising the room temperature set-point by 4ºC (from 20-22ºC to 24-26ºC) on the performance of an addition task was found. When a calculator was used to perform the task, no effect of temperature was found, i.e. raised temperature affected the cognitive task of performing mental arithmetic, not the trivial task of entering numbers on keyboards. Performance of a text-typing task was similarly unaffected. RPM offers a means of field-validating indoor environmental effects on performance that are predicted from experiments in simulated office environments or climate chambers. INDEX TERMS Office work, productivity, field intervention, temperature INTRODUCTION Remote Performance Measurement is an internet-based method that was developed to assess the effects of the indoor environment on occupants performance of simulated office tasks under field conditions, i.e. in practice. The effects of indoor environment upgrades, such as increased air change rate, improved temperature control or better air cleaning, include reduced absenteeism, and fewer symptoms and indoor environmental complaints (e.g. Milton et al. 2000; Sundell et al. 1994). Immediate effects on occupant performance have been identified in call-centres, where the recording and storage of employee output is a standard procedure (Federspiel et al. 2004; Tham 2004; Niemelä et al. 2002), and in well-controlled studies in simulated offices using simulated office work (Wargocki et al. 2000, Wargocki et al. 1999, Toftum et al. 2004). However, due to the complexity of measuring the output of office work in practice, no feasible method for the measurement of office work performance has been available. This paper describes a new method that uses simulated office tasks, which have been used successfully in simulated offices, to assess IEQ effects on performance in intervention studies in real office buildings. The tasks are administered over the internet via a standard web browser and add a performance response to already established methods for the measurement of occupant satisfaction (e.g. Zagreus et al. 2004). Until now, the method has been tested in buildings in Denmark and Italy. Preliminary results obtained in one of these buildings are presented in this paper. RPM Currently, RPM consists of a set of standard tasks and two different questionnaires that are used to characterize occupants perceptions and symptoms. One questionnaire focuses on occupants general perception of the indoor environment (e.g. during the past three to four months) and the other questionnaire focuses on conditions right now. The latter is used to assess the effects of an experimental intervention. Office work covers a wide range of different tasks involving a complex set of component skills. Typical tasks include text typing, proof reading, and different types of arithmetical calculations. If these tasks are affected by changes in the indoor environment, it is reasonable to assume that office work in general will be similarly affected. Experiments at the International Centre for Indoor Environment and Energy have demonstrated significant effects of IEQ on the performance of simulated office tasks, in exposures typically lasting five hours. In intervention * Corresponding author jt@mek.dtu.dk 357

2 studies in buildings in practice, the occupants carry out their regular work throughout the day. Applying RPM in the afternoon after more than half a workday ensures that occupants have been exposed to the indoor environment for many hours and that they are not as alert as in the morning. They are therefore likely to be more sensitive to environmental conditions than after shorter exposure times. It is thus likely that shorter task durations than were used in previous studies may result in measurable effects on performance, provided a sufficient number of occupants take part in the study. To date, the duration of individual RPM tasks has typically been between five and eight minutes, with a total duration of RPM engagement of 20 to 30 min, depending on the number of tasks. It may be difficult for employees to allocate longer periods to complete the tasks and their level of motivation might also decrease over time. Task outcome is generally quantified in terms of speed and accuracy. For each task presented to the occupant, different versions were prepared, so each occupant performs each version only once. The occupants are asked to work at their usual pace and not to exert any special effort A description of each task follows below. Text typing Occupants are asked to type successive sections of text (approximately 10 lines at a time) that are presented to them on the screen, exactly as it appears. The texts were taken from popular science magazines. To prevent occupants from simply copying and pasting the text to the textfield, the displayed text is an image embedded in the webpage. When the browser has finished loading the page, a timer starts and continues until the occupant submits the written text or until the total duration of the task has been exceeded, in which case an alert appears on the screen and the typed text is submitted automatically. As long as the total duration of the task has not been exceeded, new sections of text will appear. If an occupant needs to take a break to deal with an interruption he/she can click "Pause. The total duration of a task is extended by an amount corresponding to the total duration of pauses held during that particular task, i.e. the total time available for the task is unaffected by pauses or by the occupant choosing to re-read the instructions. For the whole duration of the text typing task, several performance measures are derived: number of characters typed per minute and an error rate. The error rate is determined by Levenshtein s Distance (LD), which is a measure of the similarity between two strings (shortest edit distance) (Levenshtein 1966). The edit distance is the smallest number of deletions, insertions, or substitutions required to transform the typed text into the original text that was presented on the screen. Addition In the addition task, occupants are asked to add columns of numbers and to enter the result in a field below each column. Several different complexities of the addition task have been used; addition of four two-digit numbers in one study and five three-digit numbers in another. The numbers are generated at random and contain no zeros. Typically, eight to ten columns of numbers (depending on the desired complexity) appear on the screen simultaneously. The timer function is similar to the text typing task, and new screens with numbers will appear as long as the time available for the task has not expired. Performance is defined in terms of number of additions completed per time unit and the percentage error rate. After the task occupants are asked if a calculator was used to perform the task. Proof reading In the proof-reading task occupants are asked to read and highlight words that are wrong in sections of texts that appear successively. The texts were again taken from popular science magazines and they contained four types of deliberately inserted errors, which on average occurred once in each four lines of text. A word considered wrong should be highlighted by clicking on it. The timer function is similar to the text typing task, and as long as the time has not expired, new texts will be presented. Three measures of performance are obtained: number of words read per time unit, correctly marked words, number of false positives and the number of words that should have been marked, but were not (omissions). 358

3 Questionnaires In addition to performance tasks, questionnaires were developed that deal with occupants general, long-term perception of their building as well as with immediate impressions and perceptions during an intervention. A typical intervention study will start with the general questionnaire to be completed prior to any interventions, followed by repeated use of the instantaneous questionnaire and by the performance tasks. FIELD APPLICATION OF RPM Temperature is a parameter which has a documented influence on human performance and it was therefore initially decided to validate the RPM approach in a temperature intervention study (Witterseh et al. 2004). It was hypothesized that possible effects of temperature on perceptions and performance would appear after a short exposure time (e.g. half a working day) and it was thus decided to limit the intervention period to two days. Three buildings, two with mechanical and one with natural ventilation, were included in the study. The results from one of the mechanically ventilated buildings are presented in this paper. The study comprised two three-storey blocks of a building complex used by the Department of Trading and Treasury Operations Support of a Danish banking company. Their participation in the study was motivated by employees frequent, but unspecific complaints about the indoor environment. Altogether, around 200 employees work in this department, most of them in open-plan offices. The offices were ventilated by a VAV system, which was forced to deliver only the minimum airflow during the intervention periods. Windows along the façades could be opened, but employees were asked not to do so during the intervention days. Radiators were located under the windows. The room temperature was controlled centrally, partly by the air supply temperature and partly by the water supply temperature. The study was conducted as a cross-over temperature intervention in which the temperature in each block on two consecutive days (Wednesday and Thursday) in two consecutive weeks was kept in the range C in one block and in the range C in the other. When no intervention was running, a temperature set-point of about C was used. In the afternoon of the day prior to the first intervention day, the regular control was overruled to stabilize the intervention temperature at the desired level when employees signed in the following day. Temperature and relative humidity were logged continuously at 3-4 locations in each office during the study period. Between 11 am and 6 pm on the intervention days, spot measurements of CO 2 and operative temperature were carried out throughout the offices. A flow finder was used to measure the flow rate at the supply and exhaust diffusers. Based on these measurements, the airflow was estimated to be between 23 and 30 L/s per person, varying somewhat between blocks and between storeys within each block. Prior to the introduction of any intervention, occupants answered the background questionnaire. Once during each intervention, employees first filled in the instant questionnaire and then completed the addition task for seven minutes and finally the text typing task for eight minutes. RESULTS The questionnaire was completed by 118 occupants in the warmer condition and by 119 occupants in the cool condition and 132 occupants completed at least one of the performance tests. The number of occupants who completed addition and text typing tasks during both interventions were 77 and 67, respectively. Figure 1 shows the distribution of occupants thermal sensation responses during the cool and warm conditions. The average room air temperature recorded during working hours was 21.4 ± 0.6 C in the cool condition and 25.1 ± 0.7 C in the warm condition. Figure 1 shows that the temperature interventions clearly affected occupants thermal sensation in the expected direction. The bipolar distribution of the votes also demonstrates that the temperature interventions could not be assumed to be blind to the occupants. Figure 2 shows the result of the addition task separately for occupants who used or did not use a calculator to solve the problems. A linear mixed effects analysis with occupant as random variable and treatment (warm/cold), calculator use, week (first/second) and calculator-treatment interaction as fixed effects showed no treatment main effect (p = 0.12), but a significant interaction effect (p = 0.03) and a significant effect of using a calculator (p = 0.01), as is apparent in Figure 2. This suggests that when the task did not require any particular mental effort, the temperature level did not affect their performance, but when occupants themselves performed mental arithmetic, 359

4 their performance decreased in the warmer environment. The analysis also found a significant learning effect (p < ) from first to second week, although this effect is cancelled out in Figure 2 due to the cross-over study design. No effect of thermal conditions on the performance of the text typing task (speed or error rate) could be observed. 60 Percent of responses (%) Warm Cool 0 Hot - W arm Warm - Slightly Warm Slightly Warm - Neutral Neutral - slightly Cool Slightly Cool - Cool C ool - C old Figure 1. Distribution of occupants thermal sensation votes in the warm and cool conditions. Correct additions per minute With calculator Without calculator 4 Cool Figure 2. Result of the addition task shown separately for occupants who used a calculator (N = 33) and those who did not (N = 44). Warm The intensity of several specific and general SBS symptoms was lower in the cooler than in the warmer environment, although this still needs to be verified statistically. No difference in self-estimated performance was detected. DISCUSSION The study provided valuable experience in the use of web-administered performance tasks in intervention studies in office buildings. Although the method allows for less control than in simulated office environments or in climate chambers and only a limited opportunity of assisting and instructing occupants, it was possible to obtain meaningful responses. Employees motivation to complete the tasks depends to a high degree on management encouragement, as also noted by Zagreus et al. (2004), and on the total duration of the tasks. In intervention studies with repeated observations in the same building, the time required for occupants to complete the tasks should be limited in order to avoid a reduction in occupants motivation to participate in both interventions. In the current study, task duration was modest, but sufficient for previously detected indoor environment effects on performance to be verified. The work rate of the addition task was about 10% lower in the warmer than in the cooler environment among the occupants who completed the task without using a calculator. A comparable result was found in a previous study in a simulated office environment in which subjects who felt warm made 56% more addition errors than did subjects who were thermally comfortable (Witterseh et al. 2004). Witterseh et al., studied a much wider temperature range (22 C 30 C) than in the current study, but even so no effect of temperature on the performance of text typing was observed, as in the present field study. The database on occupants performance will be expanded in future studies, in field studies of factors such as. indoor air quality and air change rate. Effects 360

5 of these factors on performance have previously been documented in studies in simulated office environments (Wargocki et al. 2000). CONCLUSION AND IMPLICATIONS A new tool for assessing the performance of typical office tasks was developed and tested in temperature intervention studies in two office buildings. An effect of temperature on the performance of an addition task was detected. Future studies will continue to investigate indoor environment effects on performance in practice, to field-validate effects predicted from the results of experiments in simulated office environments or in climate chambers. ACKNOWLEDGEMENT This study was supported by the Danish Technical Research Council (STVF) as part of the research programme of the International Centre for Indoor Environment and Energy that was established at the Technical University of Denmark for the period REFERENCES Federspiel CC., Fisk WJ., Price PN., Liu G., Faulkner D., Dibartolomeo DL., Sullivan DP. and Lahiff M Worker performance and ventilation in a call center: analyses of work performance data for registered nurses, Indoor Air, 14(8), Levenshtein VI Binary codes capable of correcting deletions, insertions and reversals Sov. Phys. Dokl., 6, Milton DK., Glencross PM. and Walters MD Risk of sick leave associated with outdoor air supply rate, humidification and occupant complaints, Indoor Air 10, Niemelä R., Hannula M., Rautio S., Reijula K. and Railio J The effect of air temperature on labor productivity in call centers: a case study, Energy & Buildings, 34, Sundell J., Lindvall T. and Stenberg B Associations between type of ventilation and airflow rates in office buildings and the risk of SBS symptoms among occupants, Environment International, 20(2) Tham KW Effects of temperature and outdoor air supply rate on the performance of call center operators in the tropics, Indoor Air, 4(s7), Toftum J., Langkilde G. and Fanger PO "New indoor environment chambers and field experiment offices for research on human comfort, health and productivity", Energy and Buildings, 36(9), Wargocki P., Wyon DP., Baik YK., Clausen G. and Fanger PO Perceived air quality, Sick Building Syndrome (SBS) symptoms and productivity in an office with two different pollution loads, Indoor Air, 9, Wargocki P., Wyon DP., Sundell J., Clausen G. and Fanger PO The effects of outdoor air supply rate in an office on perceived air quality, Sick Building Syndrome (SBS) symptoms and productivity, Indoor Air, 10(4), Witterseh T., Wyon DP. and Clausen G The effects of moderate heat stress and open-plan office noise distraction on SBS symptoms and on the performance of office work, Indoor Air, 14(s8), Zagreus L., Huizenga C., Arens E. and Lehrer D Listening to the occupants: a Web-based indoor environmental quality survey, Indoor Air, 14(s8),