RJ Stratton 1, RJ Stubbs 2, D Hughes 2, N King 3, JE Blundell 3 and M Elia 1

Transcription

1 European Journal of Clinical Nutrition (1998) 52, 737±741 ß 1998 Stockton Press. All rights reserved 0954±3007/98 $ Comparison of the traditional paper visual analogue scale questionnaire with an Apple Newton electronic appetite rating system (EARS) in free living subjects feeding ad libitum RJ Stratton 1, RJ Stubbs 2, D Hughes 2, N King 3, JE Blundell 3 and M Elia 1 1 MRC Dunn Clinical Nutrition Centre, Hills Road, Cambridge, CB2 2DH, England, UK; 2 Rowett Research Institute, Greenburn Road, Aberdeen, AB2 9SB, Scotland, UK; and 3 Psychology Department, University of Leeds, Leeds, LS2 9JT, England, UK Objective: Assessing the value of a newly developed electronic visual analogue scale questionnaire (Apple Newton Message Pad) with the traditional paper method for appetite rating. Design: In a random, crossover design, subjects completed both electronic and paper questionnaires to compare results obtained by the two methods; individual methods were completed consecutively to assess test-retest reliability; preference was established using a questionnaire. Setting/Subjects: Healthy, free-living adults were studied for comparison of methods (n ˆ 12), test-retest reliability (n ˆ 8) and preference (n ˆ 13). Intervention: Visual analogue scales were completed each waking hour to assess appetite. Preference was assessed after both methods were completed. Results: There was no signi cant difference in the hourly results obtained by the paper and electronic methods for `desire to eat', `how much can you eat now', `urge to eat' and `preoccupation with thoughts of food'. Small differences in `hunger' and `fullness' ratings were noted ( 5% mean difference between methods, P < 0.05), but patterns of change and sensitivity for these and all other parameters remained similar for both methods across the visual analogue scale. Test-retest reliability demonstrated was similar for both methods. Seven (54%) subjects preferred to use the paper questionnaire, ve (38%) the electronic method and one (8%) had no preference. Conclusions: The electronic Apple Newton questionnaire is as sensitive and reliable as the paper method, has the advantage that it automatically records the time of data acquisition and data collection and processing are more ef cient for the researcher. The two methods should not be used interchangeably. Sponsorship: Medical Research Council. Descriptors: human; appetite; hunger; visual analogue scale Introduction Visual analogue scales have for many years enabled subjective, quantitative assessment in the clinical setting of matters of human concern which can not be easily and objectively measured, such as pain (Huskisson, 1974; Scott & Huskisson, 1976), quality of life (Priestman & Baum, 1976; Selby et al, 1984) and appetite (Silverstone & Stunkard, 1968; Hill & Blundell, 1982; Robinson et al, 1975). Typically, these scales are 100 mm horizontal lines which represent the continuum of the subjective feeling to be rated. The lines are anchored at the two ends with the extremes of the subjective feeling to be quanti ed. For example, in the assessment of hunger, the visual analogue scale would be marked with `not at all hungry' (0 mm) at one end and `as hungry as I have ever felt' (100 mm) at the other end (Blundell, 1979). The subject marks a line through the scale at a point between the two extremes of the symptom being rated which they consider to indicate the degree of the subjective feeling being rated. Traditionally these scales have been constructed on paper, and completed with a pen. Correspondence: Dr M Elia. Received 24 February 1998; revised 9 June 1998; accepted 11 June 1998 Both in the experimental and clinical setting, visual analogue scales may be completed at set time intervals to identify the degree of the subjective experience being measured (for example, hunger) and its change over time in response to nutritional or clinical manipulations. There are, however, a number of problems inherent in this method of data collection, particularly as the completion of visual analogue scales is typically unsupervised. This may result in questions being missed, or the visual analogue scales incompletely or wrongly marked. Furthermore, there is no guarantee that the visual analogue scales will be completed at the time of day speci ed by the investigator. At the end of the study, the subsequent information obtained by the investigator from the questionnaires may be incomplete and potentially invalid. The handling and processing of data obtained from paper visual analogue scale questionnaires is extensive and time consuming, with the potential for human error. Prior to the study, questionnaires have to be prepared for each subject, and post-study, each visual analogue scale requires measurement by hand, and following this data is inputted into spreadsheets for analysis. To overcome the potential errors and inadequacies of the paper visual analogue questionnaires, we have developed an application (#RJ Stubbs and M Elia) for use on a portable, electronic notepad (Apple Newton Message Pad

2 g, 196 mm mm 6 20 mm), which allows you to create the same visual analogue scale questionnaire as used on paper. The notepad, which can easily be carried by volunteers, has a touch sensitive screen (100 mm 6 76 mm) which is operated by a special pen allowing completion of questionnaires to mimic the paper and pen method. (Figure 1). The application also has additional functionality. There is an integral alarm to prompt the subject to complete the visual analogue scales which can be set by the investigator at a desired time interval, and can be silenced for periods of non-use (namely, overnight). All the entries are date and time stamped and incomplete or incorrect entries and back reference (by subjects) are prevented. Data handling and processing are made easier and quicker as visual analogue scales are automatically measured by the application and the results can then by uploaded from the notepad directly to a computer for analysis. Similarly, questionnaires can be formulated on computer and subsequently downloaded for use on the notepad. A food diary facility enables the entering of food intake data ad hoc. The aims of this study were threefold: to compare the hourly ratings of appetite tracked over time by the new electronic Newton questionnaire with those from the traditional paper version; to examine test-retest reliability for both methods over time; and to assess which of the two methods volunteers preferred. Investigations were carried out in free living, healthy subjects feeding ad libitum. Methods Comparison study: paper vs electronic Twelve healthy volunteers, 7 women and 5 men, were recruited and consented to this study, with a randomised, crossover design. Subjects mean age was 30 y ( 12 y), mean weight 64 kg ( 8.1 kg), mean height 1.70 m ( 0.07 m) and mean body mass index 22.2 kg/m 2 ( 1.7 kg/m 2 ), characteristics presented as mean ( s.d). For this study, volunteers were asked to complete the paper questionnaire (traditionally used in studies of appetite and eating behaviour) and the newly developed electronic version (installed on an Apple Newton Message Pad), immediately one after the other, every waking hour, for two consecutive days (in a randomised order). The questionnaire used for this comparison study consisted of six visual analogue scales to rate `hunger', `fullness', `desire to eat', `how much can you eat now', `urge to eat' and `preoccupation with thoughts of food' (Blundell, 1979; Hill & Blundell, 1982). When completing the electronic version, volunteers were asked to respond to the Apple Newton alarm, (set to alarm every hour to prompt volunteers) by completing the questionnaire using the pen on the Message Pad screen (touch sensitive). Each of the same six questions were presented individually to the volunteer who was not able to progress to the next question until the current visual analogue scale was completed correctly. For the paper version of the questionnaire, volunteers were given a booklet for each day, consisting of one questionnaire for each hour of the day. Each volunteer was allowed the use of a small hourly timer. Test±retest study: paper vs paper and electronic vs electronic As a control group, eight subjects undertook a further two days of study (mean age 33 y ( 11.4), mean weight 62.8 kg ( 7.3 kg), height 1.7 m ( 0.06), body mass index 21.7 kg/m 2 ( 1.82 kg/m 2 )). In random order, on one day they completed two copies of the paper questionnaire (paper ± paper) and on the other, two copies of the electronic Newton (electronic ± electronic) questionnaire. As in the paper ± electronic comparison study, the individual questionnaires were completed immediately one after the other, hourly during waking hours. Preference study Thirteen healthy volunteers (mean age 32 y ( 13 y), mean weight 65.4 kg ( 9.2 kg), mean height 1.70 m ( 0.07 m), mean BMI 22.6 kg/m 2 ( 2.2 kg/m 2 )) were recruited and consented to take part in this study to assess which questionnaire method they preferred: the traditional paper or the electronic notepad version. Volunteers completed the questionnaires individually, every waking hour, for one day each, consecutively. At the end of the study a questionnaire was used to determine volunteers' attitudes to the two methods, and in particular, the questionnaire method they preferred and found easier to use. Figure 1 Apple Newton Message Pad (electronic questionnaire). Data processing All paper visual analogue scales were measured by hand, from left (minimum score of 0 mm) to right (maximum score of 100 mm). This was performed by one investigator

3 and all data subsequently inputted into a computer spreadsheet. The Apple Newton Message Pad automatically measures the visual analogue scales, the results of which are uploaded directly to a computer so data handling and processing is minimal. Volunteers and ethical approval Volunteers were recruited from a variety of settings, and included students, health professionals, teachers, businessmen and unemployed people. Each subject gave written, informed consent and all completed the study, upon which a small fee was paid for their participation. Volunteers were encouraged to continue with their normal daily living throughout the study period. Ethical permission for the study was granted by the Local Research Ethical Committee in Cambridge. Statistical analysis Statistical analysis was carried out using the method of Bland & Altman (1986) to assess the agreement between two methods of measurement for both hourly and mean summary results obtained over the study period. Data are presented as mean score (for the two methods) and bias (mean difference between the two methods). Correlation coef cients were calculated and Student's paired t-test performed on normally distributed data. mean summary results (n ˆ 12) for the whole study period except for fullness ratings (P < 0.05). For all appetite sensations there was a tendency for the difference between methods to become larger with increasing score (r ˆ 0.04 ± 0.18) (Bland & Altman, 1986), but overall the pattern of change and sensitivity of the two methods remained similar. A typical example is illustrated in Figure 2, for which the mean score and bias (s.d.) of the difference between methods is similar to that of the group as a whole. Statistical comparison (Bland & Altman, 1986) of the individual hourly and the mean summary results obtained by paper and electronic Newton questionnaires for each appetite sensation is presented in Table Results Paper ± electronic comparison There was no signi cant difference between the individual hourly results (observations 258 ± 265) by the two methods except for hunger and fullness ratings (P < 0.05), or in the Figure 2 Hunger pro le of one subject completing the paper and the electronic Newton questionnaire for two days (mean score 33, bias 0.97 (s.d. 4.9)). Table 1 Bland & Altman (1986) comparison of results obtained by successive completion of paper and electronic visual analogue scale questionnaires Standard deviation of bias Bias Mean score (Mean difference (Individual (Summary Comparison of visual (for both between hourly ratings for analogue scales methods) methods) ratings) a study period) b Paper ± electronic Hunger ** Fullness * Desire to eat How much can you eat now Urge to eat Preoccupation with thoughts of food Electronic ± electronic Hunger Fullness Desire to eat * How much can you eat now *** Urge to eat Preoccupation with thoughts of food Paper ± paper Hunger Fullness Desire to eat How much can you eat now Urge to eat Preoccupation with thoughts of food Bias signi cantly different from zero for *individual and summary mean data; **for individual data only; and ***mean summary data only (Paired t-test P < 0.05). a Number of paired observations: paper ± electronic n ˆ 258 ± 265 for different appetite sensations, electronic ± electronic n ˆ 63, paper ± paper n ˆ 87. b Number of paired observations: paper ± electronic n ˆ 12, electronic ± electronic n ˆ 8, paper ± paper n ˆ 8.

4 740 Within subjects, mean and bias values were similar to those indicated, but the s.d. of the difference was much smaller (1.1 ± 4.0). Correlation coef cients between electronic and paper questionnaire ratings of each appetite sensation were highly signi cant for the mean summary data for the whole study period (r ˆ 0.94 ± 0.99, SEE ˆ 1.89 ± 2.98, n ˆ 12) and for the individual hourly results (r ˆ 0.91 ± 0.94, SEE ˆ 6.84 ± 8.06, n ˆ 258 ± 265) (P < 0.001). Test ± retest reliability study: paper ± paper and electronic ± electronic There was no signi cant difference in the individual hourly results (paper n ˆ 87, electronic n ˆ 63) or in the mean summary appetite ratings for the whole study period (n ˆ 8) obtained from immediate, successive completion of two paper questionnaires or two electronic questionnaires, with the exception of `desire to eat' and `how much can you eat now' ratings on the electronic Newton questionnaire (Table 1). Despite a signi cant (P < 0.05) bias between the ratings of two electronic questionnaires for `desire to eat' (individual data and mean summary data) and `how much can you eat now' (mean summary data only), the pattern of change over the scale remained similar (`desire to eat' r ˆ 0.08 (n ˆ 63), r ˆ 0.20 (n ˆ 8), `how much can you eat now' r ˆ 0.12 (n ˆ 8)). Correlation coef cients for results obtained from the successive completion of two electronic or two paper questionnaires were highly signi cant (P < 0.001), both for the mean summary ratings for the whole study period (electronic r ˆ 0.96 ± 0.99, SEE ˆ 1.82 ± 3.91, n ˆ 8; paper r ˆ 0.98 ± 0.99, SEE ˆ 0.86 ± 2.11, n ˆ 8) and for the individual hourly ratings (electronic r ˆ 0.89 ± 0.96, SEE ˆ 5.81 ± 8.97, n ˆ 63; paper r ˆ 0.95 ± 0.98, SEE ˆ 4.71 ± 6.43, n ˆ 87). Preference study The preference questionnaire showed that of 13 volunteers, 7 (54%) preferred paper, 5 (38%) preferred electronic, and 1 (8%) had no preference. Of the study sample, all the men preferred the paper whereas 71% (5) of the women preferred the electronic questionnaire. A summary of volunteers' attitudes to the use of the two methods is shown in Table 2. Discussion This study evaluated the use of a newly developed application for use on an Apple Newton Message Pad (electronic appetite rating system, EARS) designed to replicate the traditional paper visual analogue scale questionnaire used in the assessment of appetite (Blundell, 1979; Hill & Blundell, 1982). This is the second electronic method to be introduced and follows the development of a similar application using the same questionnaire on a Psion Organiser (Delargy et al, 1996; King et al, 1997). The Newton, chosen for its likeness to paper and pen, ease of use and small size, has a number of advantages over the traditional paper method. It facilitates more complete collection of data, and in so doing improves data validity whilst also enabling more easy, ef cient and accurate data processing. Although EARS may be the investigators chosen method, from our sample of volunteers, about half favoured the paper questionnaire because their familiarity with paper made it a simpler, easier and faster method to use. Others preferred the paper questionnaire because it enabled back reference and addition to previous answers, (which from the investigators' view point is an undesirable feature of the paper method) and there was concern about losing or damaging an expensive Newton (cost 200). The stated advantages of the electronic questionnaire included the built-in timer, the compact nature of the Newton (complete with `pen') and it being more environmentally friendly than excess paper usage. In another study comparing these two methods (Stubbs et al, 1998), 9 out of 19 (45%) preferred the electronic Newton method, and 15 of them felt that the two methods were similar with respect to ease of use, tediousness, intrusiveness and consumption of time. However, further assessment of preference is essential in other population groups including children and the elderly, those with impaired vision and the sick. Using a similar penbased electronic device for quality of life questionnaires in a study of gastrointestinal patients, Drummond et al (1995) found 57% preferred the electronic to the paper version of the questionnaire. Furthermore, the electronic questionnaire method enabled more complete data collection, whilst the associated data handling and processing were simpler and faster. More important than the comparison of the practicalities of the two methods individually is the similarity of results obtained. There are no objective, de nitive methods of measurement of hunger and other facets of appetite for comparison with visual analogue scale results from either method. However, subjective ratings using paper visual analogue scale questionnaires, particularly within subjects, have generally been shown to be reproducible (Leathwood & Pollet, 1988), reliable (Silverstone & Stunkard, 1968) and sensitive indicators of appetite, following dietary (de Graaf, 1993; Stubbs et al, 1996) and pharmacological Table 2 methods Volunteers questionnaire preference: main advantage and disadvantage of paper and electronic (Newton) questionnaire Preference for paper questionnaire (n ˆ 7) Advantages of paper Disadvantages of electronic easier to use and carry around (n ˆ 4) expense (n ˆ 1) could refer back to check questions (n ˆ 1) unable to refer back and change mistakes (n ˆ 3) familiarity with paper (n ˆ 2) dif culties with technology (n ˆ 2) too heavy and cumbersome to carry (n ˆ 1) Preference for electronic (Newton) questionnaire (n ˆ 5) Advantages of electronic Disadvantages of paper Enjoy technology (n ˆ 1) Not environmentally friendly (n ˆ 1) Timer (n ˆ 2) Hard to complete on time (n ˆ 2) Compact and easy to use (n ˆ 2) Bulky and inconvenient (n ˆ 2) No preference (n ˆ 1) Disadvantages of paper Disadvantages of electronic Not environmentally friendly (n ˆ 1) Dif culty seeing the screen (n ˆ 1)

5 manipulations (Silverstone & Stunkard, 1968; Kyriakedes & Silverstone, 1979). In this study visual analogue scale ratings obtained over time by the paper and electronic Newton systems had a similar sensitivity and pattern of change, and the relationship was strengthened for intersubject comparisons. However, inter-changeable use of paper and the electronic Newton method would not be advisable because of the small but signi cant bias for hunger and fullness ratings between the methods, and the tendency towards a slightly greater discrepancy between the paper and electronic methods with increasing score. Furthermore, test±retest reliability was examined for the two individual methods. Although dif cult to demonstrate due to real differences in subjective experiences with time, test±retest reliability was ascertained for both methods (with the exception of two scales on the Newton) by immediate, successive completion of questionnaires. This study complements that of Stubbs et al (1998) undertaken in healthy volunteers who consumed xed diets under controlled conditions. They have also demonstrated the reliability of the Newton in the temporal tracking of appetite. Comparison of the results obtained by the electronic Newton questionnaire and the paper questionnaire found no signi cant difference, with a strong linear relationship between the two methods (r 2 from 62.6 ± 98.6). Another similar observation between this and the current study included the slightly greater discrepancy between paper and electronic ratings at higher scores, with a tendency to avoid extreme ends of the Newton scales. Conclusions The application (# RJ Stubbs and M Elia) developed for use on the compact Apple Newton Message Pad as an electronic appetite rating system (EARS) has been found to enable reliable, valid collection of data in the study of appetite, producing comparable although not interchangeable results to the traditional paper questionnaire. Advantages of the method have been shown to include more complete, valid data collection and greater ef ciency and ease of data handling and processing for the investigator. Furthermore the application is exible allowing full adaptation of questions used in the questionnaire, an optional timer alarm and a food diary facility (not used in this study). When selecting the questionnaire method of choice, this study suggests the advantages of the electronic method should be considered with the preference of subjects for a particular method. References Bland J & Altman D (1986): Statistical methods for assessing agreement between two methods of clinical measurement. Lancet i, 307 ± 310. Blundell JE (1979): Hunger, appetite and satiety-constructs in search of identities. In Nutrition and Lifestyles ed. M Turner, pp 21 ± 42. London: Applied Science Publishers. de Graaf C (1993): The validity of appetite ratings. Appetite 21, 156 ± 160. Delargy HJ, Lawton CL, Smith FC, King NA & Blundell JE (1996): Electronic appetite rating system (EARS): Validation of continuous automated monitoring of motivation to eat. Int. J. Obes. 20, 104 (Abstract). Drummond HE, Ghosh S, Ferguson A, Brackenridge D & Tiplady B (1995): Electronic quality of life questionnaires: a comparison of penbased electronic questionnaires with conventional paper in a gastrointestinal study. Qual. Life Res. 4, 21 ± 26. Hill AJ & Blundell JE (1982): Nutrients and behaviour: research strategies for the investigation of taste characteristics, food preferences, hunger sensations and eating patterns in man. J. Psychol. Res. 17, 203 ± 212. Huskisson EC (1974): Measurement of pain. Lancet ii, 1127 ± King NA, Lluch A, Stubbs RJ & Blundell, JE (1997): High dose exercise does not increase hunger or energy intake in free living males. Eur. J. Clin. Nutr. 51, 478 ± 483. Kyriakedes M & Silverstone T (1979): A double-blind comparison of fen uramine and dextroamphetamine on feeding behaviour in man. Curr. Med. Res. Opin. 6, 180 ± 187. Leathwood P & Pollet P (1988): Effects of slow release carbohydrates in the form of bean akes on the evolution of hunger and satiety in man. Appetite 10, 1 ± 11. Priestman TJ & Baum M (1976): Evaluation of quality of life in patients receiving treatment for breast cancer. Lancet i, 899 ± 901. Robinson RG, McHugh PR & Folstein MF (1975): Measurement of appetite disturbances in psychiatric disorders. J. Psych. Res. 12, 59 ± 68. Scott J & Huskisson EC (1976): Graphic representation of pain. Pain 2, 175 ± 184. Selby PJ, Chapman JA, Etazadi Amoli J, Dalley D & Boyd NC (1984): The development of a method for assessing the quality of life of cancer patients. Br. J. Cancer 50, 13. Silverstone JT & Stunkard AJ (1968): The anorectic effect of dexamphetamine sulphate. Br. J. Pharm. Chemo. 33, 513 ± 522. Stubbs RJ, Hughes D, Johnstone AM, Rowley E, Reid C, Elia M, Stratton R, Delargy H, King N, Blundell JE (1998): Description and evaluation of a Newton-based electronic appetite rating system (EARS) for temporal tracking of appetite in human subjects. (submitted). Stubbs RJ, Van Wyk MCW, Johnstone AM & Harbron C (1996): Breakfasts high in protein, fat or carbohydrate: effect on within-day appetite and energy balance. Eur. J. Clin. Nutr. 50, 409 ±