Patient questionnaires in primary health care. Validation of items used in asthma care

Transcription

1 International Journal for Quality in Health Care 2000; Volume 12, Number 1: pp Patient questionnaires in primary health care. Validation of items used in asthma care MALOU LINDBERG 1, JOHAN AHLNER 2, TOMMY EKSTRÖM 3 AND MARGARETA MÖLLER 4 1 Division of General Practice and Primary Care, 2 Division of Clinical Pharmacology, 3 Department of Pulmonary Medicine and 4 Department of Neuroscience and Locomotion, Department of Medicine and Care, Faculty of Health Sciences, Linköpings University, Sweden Abstract Objective. To evaluate each item in a patient questionnaire for the purpose of investigating whether the validity of each item is acceptable. Design. The questionnaire was completed by the patients at an ordinary follow-up visit for their asthma, and within 1 week a nurse interviewed them by telephone with the aim of analysing the validity of each item through the use of predetermined criteria. Settings. Patients with asthma in primary health care settings in Sweden. Study participants. Fifty-one patients were consecutively included from three different primary health care units. Results. Nine of 13 items had an acceptable validity. The four items that were not found to have acceptable validity were developed further. Conclusion. Evaluating each item in a questionnaire by means of interviews with the specific patient population is a useful method of assuring that the intention of the patient questionnaire has been met. Keywords: asthma, interview, quality improvement, questionnaire, validation different participating primary health care units were also carried out. The questionnaire was designed through teamwork by representatives of the different professions involved in asthma care in the county. An expert panel then discussed whether the questionnaire appeared to assess the desired qualities. The goal was for each item in the questionnaire to include important aspects in accordance with international guidelines [6,7]. The questionnaire was to reflect important factors in asthma management, such as the monitoring of peak ex- piratory flow and symptoms, regular asthma check-ups by health care professionals, and the patient s own knowledge about their disease, their medications and prevention. Several questionnaires which assess the patient s own ex- periences of asthma evaluate quality of life [10,11]. These quality of life questionnaires can also be used indirectly in quality assurance in asthma care, but some of them are extensive and take a long time to answer, and are therefore The symptoms of asthma include recurrent episodes of wheezing, chest tightness, shortness of breath and coughing [1]. In Sweden, the prevalence of asthma is estimated at 5 8% and the disease affects at least people [2 4]. Various international guidelines are available regarding the management of asthma [5]. These guidelines describe what is required to attain good quality in asthma care [6,7]. It is commonly the case that management/treatment is not optimal with respect to the guidelines, and a multidimensional approach in evaluating the quality of asthma care would be of great value [8]. In the county of Östergötland, Sweden, a previous study was carried out to investigate the quality of health care received by asthmatics at each primary health care unit in the county [9]. In that study a patient questionnaire was used to survey the quality of care by means of combining the patient questionnaires with administrative data from each primary health care unit (Figure 1). Comparisons among the Address reprint requests to M. Lindberg, Department of Research and Development in Primary Health Care, SE Mjölby, Sweden. Malou.Lindberg@lio.se 2000 International Society for Quality in Health Care and Oxford University Press 19

2 M. Lindberg et al. Figure 1 Patient questionnaire and agreement level results. not always applicable in the clinical situation. The purpose Patients gave their informed consent to participate in the of the questionnaire described above was to create a simple study. After the consultation with the physician they answered questionnaire that could easily be incorporated as part of the questionnaire, For the patient with asthma (Figure 1). recurrent follow-ups of the quality of asthma care in health After the questionnaire had been filled in a member of care services. The aim was for the questionnaire to conform the staff forwarded it to the Department of Research and to the standards of international goals for asthma care [6,7]. Development in Primary Health Care and reported the The aim of the present study was to evaluate the validity patient s name and telephone number by telephone to an of each individual item in the questionnaire to ascertain asthma nurse. The asthma nurse then telephoned the patient whether answers to the different items in the questionnaire within 1 week to conduct a telephone interview. Each incorrespond to what was intended by the professionals. terview took about 30 minutes and had both open and closed alternative answers. One nurse, who was trained in this interview method, performed all of the interviews (Table 1). Method The intention was to follow up each item in the questionnaire in the interview form. The interviewer regarded only episodes Asthmatics from three different primary health care units that had occurred during the week prior to the visit. Before were consecutively included in the study. Patients with the start of the study the interview form was tested in a pilot diagnosis asthma, diagnosis code 493 (ICD 9), >17, who study. were at an ordinary follow-up visit, were included. Patients Criteria for having understood each item in the ques- receiving care for acute asthma exacerbations were not included tionnaire in a correct way were stipulated in advance by the in the study. professionals. Guided by the answers given in the telephone 20

3 Patient questionnaires in primary health care Table 1 The questionnaire, the telephone interview questions and the predetermined criteria Item... Question Predetermined criteria Knowledge/technique Item 1 Do you have your own PEF-instrument? See footnote Have you been instructed in how to make a PEF- Knows what a PEF diagram is diagram? What do you measure with a PEF-instrument? Lung function. Peak Expiratory Flow How often do you measure PEF? Twice a week or when I get worse Do you know your normal PEF-value? States the PEF-value in litres/minute Do you know your alarm PEF-value? States the PEF-value in litres/minute Item 3 How long have you taken medication via inhalation? See footnote Have you received instructions on how to inhale? Item 5 What do you know about asthma prevention? Mentions actions such as taking prophylactic medication, cleaning up the environment, etc. What should you avoid being exposed to? Mentions allergens or irritants Compliance Item 2 How often do you take your asthma medication? A yes answer in the questionnaire must be confirmed by stating that medication is taken daily Care and treatment Item 4 Can you describe your written plan of action? Written orders on a special form or a medication list Item 7 What is your doctor s name? States the name of the doctor Item 8 How often do you go for an asthma check-up? Do you automatically get follow-up appointments? See footnote Symptoms Item 9 Have you visited a health care facility directly, See footnote without an appointment, in the last 6 months? Item 11 Can you describe your typical asthma attack? Did you use a B2 agonist last week because of See footnote shortness of breath? Item 12 Last week were you: A yes answer in the questionnaire must be awakened due to shortness of breath? confirmed by a yes for one of the four awakened due to coughing? symptoms awakened due to wheezing? Do you suffer from insomnia because of your asthma? Item 13 Did you have to limit your physical activity last week See footnote due to your asthma? Could you exercise the way you wanted last week See footnote without shortness of breath? Life style Item 10 Are you a current smoker? See footnote If so, how much do you smoke each day? Information/knowledge Item 6 Have you received information about the disease? See footnote What happens in your airways when you get shortness of breath? Airway constriction and inflammation 1 To satisfy the predetermined criteria, a yes answer to an item in the questionnaire had to be confirmed in the interview, as did additional information as specified. PEF, Peak expiratory flow. interviews, the asthma nurse made a subjective judgement as to whether the answers in the questionnaires were in accordance with the predetermined criteria (Table 1). For each question, the proportion of answers that were similar on both the questionnaire and the interview was calculated. This proportion will henceforth be referred to as the agreement level. The agreement level for each item in the questionnaire was calculated for the whole study group. Prior to its start the study was approved by the ethics committee. 21

4 M. Lindberg et al. Statistical analysis action plan as being the prescription itself or the information included on the packaging of the med- Our interest focused on finding items with a poor agreement ication or inside the packaging. level. For each item, we tested the null hypothesis π =0.85 (ii) Have you received information about asthma preversus the alternative hypothesis π < 0.85, where π is the vention? population agreement level, and 0.85 was chosen arbitrarily. Exact binomial probabilities were calculated, and P-values Agreement level of 66% P< < 0.05 were considered significant. With this sample size, this means that H Fifteen patients who answered yes in the ques- 0 will be rejected for items with an observed agreement level Ζ 75%. No Bonferroni adjustment was tionnaire stated in the telephone interview that they calculated because in this case it is more important to keep did not know anything about asthma prevention. a low probability of a type-i error than to reduce the Another two patients did not reach the predetermined probability of a type-ii error. The sample size was determined criteria concerning asthma prevention. in order to get 80% power if π =0.70. (iii) Did you have more than two asthma attacks last week? Agreement level of 56% P< Results Twenty-two patients who had answered no in the Sample questionnaire said in the phone interview that they had used 3 30 doses of a short-acting inhaled B2- All 51 patients invited to take part in the study agreed to agonist during the last week due to dyspnoea. participate, and filled in the questionnaire. The study group (iv) Were you bothered by symptoms during the night consisted of 32 females and 19 males, mean age 48 years last week? (range, years). Fifty patients who completed the study were on regular inhalation therapy, with a mean usage time Agreement level of 74% P< 0.05 of 11 years (range, 1 35 years). Three of the patients were Twelve patients who had answered no in the quesusing medication that did not include steroids. The remaining tionnaire stated in the phone interview that: 47 patients had an inhaled steroid prescription with a mean they had awakened with shortness of breath due diurnal-dose of 750 μg (range, μg), and three of to asthma (n=3) the patients also had daily oral steroid prescribed. Another they had been bothered by coughing due to asthma three patients had oral steroid for use during periods of (n=10) exacerbation. Forty-nine patients had a prescription for short- they had been bothered by wheezing due to asthma acting inhaled B2-agonist, and 11 patients had a prescription (n=2) for long-acting inhaled B2-agonist. they had been bothered by interrupted sleep due One-fifth of the patients reported that they did not take to asthma (n=4). their medication as prescribed, and one-third reported that they had had an acute consultation with a doctor during the (In some cases the same patient had more than one symptom.) last 6 months because of their asthma. Two-thirds reported asthma-related symptoms during the last week and two-thirds had used a mean of 11 doses (range, 1 30 doses) of short-acting Discussion inhaled B2-agonist during the last week because of dyspnoea. There was one drop-out from the telephone interview part This study focuses on the difficulty in constructing simple of the study because it was not possible to contact the person questions aimed at a specific disease population. A question within the 7-day limit. The telephone interview was carried that is addressed to a specific patient group and that seems out a mean of 2.7 days (range, 0 7 days) after the questionnaire to be expressed clearly is not always understood as intended had been answered. by the professionals who formulated it. It is important to Of a total of 13 items, nine showed an acceptable agreement evaluate the items in a questionnaire in order to ensure that level of >75%, and the highest agreement level was 98%. the questions measure what they were intended to measure. See figure 1. In the comparison between the questionnaire Despite the fact that an expert panel had discussed both and the interview, four items showed a low agreement level the face validity and the content validity of the questionnaire of Ζ 75%. The following items had a low agreement level: before its use in an earlier study [9], our study shows a low (i) Do you have a written plan of action? agreement level in four of 13 items in the questionnaire. This points out the necessity of continuing the evaluation Agreement level of 64% P< procedure in order to improve the validity of a method. When the expert panel discussed the content validity before Nine patients who had answered yes in the questionnaire using the questionnaire, their aim was to determine whether said in the phone interview that they did there were any irrelevant items or whether any important areas not have any written plan of action. Another nine had been missed. That is considered a minimum prerequisite patients said that they had understood the written before starting to administer a questionnaire [12]. 22

5 Patient questionnaires in primary health care In the present study the patients in each unit were included Have you received information about asthma prevention? consecutively by using the diagnosis code for asthma (ICD To these questions 18 and 17 patients, respectively, answered 9). Other studies have shown that patients with cardio- in the affirmative, but in spite of this their answers in the pulmonary or asthma-like conditions such as chronic ob- interviews did not meet the stipulated criteria. Additional structive pulmonary disease or functional breathing disorder questions that would lead to more precise answers are, Can have received the diagnosis of asthma (ICD 9) [13,14]. Taking you describe your written plan of action? and What do you that into consideration, some of the patients might not be know about asthma prevention? Development of the above true asthma patients. However, the primary care units that four questions with a low agreement level was done with the took part in this study had their own spirometers and a help of knowledge obtained from the interviews. special asthma practice, and they also had access to and Our intention with the evaluation procedure was to create knowledge about the local asthma consensus report, which a valid, uncomplicated patient questionnaire aimed at reis in accord with international consensus. This could indicate flecting a part of the quality of care for an asthma population that the patients in this study received the correct diagnosis in primary health care. Several other patient questionnaires [15 17]. have been constructed for the purpose of improving the The items with low agreement levels were those that management of asthma. Some of them focus on asthma concerned symptoms due to the disease and items that knowledge, quality of life, or are used in epidemiological reflected the patient s own knowledge of the disease and studies of asthma [10,20,21]; however, none has been designed treatment. Symptom items such as an item in which a patient for the purpose of studying the quality of asthma care in a with a long history of asthma is asked if he/she has been simple way, and their usefulness in clinical practice can be bothered by nocturnal symptoms are not sensitive enough. questioned. An asthmatic patient has probably adjusted to the symptoms A short, validated questionnaire would make it possible and they have become a part of their life situation. The answers for the general practitioner to evaluate the asthma care he/ to the questions consequently showed an underestimation of she provides in comparison with the quality of the care given the symptoms. The need for more specific symptom questions by others, in the same or in other primary health care units. becomes apparent through the interviews. Based on this Another application would be to evaluate interventions. This information, the question about nocturnal symptoms has patient questionnaire could be used as a pre- and postbeen expanded to include the following four questions: measure method to study whether an intervention affects (i) Were you awakened last week due to shortness of the quality outcomes in asthma care. Used together with breath? administrative data, it also would be a valuable method for (ii) Were you awakened last week due to coughing? evaluating whether the intentions stipulated in international (iii) Were you awakened last week due to wheezing? guidelines for asthma management are met in a specific (iv) Do you suffer from insomnia because of your asthma population. asthma? The possibility of conducting multidimensional quality assurance in asthma care will also give us basic data for Nocturnal worsening of asthma is a significant problem making decisions about how asthma care should be organized in the management of asthma. It has been shown to be in order to receive priority. This would confirm our ambition an important symptom in relation to the severity of the to use evidence-based management in asthma care, which is disease, and it is also reflected in mortality statistics [18]. important as public health care has limited resources. It is therefore of great importance to pinpoint those patients who suffer from nocturnal symptoms for therapeutic consideration [19]. The item in the questionnaire that was intended to register Conclusion asthma attacks also had a low agreement level, as the patients under-rated their asthma attacks. The interview made it clear There is an obvious risk that questions in patient questhat to the patient, an asthma attack meant a more serious, tionnaires will not be understood as intended by the proacute situation, resulting in a visit to the emergency ward, fessionals who formulated them. Therefore it is important than the professionals intended when they constructed the that the validity of a patient questionnaire is determined not item. For this reason, another question should follow this only by the judgement of an expert panel. Evaluating each one namely, Did you use a B2-agonist last week due to item in a questionnaire by means of interviews with the shortness of breath? The aim was to be able to assess the specific patient population is one method of assuring that frequency of the patient s asthma attacks. There were patients the intention of the patient questionnaire has been met. who denied having had asthma attacks. Nevertheless, they had used 3 30 doses of a B2-agonist during the last week due to dyspnoea. None of the patients who reported asthma attacks seemed to exaggerate. Acknowledgements The other two items that showed a low agreement level dealt with the patient s own knowledge of the disease. These We thank Olle Eriksson, Department of Mathematics, Uni- questions were, Do you have a written plan of action? and versity of Linköping, Sweden, for statistical advice. 23

6 M. Lindberg et al. References 12. Streiner DL, Norman GR. Health Measurements Scales. A Practical Guide to Their Development and Use, 2nd edn. Oxford: Oxford Medical Publications, Dantzker D, Pingleton S, Pierce J et al. Standards for the Diagnosis and Care of Patients with Chronic Obstructive 13. Martinez FJ. Diagnosing chronic obstructive pulmonary disease. Pulmonary Disease (COPD) and Asthma. Am Thorac Soc, 1986; The importance of differentiating asthma, emphysema, and chronic bronchitis. Postgrad Med 1998; 103(4): , , Åberg N, Lindberg U. Allergic diseases in Swedish school children. Acta paediatr Scand 1988; 77: Åberg N. Asthma and allergic rhinitis in Swedish conscripts. Clin Allergy 1989; 19: Norrman E. High prevalence of asthma and related symptoms in teenagers in Northern Sweden. Eur Respir J 1993; 6: Richard A. National and international guidelines for the diagnosis and treatment of asthma. Curr Opin Pulmon Med 1997; 3: International consensus report on diagnosis and treatment of asthma. Bethesda: National Heart, Lung, and Blood Institute, National Institute of Health. Publication no , March Eur Respir J 1992; 5: Guidelines for the Diagnosis and Management of Asthma. Bethesda: National Asthma Education and Prevention Program. National Institutes of Health publication, Liza C, Reynold A. Assessing Quality of Care in Asthma Management. Drug Benefit Trends 1998; 10(8): Lindberg M, Ekström T, Möller M. Quality Assurance of the care of Bronchial Asthma patients treated at the Primary Health Care Level in Östergötland (in Swedish). Report from the Department of Research and Development in Primary Health Care 1996: p Hyland ME. The Living with Asthma Questionnaire. Respiratory Med 1991; 85: Ringsberg K. Patients with asthma-like symptoms but negative asthma test and patients with bronchial asthma. Physiological, psychological and social characteristics. Linköping University, Medical Dissertations. No.522, Adelroth E, Hargreave FE, Ramsdale EH. Do physicians need objective measurements to diagnose asthma. Am Rev Respir Dis 1986; 134: Joyce D, Chapman KR, Kesten S. Prior diagnosis and treatment of patients with normal methacholine challenge and unexplained respiratory symptoms. Chest 1996; 109: Ringsberg K, Löwhagen O, Sivik T. Psychological differences between asthmatics and patients suffering from an asthma-like condition, functional breathing disorder: a comparison between the two groups concerning personality, psychosocial and somatic parameters. Integr Physiol Behav Sci 1993; 28: Robertson CE, Rubinfeld AR, Bowej G. Deaths from asthma in Viktoria: a 12-month study. MedJAust1990; 152: Martin RJ, Kraft M. Nocturnal asthma: therapeutic considerations. Clin Immunother 1996; 6: Fitzclarence CAB, Henry RL. Validation of an asthma knowledge questionnaire. J Paedistr Child Health 1990; 26: Asher MI, Keil U, Anderson HR et al. International study of asthma and allergies in childhood (ISAAC): rationale and methods. Eur Respir J 1995; 8: Marks GB, Dunn SM, Woolcock AJ. A scale for the measurement of quality of life in adults with asthma. J Clin Epidemiol 1992; 45: Accepted for publication 2 November