Poor adherence is at least partially responsible for

Transcription

1 Accuracy of Three Electronic Monitors for Metered-Dose Inhalers* Steven M. Julius, MD; James M. Sherman, MD; and Leslie Hendeles, PharmD Background: Prior studies indicate that some devices used to monitor metered-dose inhaler (MDI) use are not accurate. Objective: To assess the accuracy of the Doser CT (NEW-MED Corporation; Waltham, MA), the MDILog (Medtrac Technologies; Lakewood, CO), and the SmartMist (Aradigm Corporation; Hayward, CA) in a bench-top study. Design: One, two, and four puffs of fluticasone propionate MDI (Flovent; GlaxoWellcome; Research Triangle Park, NC) were actuated twice daily for 30 days with two units of each device. The date and time of each actuation were recorded in a log and then compared with the output of the device. The percentage of doses recorded accurately was compared by analysis of variance. Measurements and results: The SmartMist was 100% accurate, while both the Doser CT and MDILog devices occasionally recorded spurious actuations. The Doser CT also had missed actuations after the counter had prematurely reached zero secondary to the spurious recordings. The accuracy (mean SD) was % for the Doser CT and % for the MDILog (p 0.21). The dose regimen actuated and duration of use did not significantly affect accuracy. Conclusion: All three devices are sufficiently accurate to monitor adherence in most clinical settings. (CHEST 2002; 121: ) Key words: adherence; device; metered-dose inhaler Abbreviations: LCD liquid crystal display; MDI metered-dose inhaler *From the Pediatric Pulmonary Division (Drs. Julius, Sherman, and Hendeles) and the College of Pharmacy (Dr. Hendeles), University of Florida, Gainesville, FL. This study was conducted without industry support. The manufacturers supplied at no charge the test devices, and GlaxoWellcome provided the Flovent canisters. Manuscript received March 5, 2001; revision accepted September 24, Correspondence to: Steven M. Julius, MD, Georgia Pediatric Pulmonary Associates, 1100 Lake Hearn Dr, Suite 450, Atlanta, GA 30342; StevenJulius@aol.com Poor adherence is at least partially responsible for asthma-related morbidity 1,2 and mortality. 3,4 It poses an added financial burden in clinical circumstances and could affect the results of efficacy and safety studies on asthma therapy. Most medications prescribed for the treatment of asthma, for maintenance or rescue, are administered via a metereddose inhaler (MDI). Therefore, monitoring adherence with a MDI is often necessary to differentiate poor adherence from other causes of treatment failure. 5 When evaluating adherence with controller medications, patients and/or parents tend to overestimate usage, 2,6 10 presumably secondary to recall bias or as an effort to avoid criticism. Prescription refill histories and canister weighing are more objective measures, but they do not reflect pattern of usage. Prior studies 8 10 comparing adherence by electronic monitors to canister weight have shown that many subjects were inaccurately characterized as adherent. Nevertheless, a pharmacist s dispensing record that reveals infrequent refilling of medication is strongly supportive of poor adherence. 5 Several studies 7 11 utilizing electronic monitors have identified multiple simultaneous activations, or a dumping phenomenon, defined as a cluster of actuations in a short time period typically preceding an evaluation and/or after a period of underuse presumably to make oneself appear adherent before the clinician. Therefore, electronic devices provide a more realistic representation of patient usage, unless the patient is actively engaged in an effort to deceive the monitor. Monitoring devices have an added benefit in that they may increase adherence in those users informed of the capabilities of the device, especially when accompanied with feedback. 8,9,12 Studies 8 10,12 16 have shown that electronic monitors often malfunction. Both the loss of data and the recording of nonexistent doses were documented. This study assessed the reliability of the SmartMist (Aradigm Corporation; Hayward, CA), as well as two devices currently available in the United States, the CHEST / 121 / 3/ MARCH,

2 Doser CT (NEW-MED Corporation; Waltham, MA) and the MDILog (Medtrac Technologies; Lakewood, CO). The SmartMist is a larger device that accommodates the MDI and actuator with only the mouthpiece exposed (Fig 1). It records the date and time of every actuation. A plunger actuates the MDI after a specified volume is inspired (250 to 500 ml) and at a specific inspiratory flow rate (25 to 60 L/min) in order to minimize technique error. It provides immediate guidance on technique with a flashing red light when inspiration is too rapid ( 60 L/min), a solid green light when the inspiratory flow is appropriate (25 to 60 L/min), or no light indicating insufficient inspiratory flow ( 25 L/min). Information can be downloaded with the accompanying cable onto a personal computer. The data can be viewed and printed as a list of events or a series of graphs. The Doser CT is a small device with a plastic sleeve that allows it to be placed easily on the end of the canister (Fig 1). When pressed, an electromechanical switch completes a circuit, recording the actuation. The Doser CT has two counters. One counter is preset to the total number of actuations per canister and subtracts one with each actuation. The second counter displays the total number of actuations per day and resets at midnight, storing that value in memory. A history consisting of the total number of actuations per day for the prior 45 days can be displayed by scrolling. The device does not record the date or time of each actuation, nor does it have the capability to download the data onto a computer for review. The MDILog is a small device that is permanently attached to the actuator. The actuator must be sent to the manufacturer for installation. It has a switch that extends through the actuator to contact the canister during an actuation (Fig 1). It records a shake as well as the date and time of every actuation. A thermistor identifies an inspiratory effort that is recorded as no inhale (no inhalation noted within the first 6 to 8 s after an actuation), inhalation (inhalation noted within 0.7 s after an actuation) or late inhale (inhalation noted 0.7 s but 6to8s after an actuation.) The information can be downloaded with the accompanying docking station onto a personal computer. The data can be viewed and printed as a list of events or a series of graphs. The purpose of this study was to assess the Figure 1. The Doser CT device can be easily placed on the top of most canisters using the plastic sleeve. The LCD screen displays two counters: one counts down from the total number of actuations remaining, and the other counts the total number of actuations per day. The MDILog fits on the actuator with a switch that extends through a hole to contact the canister. The device communicates with docking station via infrared transmissions. The SmartMist canister with actuator fits into the device. An internal plunger actuates the MDI when volume and flow rate requirements are met. Small lights on the side prompt the user for optimal technique. 872 Laboratory and Animal Investigations

3 accuracy and hence the reliability of these three devices. Comparisons by device type were performed. The impact of dosing schedule (one puff, two puffs, or four puffs) and period (first 15 days vs second 15 days) were investigated to assess the origin of reporting errors. Materials and Methods Six units each of the SmartMist, MDILog, and Doser CT devices were evaluated concurrently. A MDI containing 110 g/puff of fluticasone propionate (Flovent; GlaxoWellcome; Research Triangle Park, NC) was appropriately paired with each device. The MDILog and SmartMist were connected via a series port to a personal computer and initialized according to the instructions of the manufacturer. The MDILog was calibrated during the first initialization and assignment according to the suggestions of the manufacturer. Each MDI was actuated once, twice, or four times during a 15-min session based on the schedule for that particular MDI. Sessions were repeated twice daily on 30 nonconsecutive days during a 45-day period, between 7 am and noon, and between noon and midnight. Consecutive actuations for the same MDI were at least 1 min apart. Canisters were shaken before each actuation. The mouthpiece of each MDI actuator within the SmartMist was connected to a vacuum that maintained a constant flow rate of 30 L/min to activate the MDI. The date, time, and description of all activities were immediately recorded in a written logbook after each actuation, which were preformed by the same investigator. Two units of each device were actuated once, twice, or four times, twice daily, for a total of 60, 120, or 240 times, respectively, during the study period. Because each canister contains approximately 120 actuations, devices at a schedule of four puffs required a new canister midway through the study. The Doser CT and the MDILog were reprogrammed after the canister change, so that 120 actuations were remaining in their respective memories. Both values displayed by the Doser CT were recorded in the logbook after each actuation. Since this device does not record time of actuation, these values were compared to the activity recorded by the investigator to assess the validity. Also, the Doser CT has a feature that allows the user to press a button and display the total number of actuations per day for the previous 45 days. This function was activated after 15 days when canisters were changed as well as at the end of the study. Results at the end of each 15-day period were compared to the logbook entries. Data from the MDILog and the SmartMist were downloaded and then compared to data recorded in the logbook by the investigator to determine accuracy. Data Analysis and Statistical Methods Accuracy was defined for the Doser CT as an agreement between the device display and the logbook immediately after each actuation. For the MDILog and SmartMist, accuracy was defined as a recorded date and time that was within 15 min of administration according to the logbook. Any actuation recorded in the logbook and not by the device (missed actuation) or any actuation recorded by the device and not by the logbook (additional actuation) was considered inaccurate. In addition, multiple-dose errors displayed on the liquid crystal display (LCD) screen of the MDILog during operation were compared to the information downloaded. According to the specifications of the manufacturer, multiple-dose errors occur when an actuation is within 6 to8softheprevious actuation. However, in this study, consecutive actuations were at least 60 s apart. Therefore, reports of multiple-dose errors were counted as inaccurate doses. At the study completion, the percentage of doses recorded accurately was determined by the criteria outlined above. An analysis of variance was used to evaluate the accuracy of the doses recorded. Fixed variables were device (Doser CT, MDILog, or SmartMist), dosing schedule (one puff, two puffs, or four puffs), and period (first 15 days vs second 15 days). The significance level was selected as All analyses were performed on the percentage of doses recorded accurately using software (SAS version 6.12; SAS Institute; Cary, NC). For several measures, the device (SmartMist) and period (Doser CT during the first 15 days only) were 100% accurate. In cases where no variability was observed, analysis of variance could not be performed and descriptive statistics were therefore used. When variability was present, the results are expressed as a mean SD. SmartMist Results All six SmartMist devices were 100% accurate (Table 1). They recorded every actuation within the specified time, without spurious extra or missed doses. The devices never failed to actuate the MDIs when the vacuum was applied. Doser CT All six Doser CT units recorded additional actuations. They were noted immediately following a scheduled actuation and never occurred spontaneously. The display of the number of actuations remaining in the canister on the devices scheduled at Table 1 Percentage of Doses Recorded Accurately for Each Device as a Function of Schedule and Observation Period Percentage of Doses Recorded Accurately SmartMist Doser CT MDILog Schedule (Twice a Day) First 15 Days Second 15 Days Total First 15 Days Second 15 Days Total First 15 Days Second 15 Days Total One puff Two puffs Four puffs Total CHEST / 121 / 3/ MARCH,

4 two puffs and four puffs, but not the one puff regimen prematurely reached zero. Prior to reaching zero, the Doser CT units recorded all actuations. By design, the Doser CT does not record actuations after the counter that reflects the total number of actuations remaining has reached zero. Although secondary to additional actuations, these missed doses were considered inaccurate recordings. The overall accuracy of the Doser CT was %. There was no statistically significant difference between the three dosing schedules (p 0.63). There was a trend toward decreasing accuracy when the mean of the second 15 days ( %) was compared to the mean of the first 15 days (100%). This trend was due to the recording of additional actuations, which first appeared during the second half of the study, as well as missed doses after the counter had prematurely reached zero. Prior to the counter reaching zero, the number of actuations at the end of activity of each day was in complete agreement with the total number of actuations per day when reviewed with the retrospective, scroll feature midway and at the end of the study. MDILog The MDILog recorded every actuation within the specified time. Occasionally, it recorded more doses than were actuated. All extra doses were noted immediately following a scheduled actuation in the form of a multiple-dose error and never occurred spontaneously. According to manufacturer specifications, a multiple-dose error is recorded when two actuations occur within 6 to8sofoneanother. Since each actuation was separated by at least 1 min, all multiple-dose errors were considered inaccurate. The overall accuracy of the MDILog was %. There was no statistically significant difference between the three dosing schedules (p 0.90) as well as between the first half of study when compared to the second half (p 0.90). However, there was a trend toward decreasing accuracy when the second 15 days ( %) were compared to the first 15 days ( %). Of note, the spurious multiple-dose messages on the LCD display were consistent with the information retrieved from the devices after they were downloaded. Comparison of the Doser CT to the MDILog There was no statistically significant difference between the Doser CT and the MDILog for overall accuracy (p 0.21). A comparison of the mean difference in percentage of actuations recorded accurately between the first half and the second half of the study was not significant (p 0.09), but there is the possibility of a type II statistical error ( 0.60). Comparison of the Doser CT and the MDILog to the SmartMist Since all six SmartMist devices were 100% accurate without variability, a statistical comparison to the other devices could not be made. Discussion This study tested the ability of each device to accurately record actuations. It was not the purpose of this study to determine if either the MDILog or the SmartMist detect an inspiratory effort. However, application of inspiratory flow was necessary to trigger an actuation when using the SmartMist. Therefore, we can only comment on the reliability of each individual device at recording an actuation. Based on our study, the SmartMist was most accurate, since all six devices recorded all actuations within the specified time without extra recordings. The Doser CT and MDILog were slightly less accurate because they occasionally recorded additional actuations. Schedule and timing (first half vs second half of study) had no significant impact on accuracy, yet there was a trend toward decreasing accuracy for both the Doser CT and the MDILog. One possible explanation is that battery decay contributed to dosing errors as was seen in earlier studies. 14,15 The Doser CT devices had the greatest decrement in accuracy during the second half of the study compared to the other two device types. This is in part due to the fact that the Doser CT, by design, no longer records actuations after the preset counter reaches zero. Therefore, additional actuations led to the premature arrival of this counter at zero and subsequent inability to record further doses. All devices and medication used were provided by their respective manufacturers and without obligation. There is no way to ensure that devices were not pretested by the manufacturers prior to shipment. Ideally, devices should be randomly selected to avoid this confounding factor. This is the first report on the accuracy of the MDILog. Its predecessors, the Nebulizer Chronolog model NC-100 and the Nebulizer Chronolog model NC-300 (Medtrac Technologies) utilized a microswitch to record actuations. Both models provided objective data of MDI usage; however, many of the devices malfunctioned, resulting in a loss of data. 8 10,12,13 In one such study, approximately 50% of the devices had malfunctioned at one point, most commonly from battery drainage. 13 Nevertheless, testing of these devices illustrated that diary or self-report overestimate usage, 2,6 10 even after nonadherent patients were excluded from the study. 2 Had adherence been determined by a change in 874 Laboratory and Animal Investigations

5 canister weight, many of these subjects would be categorized as adherent. In addition, several studies identified multiple simultaneous activations, or a dumping phenomenon preceding an evaluation and/or after a period of underuse The model that followed, Nebulizer Chronolog model MC-311 (Medtrac Technologies), was equipped with a thermistor to detect actuations. Once again, several devices malfunctioned resulting in loss of data Brueckner et al 14 and later Wamboldt et al 15 measured battery voltage and determined that loss of data was associated with battery drain. Also, some devices had spurious data on initialization and others recorded frequent clusters of spurious actuations described as seizure-like activity. 15 The MDILog units also malfunctioned, but there was no loss of data. Simmons et al 16 studied the Doser during three experiments at five centers and compared them to the Nebulizer Chronolog model MC-311 during two of the three experiments. 16 Overall, Doser devices were in agreement with actual usage (determined by change in weight) 87.8% of the time. In our study, the Doser CT was in agreement with the diary 94.3% of the time. While the errors that occurred in their study were mostly due to underreporting by the devices, the errors we observed were all secondary to the spurious recording of extra doses. As in our study, the overall accuracy of the Doser and MDILog devices were not significantly different. Of note, 2 of 101 Doser units and 9 of 48 Nebulizer Chronolog units failed to record data. 16 There have been no published articles to date that assess the accuracy of the SmartMist. All errors occurred immediately after a scheduled actuation and were displayed on LCD as either an artificially high number of actuations (Doser CT) or as a multiple-dose error message (MDILog). Therefore, we suspect that the errors we observed would be brought to the attention of the clinician by the patient. The Doser also missed doses after the counter had reached zero. Since the Doser recorded all actuations prior to reaching zero, we speculate that had the counter been set to account for the additional actuations that were recorded, these later actuations would have been recorded. In the context of maintenance therapy, a device that overestimates usage may be limited to defining the maximum possible number of actuations administered. If such a device indicates that the patient has received less than the prescribed number of actuations, it is a clear indication that the patient has decreased adherence. Devices that record additional spurious actuations would be less useful when used for monitoring adherence with rescue inhalers. It is reasonable to suggest that all inhalers be equipped with a simple counter to allow the patient to monitor usage. All three devices are designed to monitor adherence, yet they are all very different. This study did not consider factors such as cost, ease of use (device and software), and other features. The Doser is considerably less expensive then the MDILog or the SmartMist but lacks many of the advanced features. We had no technical problems with the Doser CT or the SmartMist. We had considerable difficulty downloading the software that accompanied the MDILog onto several computers. Two MDILog devices did not report the data in chronological order. With one MDILog device, data could not be retrieved through the methods outlined in the product handbook but were successfully downloaded by alternative methods. Two months after completion of the primary study, several MDILog devices began to report a low-battery voltage warning. In conclusion, when testing the ability of each device to record an actuation, the MDILog and the Doser CT are sufficiently reliable. The SmartMist is not only reliable but may also be indicated for clinical research or any other scenario where 100% reliability is essential. ACKNOWLEDGMENT: The authors thank Doug Theraque and Alan Hutson from the Department of Biostatistics for assistance with data analysis, and Kathy Rice for preparation of the article. References 1 Birkhead G, Attaway NJ, Strunk RC, et al. Investigation of a cluster of deaths of adolescents from asthma: evidence implicating inadequate treatment and poor patient adherence with medications. J Allergy Clin Immunol 1989; 84: Milgrom H, Bender B, Ackerson L, et al. Noncompliance and treatment failure in children with asthma. J Allergy Clin Immunol 1996; 96: Patterson R, Greenberger PA, Patterson DR. Potentially fatal asthma: the problem of noncompliance. Ann Allergy 1991; 67: Rea HH, Scragg R, Jackson R, et al. A case-control study of deaths from asthma. Thorax 1986; 41: Sherman JM, Hendeles L. Improving adherence to asthma medications. Contemp Pediatr 1999; 16: Spector SL, Kinsman R, Mawhinney H, et al. Compliance of patients with asthma with an experimental aerosolized medication: implications for controlled clinical trials. J Allergy Clin Immunol 1986; 77: Coutts JA, Gibson NA, Paton JY. Measuring compliance with inhaled medication in asthma. Arch Dis Child 1992; 67: Nides MA, Tashkin DP, Simmons MS, et al. Improving inhaler adherence in a clinical trial through the use of the Nebulizer Chronolog. Chest 1993; 104: Tashkin DP, Rand CS, Nides MA, et al. Nebulizer Chronolog to monitor compliance with inhaler use. Am J Med 1991; 91:33S 36S CHEST / 121 / 3/ MARCH,

6 10 Rand CS, Wise RA, Nides MS, et al. Metered-dose inhaler adherence in a clinical trial. Am Rev Respir Dis 1992; 146: Mawhinney G, Spector SL, Kinsman RA, et al. Compliance in clinical trials of two nonbronchodilator, antiasthma medications. Ann Allergy Asthma Immunol 1991; 66: Simmons MS, Nides MA, Rand CS, et al. Trends in compliance with bronchodilator inhaler use between follow-up visits in a clinical trial. Chest 1996; 109: Gong H, Simmons MS, Clark VA, et al. Metered-dose inhaler usage in subjects with asthma: comparison of Nebulizer Chronolog and daily diary recordings. J Allergy Clin Immunol 1988; 82: Brueckner JW, Marshik P, Sherman J, et al. Reliability of the Medtrac MDI Chronolog. J Allergy Clin Immunol 1997; 100: Wamboldt FS, Bender BG, O Connor SL, et al. Reliability of the model MC-311 MDI Chronolog. J Allergy Clin Immunol 1999; 104: Simmons MS, Nides MA, Kleerup EC, et al. Validation of the Doser CT, a new device for monitoring metered-dose inhaler use. J Allergy Clin Immunol 1998; 102: Laboratory and Animal Investigations