Methods of therapeutic trials Pain Measurement for Trials Categorical and visual analogue scales Figure 1: Categorical and Visual Analogue Scales

Transcription

1 Methods of therapeutic trials Pain Measurement for Trials Categorical and visual analogue scales Figure 1: Categorical and Visual Analogue Scales Other tools Restricting to moderate and severe initial pain intensity Out-of-hospital studies Figure 2: Oxford Pain Chart Analysis of pain scale results - summary measures Figure 3: Calculating percentage of maximum possible pain relief score Outcomes other than pain Outputs from trials Figure 4: League Table of number needed to treat (NNT) for postoperative pain Study Design and Validity Placebo Randomised Controlled Trials Double-blinding Trial Designs Figure 5: Parallel and Crossover Trial Designs Parallel Group and Crossover Nof1 Audit Sensitivity Equivalence: A versus B designs Figure 6: Using placebo or active comparators to protect against A versus B negative results Choice of standard analgesic Problems No gold standard When there is no pain to begin with Pain Models Figure 7: Which patient population? Trial Size The observation - success (event) rates vary Figure 8: L'Abbé plot of percentage of patients with at least 50% pain relief with placebo or ibuprofen 400 mg in randomised double-blind trials Implications of event rate variability Implications for interpretation Source of event rate variability Trial design Population Environmental Random effects Figure 9: Percent of maximum pain relief obtained in single-dose randomised double-blind trials in

2 postoperative pain for 826 patients given placebo and 3157 patients given analgesics Investigating variability Figure 10: Two-dimensional L'Abbé plot of the probability density for trials in acute postoperative pain So where does that leave us? Adverse Effects Conclusion References Legends to Figures

3 Methods of therapeutic trials HJ McQuay, DM, Clinical Reader in Pain Relief RA Moore, DSc, Consultant Biochemist Pain Research Nuffield Department of Anaesthetics University of Oxford Oxford Radcliffe Hospital The Churchill Oxford OX3 7LJ UK Tel: Fax:

4 We use clinical trials to show that our analgesic interventions, be they drugs, injections, operations, psychological or physical manoeuvres, or even prayer, are effective and safe. Clinical trials need to produce credible results. To make the results credible it is vital to design, conduct and analyse trials in such a way as to minimise bias. Then we can achieve the credibility which we need. We have to judge the efficacy of analgesic interventions by the change they bring about in the patient's report of pain. A brief description of methods of pain measurement is followed by sections on trial design and pain models. Pain Measurement for Trials Pain is a personal experience which makes it difficult to define and measure. It includes both the sensory input and any modulation by physiological, psychological and environmental factors. Not surprisingly there are no objective measures - there is no way to measure pain directly by sampling blood or urine or by performing neurophysiological tests. Measurement of pain must therefore rely on recording the patient's report. The assumption is often made that because the measurement is subjective it must be of little value. The reality is that if the measurements are done properly, remarkably sensitive and consistent results can be obtained. There are contexts, however, when it is not possible to measure pain at all, or when reports are likely to be unreliable. These include impaired consciousness, young children, psychiatric pathology, severe anxiety, unwillingness to co-operate, and inability to understand the measurements. Such problems are deliberately avoided in trials. Most analgesic studies include measurements of pain intensity and/or pain 2

5 relief, and the commonest tools used are categorical and visual analogue scales. CATEGORICAL AND VISUAL ANALOGUE SCALES Categorical scales use words to describe the magnitude of the pain. They were the earliest pain measure [1]. The patient picks the most appropriate word. Most research groups use four words (none, mild, moderate and severe). Scales to measure pain relief were developed later. The commonest is the five category scale (none, slight, moderate, good or lots, and complete). For analysis numbers are given to the verbal categories (for pain intensity, none=0, mild=1, moderate=2 and severe=3, and for relief none=0, slight=1, moderate=2, good or lots=3 and complete=4). Information from different subjects is then combined to produce means (rarely medians) and measures of dispersion (usually standard errors of means). The validity of converting categories into numerical scores was checked by comparison with concurrent visual analogue scale measurements. Good correlation was found, especially between pain relief scales using cross-modality matching techniques [2, 3, 4]. Results are usually reported as continuous data, mean or median pain relief or intensity. Few studies present results as discrete data, giving the number of participants who report a certain level of pain intensity or relief at any given assessment point. The main advantages of the categorical scales are that they are quick and simple. The small number of descriptors may force the scorer to choose a particular category when none describes the pain satisfactorily. Figure 1: Categorical and Visual Analogue Scales 3

6 Visual analogue scales (VAS), lines with left end labelled "no relief of pain" and right end labelled "complete relief of pain", seem to overcome this limitation. Patients mark the line at the point which corresponds to their pain. The scores are obtained by measuring the distance between the no relief end and the patient's mark, usually in millimetres. The main advantages of VAS are that they are simple and quick to score, avoid imprecise descriptive terms and provide many points from which to choose. More concentration and co-ordination are needed, which can be difficult post-operatively or with neurological disorders. Pain relief scales are perceived as more convenient than pain intensity scales, probably because patients have the same baseline relief (zero) whereas they could start with different baseline intensity. A patient with severe initial pain intensity has more scope to show improvement than one who starts with mild pain. Relief scale results are thus easier to compare across patients. They may also be more sensitive than intensity scales [4, 5]. A theoretical drawback of relief scales is that the patient has to remember what the pain was like to begin with. The evidence we have [6] is that, within limits, the choice of pain measurement scale is not crucial. One point about scales is that we rarely know how much movement on a particular scale equates to a clinically meaningful change. Even using the binary outcome of pain 50% relieved we do not know if this degree of relief is adequate for the patient. Determining the significance of any differences observed is important for patients and for clinicians who will be applying the trial results. 4

7 One example of where scale change and clinical importance were tested comes from McMaster. They used seven point Likert scales measuring dyspnoea, fatigue, and emotional function in patients with chronic heart and lung disease to determine how much change constituted a minimal clinically important difference (MCID). The answer was that the MCID was a mean change in score of approximately 0.5 per item on the seven point scale [7]. We need the same spadework for our pain scales. O THER TOOLS Verbal numerical scales and global subjective efficacy ratings are also used. Verbal numerical scales are regarded as an alternative or complementary to the categorical and VAS scales. Patients give a number to the pain intensity or relief (for pain intensity 0 usually represents no pain and 10 the maximum possible, and for pain relief 0 represents none and 10 complete relief). They are very easy and quick to use, and correlate well with conventional visual analogue scales [8]. Global subjective efficacy ratings, or simply global scales, are designed to measure overall treatment performance. Patients are asked questions like "How effective do you think the treatment was?" and answer using a labelled numerical or a categorical scale. Although these judgements probably include adverse effects they can be the most sensitive discriminant between treatments. One of the oldest scales was the binary question "Is your pain half gone?". Its advantage is that it has a clearer clinical meaning than a 10 mm shift on a VAS. The disadvantage, for the small trial intensive measure pundits at least, is that all the potential intermediate information (1 to 49% or greater than 50%) is discarded. 5

8 Analgesic requirements (including patient-controlled analgesia, PCA), special paediatric scales, and questionnaires like the McGill are also used. The limitation to guard against is that they usually reflect other experiences as well as or instead of pain [9]. PCA in particular is a fraught pain outcome. Individual variation is huge, so that large trial group size is necessary to show any difference. If PCA is used with a pain scale then any difference between trial groups in PCA consumption is only valid at similar pain scale values. Judgement by the patient rather than by the carer is the ideal. Carers overestimate the pain relief compared with the patient's version. RESTRICTING TO MODERATE AND SEVERE INITIAL PAIN INTENSITY The trail blazers of analgesic trial methodology found that if patients had no pain to begin with, it was impossible to assess analgesic efficacy, because there was no pain to relieve. To optimise trial sensitivity a rule developed, which was that only those patients with moderate or severe pain intensity at baseline would be studied. Those with mild or no pain would not. For those using VAS scales we know from individual patient data that if a patient records a baseline VAS pain intensity score in excess of 30 mm they would probably have recorded at least moderate pain on a four point categorical scale [10]. This, the requirement that only patients with moderate or severe baseline pain intensity should be studied, presents particular problems for preemptive techniques and local anaesthetic blocks. With pre-emptive techniques the whole idea is that there is no pain when the intervention is made. The sensitivity of the subsequent measurements, such as time to 6

9 further analgesic requirement, is then of supreme importance. The same applies to local anaesthetic blocks given during surgery, because we cannot be sure that the patient would have had any pain. We know that a proportion of patients (6% after minor orthopaedic operations [11]) have little or no analgesic requirement after surgery. O UT-OF-HOSPITAL STUDIES Reduced length of stay in hospital has forced acute pain investigators to develop methods which work out of hospital. For chronic pain outpatients this has always been necessary. Most investigators use patient diaries, supplemented by telephone calls. We have little empirical information to help choose between particular scales and methods of presentation, just examples of particular trials which proved to be sensitive. Over the years our diaries have become simpler, and an examplar is shown in the Figure. For chronic long-term use we tend to ask patients to complete the diary just before bed, noting their current pain intensity and their typical pain intensity for the day. In such long term studies the average weekly typical pain intensity is a useful outcome. Figure 2: Oxford Pain Chart ANALYSIS OF PAIN SCALE RESULTS - SUMMARY MEASURES In the research context pain is usually assessed before the intervention is made and then on multiple occasions. Ideally the area under the timeanalgesic effect curve for the intensity (sum of pain intensity differences; SPID) or relief (total pain relief; TOTPAR) measures is derived. 7

10 SPID = n PIDt TOTPAR = PRt t=0 6 n t=0 6 Where at the t th assessment point, (t= 0, 1, 2,, n) Pt and PR t are pain intensity and pain relief measured at that point respectively, P0 is pain intensity at t=0 and PID t is the pain intensity difference calculated as (P0- Pt). Figure 3: Calculating percentage of maximum possible pain relief score These summary measures reflect the cumulative response to the intervention. Their disadvantage is that they do not provide information about the onset and peak of the analgesic effect. If onset or peak are important then time to maximum pain relief (or reduction in pain intensity) or time for pain to return to baseline are necessary. OUTCOMES OTHER THAN PAIN Currently there is growing awareness that we should not focus on pain to the exclusion of other outcomes. Mobility, satisfaction and length of stay are important in the acute context, mobility or disability (function) and satisfaction are important in the chronic context. In chronic pain an analgesic intervention which improves pain by as little as 10% may be very important to the patient because this small shift in pain allows an important shift in function. Measured solely as pain reduction this might be missed. We still do not know if 'global' quality of life scales such as the SF36 are adequate to pick up these niceties - one suspects not. 8

11 For function (disability) the researcher often has the choice of off-the-shelf validated scales developed in other clinical contexts or to develop their own. In chronic pain we have found that the small shifts in function which matter to patients are picked up poorly (if at all) by scales developed for advanced cancer. A fruitful approach may be to determine which outcomes matter to patients, for instance by using patient focus groups. Given adequate consensus the output may then be used to fashion a function outcome scale for the trial, with minimal clinically important difference predetermined. This will take time to develop and validate. OUTPUTS FROM TRIALS There are a number of statistical ways to examine results of clinical trials, which include p values, odds ratios, relative risk, relative risk reduction or increase and so on. All may have their place, but they are difficult outputs for the non-specialist to interpret. In order to overcome this, we use the number needed to treat (NNT). The NNT, as the name implies, is an estimate of the number of patients that would need to be given a treatment for one of them to achieve a desired outcome. The NNT should specify the patient group, the intervention, and the outcome. Using postoperative pain as the example, the NNT describes the number of patients who have to be treated with an analgesic intervention for one of them to have at least 50% pain relief over four to six hours who would not have pain relief of that magnitude with placebo. That does not mean that pain relief of a lower intensity will not occur. 9

12 For an analgesic trial, the NNT is calculated very simply as: NNT = 1/ (proportion of patients with at least 50% pain relief with analgesic - proportion of patients with at least 50% pain relief with placebo) Taking a hypothetical example from a randomised trial: 50 patients were given placebo, and 10 of them had more than 50% pain relief over 6 hours. 50 patients were given ibuprofen, and 27 of them had more than 50% pain relief over 6 hours. The NNT is therefore: NNT = 1/(27/50) - (10/50) = 1/ = 1/0.34 = 2.9 The best NNT would, of course, be 1, when every patient with treatment benefited, but no patient given control. Generally NNTs between 2 and 5 are indicative of effective analgesic treatments. Figure 4 shows a league table for single dose postoperative NNTs, using the criterion of at least 50% pain relief over 4-6 hours in patients with moderate to severe pain [12]. For adverse effects, we can calculate a number needed to harm (NNH), in exactly the same way as an NNT. For an NNH, large numbers are obviously better than small numbers. 10

13 Questions have been raised in the past about the wisdom of combining information gathered in analgesic trials using different pain models (dental versus postoperative or episiotomy pain), or different pain measurements, or different durations of observation. Analysis of the great mass of information on aspirin has shown that none of these variables has any effect on the magnitude of the analgesic effect [6]. Figure 4: League Table of number needed to treat (NNT) for postoperative pain Study Design and Validity Pain measurement is one of the oldest and most studied of the subjective measures, and pain scales have been used for over 40 years. Even in the early days of pain measurement there was understanding that the design of studies contributed directly to the validity of the result obtained. Trial designs which lack validity produce information that is at best difficult to use, and which much of the time will be useless, and therefore unethical. PLACEBO People in pain respond to placebo treatment. Some patients given placebo obtain 100% pain relief. The effect is reproducible, and some work has been done to try and assess the characteristics of the "placebo responder", by sex, race and psychological profile. Older women, church-attending but not necessarily God-believing, reputedly are more likely to respond to placebo [13]. 11

14 Two common misconceptions are that a fixed fraction (one third) of the population responds to placebo, and that the extent of the placebo reaction is also a fixed fraction (again about one third of the maximum possible [14]). As Wall points out, these ideas stem from a misreading of Beecher s work of forty years ago [15]. In Beecher s five acute pain studies, 139 patients (31%) of 452 given placebo had 50% or more relief of postoperative pain at two checked intervals [13]. The proportion of patients who had 50% or more relief of pain varied across the studies, ranging from 15% to 53%. There was neither a fixed fraction of responders, nor a fixed extent of response. Placebo responses have also been reported as varying systematically with the efficacy of the active analgesic medicine. Evans pointed out that in seven studies the placebo response was always about 55% of the active treatment, whether that was aspirin or morphine: the stronger the drug, the stronger the placebo response [16], and this observation suggests that significant observer bias (see below) occurs. We believe that the idea that there is a constant relationship between active analgesic and placebo response is an artefact of using an inappropriate statistical description (using a mean when the distribution is not normal) [17]. Gøtzsche has confirmed similar magnitudes of effect for non-steroidal antiinflammatory drugs in active and placebo-controlled studies [18], showing that the presence of a placebo does not affect the active treatment. For many investigators the issue of whether or not to include a placebo group in pain trials causes great angst, personal or institutional. The mechanics of using placebo are important here. A patient is not left to suffer for an indeterminate time. An 'escape' analgesic is given after a set time if the patient has no relief. This interval is usually an hour in oral 12

15 postoperative studies. The patient is also free to withdraw from the trial at any time. This is workable and necessary in circumstances where the variation in response to a pain is large [19]. When placebos are not possible then great care must be taken in the trial design to provide indices of validity and sensitivity. Many times placebo can be incorporated in contexts where it initially seems unthinkable by using an add-on design, with existing medication providing 'cover' and a new drug (or placebo control) being added on. An additional level of sophistication may be achieved by using an 'active' placebo which mimics any adverse effect of the active treatment [20]. RANDOMISED CONTROLLED TRIALS Because the placebo response was an established fact in analgesic studies, randomisation was used from the early 1950s to try to avoid any possibility of bias from placebo responders, and to equalise their numbers in each treatment group. Randomisation is necessary even in studies without placebo, since an excess of placebo responders in an active treatment arm of a study might inflate the effects of an analgesic. The randomised controlled trial (RCT) is the most reliable way to estimate the effect of an intervention. The principle of randomisation is simple. Patients in a randomised trial have the same probability of receiving any of the interventions being compared. Randomisation abolishes selection bias because it prevents investigators influencing who has which intervention. Randomisation also helps to ensure that other factors, such as age or sex distribution, are equivalent for the different treatment groups. Inadequate randomisation, or inadequate concealment of randomisation, lead to exaggeration of therapeutic effect [21]. In broad 13

16 terms methods of randomisation which do not give each patient the same probability of receiving any of the interventions being compared, such as allocation by date of birth, day of the week or hospital number, are bad, whereas tossing a coin, tables of random numbers or the computer variant, which do give the same probability to each patient, are good. An example of the impact of randomisation on the conclusions one draws is the use of transcutaneous electrical nerve stimulation (TENS) in postoperative pain. In a systematic review 17 reports on 786 patients could be regarded unequivocally as RCTs in acute postoperative pain. Fifteen of these 17 RCTs demonstrated no benefit of TENS over placebo. Nineteen reports had pain outcomes but were not randomised controlled trials; in 17 of these 19, TENS was considered by their authors to have had a positive analgesic effect [22]. Stratification, deliberately making sure that patients with factors known or suspected to influence the outcome are equally distributed (randomised) into the trial groups may be incorporated (see [23] for discussion). The randomisation may be organised in blocks, which is helpful where multiple institutions are involved in a study, or where there are multiple observers. Each institution or observer works their way through a particular block(s). DOUBLE-BLINDING Double-blinding means that neither the investigating team nor the patient know which of the interventions under test the patient is actually receiving. Double-blinding is relatively easy to organise for drug trials. With non-drug interventions it may be difficult or impossible. While 14

17 people have struggled to blind TENS or acupuncture it is hard to see how twice-a-day versus once-a-day physiotherapy can be blinded. Does it matter? We know that studies which are not blinded overestimate treatment effects by some 17% [24]. With a subjective outcome like pain the ideal is clearly that the study should be both randomised and doubleblind. If the intervention cannot be blinded then the study should be randomised and open. The study size will in all likelihood have to be increased for the open condition compared with the double-blind. Precisely how much bigger it will need to be will depend on the intervention and on the outcome. TRIAL DESIGNS The two classic clinical trial designs used in pain are the parallel group and the crossover (Figure 5). Max and Laska provide a good review of these classic designs as they apply to single dose trials [25]. Figure 5: Parallel and Crossover Trial Designs Parallel Group and Crossover The advantage of the parallel group is its simplicity. Whereas the crossover design assumes that the underlying pain will not change from treatment period 1 to period 2 no such assumption is required for a parallel group design. A second assumption of the crossover design is that it assumes that there is negligible treatment carryover effect from treatment period 1 to period 2. A relative disadvantage of parallel group compared with crossover is that more patients are needed for parallel group. James et al [26] argued that 2.4 times as many subjects would have 15

18 to be recruited in a non crossover design to obtain precision equivalent to that of the crossover design, given negligible treatment carryover effect. These arguments are dealt with in detail in [27]. Parallel group designs may be used in both acute and chronic pain, but it is unusual to find crossover designs in acute pain, because patients go home much earlier than they did and because postoperative pain wanes, so that there may be a decrease in pain intensity in period 2 compared with period 1. Crossover designs are attractive in chronic pain because fewer patients are necessary, and this is particularly important for study of homogeneous groups with rare syndromes. Nof1 Single patient or Nof1 designs are really crossover designs in single patients [28]. Each patient has multiple 'pairs' of treatments, for instance dextromethorphan and placebo, with the order of the pairs randomised [29]. If five pairs are used some sort of statistical significance can be adduced. If multiple patients are used then the trial can be analysed like a normal crossover design. Examples are trials of amitriptyline in fibromyalgia [30] and of paracetamol in osteoarthritis [31]. In reality while this may be a better way of doing early testing than open studies, and while it may be helpful for single patient therapeutic decisions, it is more onerous and no more informative than conventional crossover designs for 'formal' trials. AUDIT 16

19 Most of us use a form of before-and-after audit to introduce new proven interventions, and hopefully all of us use audit as part of quality control on the care we deliver. Such audits can also help to generate hypotheses, but the problems of case-mix (you treat worse cases than I do) and selection and observer bias mean that audit results are not generalisable in the way that RCT results should be. Their value lies in telling us how well established treatment protocols work, and in control of quality of care [32]. SENSITIVITY Particularly for a new analgesic, a trial should prove its internal sensitivity - that is that the study was an adequate analgesic assay. This can be done in several ways. For instance, if a known analgesic (paracetamol) can be shown to have statistical difference from placebo, then the analgesic assay should be able to distinguish another analgesic of similar effectiveness. Alternatively, two different doses of a standard analgesic (e.g. morphine) could be used - showing the higher dose to be statistically superior to the lower dose again provides confidence that the assay is sensitive. EQUIVALENCE: A VERSUS B DESIGNS Studies of analgesics of an A versus B design are notoriously difficult to interpret. If there is a statistical difference, then that suggests sensitivity. Lack of a significant difference means nothing - there is no way to determine whether there is an analgesic effect which is no different between A and B, or whether the assay lacks the sensitivity to measure a difference that is actually present. 17

20 Figure 6: Using placebo or active comparators to protect against A versus B negative results This is not just a problem for pain studies [33, 19, 23]. Designs which minimise these problems include using placebo or using two doses of a standard analgesic. In the latter case simple calculations could show what dose of the new analgesic was equivalent to the usual dose of the standard analgesic. CHOICE OF STANDARD ANALGESIC For drug trials in acute pain the standard injectable drug is morphine. Current standard oral drugs include paracetamol, ibuprofen and aspirin. This over-simplification conceals problems of opioid versus NSAID, and of the sensitivity of the pain model used. If there is a conflict it is between the pragmatic and the explanatory [34]. The pragmatic is the clinical need to know whether the new drug is better than the standard (or as good with fewer adverse effects). The explanatory is to know whether the intervention works at all. For the pragmatic mode the standard analgesic needs to be current standard treatment or a close relation. For the explanatory mode the control may be placebo (negative control) or active drug (see Figure 5 bottom panel). Clever pain trial designs can combine pragmatic and explanatory. For drug trials in chronic nociceptive pain the same standards apply. In chronic neuropathic pain life is more complicated, because in some pain syndromes there is as yet no gold standard, although tricyclic antidepressants or carbamazepine are achieving that status. 18

21 For non-drug trials in both acute and chronic pain life is more complicated. Again the pragmatic need is to show how the new intervention performs against the current standard treatment. If that means a trial of drug versus non-drug then that is what the randomisation should be, and if blinding is not feasible the trial should be randomised and open. PROBLEMS The correct design of an analgesic trial is situation dependent. In some circumstances very complicated designs have to be used to ensure sensitivity and validity. No gold standard There may be circumstances in chronic pain when there is no established analgesic treatment of sufficient effectiveness to act as the gold standard against which to measure a new treatment. A negative (no difference) equivalence study of one useless treatment against the new treatment would not help, because we would not know if both were good or both were bad. In this context placebo or no-treatment controls may be vital, especially when effects are to be examined over prolonged periods of weeks or months. But paradoxically it is often precisely these circumstances in which ethical constraints act against using placebo or non-treatment controls because of the need to do something. Add-on designs provide one solution. When there is no pain to begin with 19

22 As suggested above when there is no pain it is difficult to measure an analgesic response. Yet a number of studies seek to do this by pre-empting pain, or by intervening when there is no pain (intraoperatively, for instance) to produce analgesia when pain is to be expected. These are difficult, but not impossible, circumstances in which to conduct research. Meticulous attention to trial design is necessary to be able to show differences. A current example of this dilemma is intra-articular morphine [35]. Pain Models The word model here is used as a shorthand for the patient population to be studied. Often there is much agonising over which is the most appropriate population for study. In reality in nociceptive pain a drug which is an analgesic in one population will also be an analgesic in other populations. This is a splitter versus lumper argument. Splitters believe that pain in the foot cannot be managed with a drug which is good for treating pain in the arm. Lumpers hold that a drug which works as an analgesic at one site will work at other sites. We side with the lumpers, and the choice of pain model should be made on the basis of the question you want to answer, again using the explanatory/pragmatic yardstick. If the question is pragmatic, such as which is the best treatment in a particular setting, then there is no point in running the trial in a diametrically opposed population. The evidence we have [6] is that choice of pain model makes no difference to the measured efficacy of an analgesic (but see caveat for opioids below). In acute pain over recent years the removal of lower third molars has proved a sensitive and reliable testbed for investigation of oral analgesics, 20

23 and would be our model of choice for an explanatory trial of an oral analgesic. The splitters do have an argument in this context, because opioids perform slightly less well relative to NSAIDs in oral surgery compared with other models [36]. It is becoming increasingly difficult to test injectable drugs as hospital length of stay shrinks, but injections are still given on the day of surgery to major abdominal and orthopaedic surgery patients. Figure 7: Which patient population? In chronic pain (as usual) life is more complicated. First patients take drugs long term. Most analgesics are proven in acute pain, because trials are easier in acute pain, and the drugs are then used in chronic pain. Single dose trial results by and large do extrapolate to multiple dosing, but single dose trials may underestimate efficacy in multiple dosing, particularly for opioids, and may underestimate the incidence of adverse effects. Second there is the conundrum of neuropathic pain. The problem with neuropathic pain is that putative remedies cannot be tested in the nociceptive pain, which would be much easier. Drugs such as antidepressants and anticonvulsants which have proven efficacy in neuropathic pain have been shown to be ineffective in nociceptive pain. A negative trial result in acute (or chronic) nociceptive pain does not therefore mean that the drug will not work in neuropathic pain. Our remedies for neuropathic pain have to be tested in neuropathic pain. The constraint here are limited numbers of patients in any one centre, and continuing uncertainty about the generalisability of results in one neuropathic pain syndrome to others. Again drugs such as antidepressants and anticonvulsants have proven efficacy in a variety of pain syndromes, 21

24 but systemic local anaesthetics appear to work in some syndromes but not in others [37]. The likelihood is that lumping all chronic neuropathic syndromes together is naive, and increasingly we shall need to subdivide as the years pass. The problem for the current researcher is knowing if a result, positive or negative, in one syndrome is predictive for the others. This may be a context in which the Nof1 designs are useful, because the intervention can be tested on patients with different neuropathic pain syndromes in an explanatory design [29]. Trial Size The variability in patients' response to interventions, seen in both acute pain and chronic pain, may have huge impact on trial results. Many explanations such as trial methods, environment or culture have been proposed, but we believe that the main cause of the variability may be random chance, and that if trials are small their results may be incorrect, simply because of the random play of chance. This is highly relevant to the questions of How large do trials have to be for statistical accuracy? and How large do trials have to be for clinical accuracy?. Words can be confusing. We are talking here about the variability in patients' response to an intervention, whether the intervention is an experimental treatment or control, which could be a placebo. If we decide on some indication of success of the treatment, such as relief of at least 50% of a symptom, then a proportion of patients will achieve success with the experimental treatment, and a proportion of patients will achieve success with the control. We use the phrase experimental event rate (EER) to describe the proportion of patients achieving success (the event) with the experimental treatment, and the phrase control event rate (CER) to 22

25 describe the proportion of patients achieving success (the event) with the control treatment. Clinical efficacy of the intervention is then analysed as the number-needed-to-treat (NNT) [38]. THE OBSERVATION - SUCCESS (EVENT) RATES VARY The medical literature contains many examples of clinical trials which reach different conclusions about how successful an intervention may be, or whether it works at all. In pain research, for instance, one study with tramadol concluded that it was an excellent analgesic [39] and another that it had no analgesic effect at all [40]. The reality is that the proportion of patients who respond to treatment, either with placebo or active therapy, varies, and the extent of that response also varies. Which of the tramadol papers was correct? What follows is concerned with the impact that this variation may have on trial results, about the causes of the variation, and about how the variation can be explained. This variation is seen in both acute pain and chronic pain, and also in other areas of medicine, but here the examples are taken from acute pain. For example, with ibuprofen, there was a huge range in response rates for placebo and ibuprofen 400 mg in randomised, double-blind studies in patients with moderate or severe postoperative pain (Figure 8). In individual trials between 0% and 60% of patients achieved at least 50% pain relief with placebo, and between about 10% and 100% with ibuprofen 400 mg. Figure 8: L'Abbé plot of percentage of patients with at least 50% pain relief with placebo or ibuprofen 400 mg in randomised double-blind trials 23

26 What is going on? Attempts have been made to try to understand or explain this variability [41], especially the variability in control event rate [16, 17] but it is important to recognise that the variability is not unique to pain. Variation in event rates is also seen in trials of anti-emetics in postoperative vomiting [42], of antibiotics in acute cough [43] and in trials of prophylactic natural surfactant for preterm infants the control event rate with placebo for bronchopulmonary dysplasia varied between 24-69% [44]. IMPLICATIONS OF EVENT RATE VARIABILITY The importance of event rate variability is that it undermines the credibility of trial results, particularly results from single trials or from small trials. Implications for interpretation There is a danger that we seize on the latest report of an RCT and, acting on its findings, change our practice. A pain example might be nitroglycerin (NTG) patches for shoulder pain [45]. Randomisation was between a daily 5 mg NTG transdermal patch and an identical placebo applied in the most painful area. At the start, and after 24 and 48 hours, pain intensity was measured on a 10-point scale. At 48 hours, pain intensity was 2 or less (out of 10) in 9/10 patients given NTG compared with 0/10 given placebo. There was no reduction in pain intensity in placebo patients. The NNT for pain intensity of 2 or less for NTG compared with placebo was 1.1 ( ). 24

27 This near perfect result in a small (10 patients per group) randomised trial is so good because 9 of 10 patients did well on treatment and none did well on control (placebo). If some of the controls (three more) had improved and a few of those with NTG (two fewer) did not, then the NTG would look much less impressive, with confidence intervals indicating no benefit of treatment over placebo. Success comes down to the results in a few patients. Knowing that event rates vary, how safe is the published trial result? Could it just be a fluke that in this small trial none of the control patients responded? SOURCE OF EVENT RATE VARIABILITY Trial design One obvious source is trial design. Could there be undiscovered bias despite randomisation and the use of double-blind methods, which if true would undermine the confidence placed in analgesic trial results? Randomisation controls for selection bias, and the double-blind design is there to control observer bias. Patients may know a placebo was one possible treatment, and investigators know the study design and active treatments; it has been suggested that this can modify patients' behaviour in trials [46, 47]. Patients may have opportunities to communicate with each other. Doctors know the trial design when recruiting patients, which may be a source of bias [48]. Nurse observers spend most time with patients, and the nurse might be able to influence a patient s response by his/her demeanour based on experience of other patients reactions. That would produce time-dependent changes in study results as has been seen before [49]. 25

28 Population The reason for large variations in control event rates with placebo may have something to do with the population studied - Scottish stoics versus Welsh wimps. There is little evidence for this, but there may be differences between men and women, or in response in different pain models [36]. Environmental Another explanation is the environmental situation in which a trial is conducted. Inpatients in a nice hospital with a charming nurse might have a good response while outpatients filling in diaries alone at home might not [50]. Other clinical or societal factors which we have yet to recognise may influence event rates. Random effects An individual patient can have no pain relief or 100% pain relief. That is true whether they get control (placebo) or active treatment (Figure 9) [17]. Clearly if we choose only one patient to have placebo and only one patient to have treatment, either or both could pass or fail to reach the dichotomous hurdle of at least 50% pain relief. The more patients who have the treatment or placebo, the more likely we are to have a result which reflects the true underlying distribution. But how many is enough for us to be comfortable that random effects can be ignored? 26

29 Figure 9: Percent of maximum pain relief obtained in singledose randomised double-blind trials in postoperative pain for 826 patients given placebo and 3157 patients given analgesics Until the full effects of the random play of chance are appreciated, we cannot begin to unravel effects of trial design, or population or environmental effects. Here we discuss the effects of random chance in trials of single doses of analgesics in acute pain of moderate or severe intensity. The next section describes the origin of the data used to determine real control event rate (CER) in acute pain trials and the questions to be addressed by the calculations and simulations. INVESTIGATING VARIABILITY The true underlying control event rate (CER) and experimental event rate (EER) were determined from single-dose acute pain analgesic trials in over 5000 patients [51]. Trial group size required to obtain statistically significant and clinically credible (0.95 probability of number-needed-to-treat within ± 0.5 of its true value) results were computed using these values. Ten thousand trials using these CER and EER values were then simulated, using varying trial group sizes, to investigate the variation in observed CER and EER due to random chance alone. Figure 10: Two-dimensional L'Abbé plot of the probability density for trials in acute postoperative pain Most common analgesics have EERs in the range 0.4 to 0.6 and CER of about With such efficacy, to have a 90% chance of obtaining a 27

30 statistically significant result in the correct direction requires group sizes in range 30 to 60. For clinical credibility nearly 500 patients are required in each group. Only with an extremely effective drug (EER > 0.8) will we be reasonably sure of obtaining a clinically credible NNT with commonly used group sizes of around 40 patients per treatment arm. The simulated trials showed substantial variation in CER and EER, with the probability of obtaining the correct values improving as group size increased. SO WHERE DOES THAT LEAVE US? We contend that much of the variability in control and experimental event rates is due to random chance alone. Single small trials are unlikely to be correct. If we want to be sure of getting correct (clinically credible) results in clinical trials we must study more patients than the conventional 40 patients per group. Acute pain trials with 1000 patients are rare, so that credible estimates of clinical efficacy are only likely to come from large trials or from pooling multiple trials of conventional (small) size. Size is everything. The variability in the response rates to both placebo and active treatments means that if we want to be sure of getting the correct (clinically credible) result in clinical trials we must study more patients than the conventional 40 patients per group, a number chosen to be sure (statistically) of not getting the wrong answer. This variability in the response rates to both placebo and active treatments has been recognised before, and was blamed on either flaws in trial design and conduct or on non-specific effects of placebo [16]. We contend that much of this variability is due to random chance alone, and we need not 28

31 search for abstruse causes. This variability is the likely cause of the two discordant reports of tramadol's efficacy [39, 40]. It also justifies clinical conservatism, the caution necessary before taking the results of a single (small) trial into practice. Such a single small trial is unlikely to be correct. A trial with group sizes of forty could have NNT values between 1 and 9 just by chance, when the true value was 3. The variability is not a painspecific problem [42] [43] [44]. Most clinical trials of analgesics are performed to demonstrate statistical superiority over placebo, and are powered to be sure (statistically) of not getting the wrong answer. To achieve this, group sizes of about forty patients are used; 95% of the time this will yield the desired statistical superiority over placebo, given a useful intervention like 400 mg of ibuprofen (Table 1). But to reach a clinically credible estimate of efficacy, defined as a NNT within ±0.5 of the true value, we need ten times as many patients (Table 2). Acute pain trials with 1000 patients do not happen. This means that credible estimates of clinical efficacy are only likely to come by doing such large trials, or from pooling multiple trials of conventional (small) size. Those estimates also need data on 1000 patients to achieve this credibility. Comparing Figure 8 and Figure 10, all the points on Figure 8 fall within the variability predicted due to random chance alone. No other explanation is necessary. Only when we have substantial data should we investigate other possible influences such as pain model [36], population studied and nebulous environmental factors. 29

32 Powering trials for statistical significance is arguably not good enough, because the true size of the clinical effect will still be uncertain. Clinically useful trials also need clinically useful outcomes, as well as trial size big enough to allow us to be confident about effect size. We need to know what degree of improvement on a particular scale matters to the patient [52]. This is quite a challenge to the way we do clinical trials at present, where the focus is on the minimum size necessary for statistical significance. Adverse Effects Clinical trials concentrate on efficacy [53] and adverse effects are reported almost as an afterthought, even though good information was collected. But adverse effects are often the reason why patients stop taking the drug, or cannot tolerate an effective dose. In single-dose analgesic studies adverse effects of any severity are rare, and statistical power is calculated for efficacy and not adverse effects. Multiple-dose studies are more representative of clinical practice, and can yield dose-response relationships for both efficacy and adverse effects [54]. There are some obvious distinctions between the various ways of assessing adverse effects, and some more subtle ones. Perhaps the most important is whether or not a checklist is used. This could be presented verbally or on paper, and of course begs the question of how extensive a checklist. The alternative is a more open question(s), such as Have you had any problem with the drugs?. The open question might result in lower reported adverse effect incidence than the checklist, and verbal lower than paper [55, 56]. The significance of any differences in incidence using the different methods is not clear. These complexities are often 30

33 forgotten. The CONSORT guidelines for the reporting of clinical trials did include adverse effects [57]. Their recommendation was that trialists should define what constituted adverse events and how they were monitored by intervention group. In a systematic review of adverse effect reporting [58] information on adverse effect assessment and reported results was extracted from 52 randomised single dose postoperative trials of paracetamol or ibuprofen compared with placebo. Only two of the 52 trials made no mention of adverse effects. No method of assessment was given in 19 trials, patient diaries were used in 18, spontaneous reporting in seven and direct questioning in six. Clearly the standard of reporting could be improved. Studies which used patient diaries yielded a significantly higher incidence of adverse effects than those which used other forms of assessment. In that review the single dose studies were able to detect a difference between ibuprofen 400 mg and placebo for somnolence/drowsiness with ibuprofen 400 mg (number needed to harm 19 (95% confidence interval 12 to 41)). Nine out of the 10 trials reporting somnolence/drowsiness with ibuprofen 400 mg were in dental pain. Similarly in a review of 72 randomised single-dose trials of postoperative aspirin versus placebo, single-dose aspirin 600/650 mg produced significantly more drowsiness and gastric irritation than placebo, with numbers-needed-to-harm of 28 (19 to 52) and 38 (22 to 174) respectively [6]. The recommendations of the adverse effect review [58] were that reports of trials should provide: details of the type of anaesthetic used (if relevant) 31

34 a description of the format of questions and/or checklists used in the assessment of adverse effects details of how the severity of adverse effects was assessed full details of the type and frequency of adverse effects reported for active drug and for placebo; details of the severity of the reported adverse effects; full details of adverse effect related patient withdrawals; and, where possible, the likely relationship between the adverse effect and the study drug. Conclusion Clinical trials in acute and chronic pain can achieve high levels of precision if they adhere to some simple rules. What current trials cannot give us is an accurate picture of the clinical effectiveness of an analgesic intervention, or a fair representation of the harm that may be caused. These both need much larger numbers of patients, and in future trial design may have to change to take this into account. 32