The architecture of diagnostic studies is built

Transcription

1 The Architecture of Diagnostic Research: From Bench to Bedside Research Guidelines Using Liver Stiffness as an Example Agostino Colli, 1,2* Mirella Fraquelli, 1,3* Giovanni Casazza, 1,4 Dario Conte, 1,3 Dimitrinka Nikolova, 1 Piergiorgio Duca, 4 Kristian Thorlund, 1,5 and Christian Gluud 1 The diagnostic research process can be divided into five phases, designed to establish the clinical utility of a new diagnostic test the index test. The aim of the present review is to illustrate the study designs that are appropriate for each diagnostic phase, using clinical examples regarding liver fibrosis diagnosed with transient elastography, when possible. Phase 0 is the preclinical pilot phase during which the validity, reliability, and reproducibility of the index test are assessed in healthy and diseased people. Phase I is designed to describe the distribution of the index test results in healthy people and its normal values. Phase IIA comprises studies designed to estimate the accuracy (sensitivity and specificity) of the index test in discriminating between diseased and nondiseased people in a clinically relevant population. Phase IIB studies allow the comparison of the accuracy of different index tests; Phase IIC studies aim to evaluate the possible harms of incorporating the index test in a diagnostic-therapeutic strategy. In phase III, diagnostic test-therapeutic randomized clinical trials aim to assess the benefits and harms of the new diagnostic-therapeutic strategy versus the present strategy. Phase IV comprises large surveillance cohort studies that aim to assess the effectiveness of the new diagnostic-therapeutic strategy in clinical practice. Conclusion: As common in clinical research, giving excessive weight to the results of single studies and trials is likely to divert from the totality of evidence obtained through the systematic reviews of these studies, conducted with rigorous methodology and statistical methods. (HEPATOLOGY 2014;60: ) The architecture of diagnostic studies is built upon studies conducted to answer different questions, having different designs and predefined inclusion criteria. 1-4 The research questions follow a logical order for establishing the clinical value of a diagnostic test, analogous to that of the clinical trial phases in interventional research (e.g., for drugs and medical devices). A diagnostic test under assessment is called the index test, and a target disease is the pathological condition that the index test aims to identify. A reference standard is a test or procedure assumed to best identify the true state (diseased or nondiseased) of the patient. The early phases of the diagnostic studies aim to assess the accuracy of the index test as well as its sensitivity and specificity, whereas the later phases aim to assess the effect of the information, obtained from testing, on patient-centered outcomes. In the present article, we describe the phases of the architecture of diagnostic research (Fig. 1; Table 1) in greater detail than previously described. 1-4 Each and every of the five study phases is outlined and Abbreviations: AEs, adverse events; ALT, alanine aminotransferase; BMI, body mass index; CT, computed tomography; DCP, des-gamma carboxy-prothrombin; HCC, hepatocellular carcinoma; RCT, randomized, clinical trial; SD, standard deviation; TE, transient elastography; US, ultrasonography. From the 1 The Cochrane Hepato-Biliary Group, Copenhagen Trial Unit, Center for Clinical Intervention Research, Department 7812, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark; 2 Department of Internal Medicine, Ospedale A. Manzoni, Lecco, Italy; 3 Second Division of Gastroenterology, IRCCS Fondazione Ca Granda Ospedale Maggiore Policlinico, Universita degli Studi di Milano, Milan, Italy; 4 Department of Biomedical and Clinical Sciences, L. Sacco, Universita degli Studi di Milano, Milan, Italy; 5 Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada. Received July 1, 2013; accepted November 21, *These authors contributed equally to this article. 408

2 HEPATOLOGY, Vol. 60, No. 1, 2014 COLLI, FRAQUELLI, ET AL. 409 Fig. 1. Flow chart of the phases that make up the architecture of diagnostic research. accompanied by relevant clinical examples, mostly from the diagnosis of liver fibrosis. Guidance on the calculation of the sample size for each of the study phases are made available as supplementary contents and summarized in Table 2. Phase 0: The Preclinical Phase The aim of phase 0 study is to establish the properties of an index test in terms of its (1) validity (i.e., the extent to which a test measures what it is intended to measure) and (2) reliability (i.e., the consistency of any variation of the test measurements with the true variations). It is usually referred as a) repeatability (i.e., the ability to reproduce the same results in identical settings (same device, same operator) when testing the same patient and b) reproducibility (i.e., the ability to obtain the same results when testing the same patient under changing conditions (different operators or devices). In order to meet the aims, phase 0 studies use animals and small numbers of healthy or diseased persons as participants. However, this information is seldom found extensively covered in specific phase 0 study reports. Such information is more often quoted within reports of Address reprint requests to: Mirella Fraquelli, M.D., Ph.D., Second Division of Gastroenterology, Fondazione IRCCS Ca Granda-Ospedale Maggiore Policlinico, Universita degli Studi di Milano, Via Francesco Sforza 35, Milan, Italy. mfraquelli@yahoo.it; fax: Copyright VC 2014 by the American Association for the Study of Liver Diseases. View this article online at wileyonlinelibrary.com. DOI /hep Potential conflict of interest: Nothing to report. Additional Supporting Information may be found in the online version of this article.

3 410 COLLI, FRAQUELLI, ET AL. HEPATOLOGY, July 2014 Table 1. Phases of the Architecture of Diagnostic Research Phase Question Design Participants Comment 0 What is the validity, reliability, and reproducibility of the index test? I What is the range of the index test results in healthy people and the influence of sex, age, BMI, and so on, on the results? II Preliminary Question Do test results in affected patients differ from those in healthy individuals? Preliminary Question Are the patients with the target disease more likely to have a certain test result? Preliminary Question Are patients with a more advanced stage of the target disease more likely to have abnormal index test values than those with less severe target disease? Preliminary Question Do the test results in patients with only the target disease differ from those inpatients with the target disease plus concurrent diseases? Preliminary Question Does a disease different from the target disease produce abnormal test results? Phase IIA Is the index test able to detect the target condition among patients suspected of having the target disease? Phase IIB Which of two or more tests is more accurate? Phase IIC What is the proportion of false results when comparing the current diagnostic evaluation with that incorporating the index test? III Do the tested patients fare better than comparable patients tested by a standard method or not tested? 1. Do the patients that test positive with test A and receive treatment fare better than those patients that test positive with test B and receive the same treatment? 2. Do the patients with positive test A fare better than the patients with positive test B, given that the same treatment strategy is employed for the two tests (including the Phantom studies, animal studies, and, often, small series Preclinical studies and those with healthy participants and patients Often quoted within the report of subsequent phases studies Case (healthy) series Healthy individuals Aim: to describe the distribution of the test result in a healthy population Patient control Patients: individuals affected by the target disease; healthy controls Patient control Patients: individuals with the target disease; controls: individuals not surely affected by the target disease, but suspected of having the target disease Patient control Patients with different stages of the target disease Patient control Patients affected by both the target and concurrent disease; controls affected only by the target disease Patient control Patient with a disease (different from the target disease); controls without this disease Cross-sectional Consecutive patients likely harboring the target disease Cross-sectional or RCT Consecutive patients likely harboring the target disease RCT Consecutive patients likely harboring the target disease RCT with different design according to questions 1, 2, and 3 Consecutive patients likely harboring the target disease No more healthy controls Aim: to estimate the index test accuracy (sensitivity, specificity) In the case of cross- sectional design, the test results are compared in the same patients; in the case of RCTs, indirect comparison in two group of patients made comparable by randomization From the estimate of accuracy to assessment of the downstream consequences of false test results Before introducing the index test in clinical practice

4 HEPATOLOGY, Vol. 60, No. 1, 2014 COLLI, FRAQUELLI, ET AL. 411 TABLE 1. Continued Phase Question Design Participants Comment After introducing the index test in clinical practice RCT Consecutive patients likely harboring the target disease reassessment of negative results with possible treatment in case of subsequent positivity)? 3. Do the patients that receive test A as the subsequent test fare better than those who receive test B as the subsequent test? IV Are benefits and harms of the index test-treatment versus the reference test-treatment strategy confirmed after the index test has been introduced into clinical practice? Cohort Large cohort of consecutive patients likely harboring the target disease and tested with the index test Are the diagnostic test/therapeutic results in large cohort studies in accordance with the results from randomized clinical trials? Cohort Large cohort of consecutive patients likely harboring the target disease and tested with the index test What is the proportion of patients with rare AEs resulting from the diagnostic test-therapeutic strategy? Cohort Cohort of consecutive patients with false results of the index test (discordant results of the index test and of the reference standard) In the case of discordance between the index test results and those for the reference standard, what are the right results? studies of the subsequent phases. It may also happen that phase 0 studies make their appearance many years after a test is introduced into clinical practice: for example, the competitive-binding luminescence assay, a commercialtestthatiscentralforestablishingthecobalamin level in the diagnosis of vitamin B 12 deficiency. This test has been replacing microbiologic and radioisotopedilution assays for more than a decade, but its validity has only recently been assessed. 5 Example. A new test, transient elastography (TE) (FibroScan; Echosens, Paris, France), is a noninvasive technique that was introduced to indirectly assess hepatic fibrosis by measuring liver stiffness. Pertinent phase 0 studies have defined the intrinsic properties of TE. 6,7 Phase I: The Clinical Reference Interval Phase I studies aim to assess the distribution of the results of the candidate index test in healthy people and define the index test normal values, that is, to find out the range of the test results in healthy people and examine the influence of factors, such as sex, age, body mass index (BMI), and others, on the results. When test results are expressed on a continuous scale, the features of their statistical distribution (symmetric or asymmetric) and some typical values (mean and standard deviation [SD], minimum and maximum values, quartiles and percentiles) have to be defined. In case of symmetric distribution, the reference values fall in the interval of mean SDs for healthy people. In case of asymmetric distribution, the reference values fall between the 2.5th and 97.5th percentile of the distribution for healthy individuals. If we consider the 97.5th percentile as the upper limit of the reference values, then 2.5% healthy people should, by definition, have abnormal values and should be tested further to assess whether they are true false positive or not. In fact, the positivity of the index test may reflect the presence of the target disease or other unsuspected (at that time) diseases. When test results are expressed as dichotomous outcomes, the aim of phase I studies is to estimate the proportion of positive results in healthy people. A very low falsepositive fraction should, ideally, be detected. Example. Using TE, liver stiffness was assessed in healthy volunteers, blood donors, and general population; the distribution of reference values was found to be very similar in all three groups. Furthermore, liver stiffness values were not influenced by age and were positively related to male sex, increased BMI, fatty liver, and metabolic syndrome variables. 8,9

5 412 COLLI, FRAQUELLI, ET AL. HEPATOLOGY, July 2014 Table 2. Methods for the Calculation of the Sample Size According to the Different Phases of Diagnostic Studies Phase Comparison Aim Method I One sample estimate Sample large enough to show that the observed false-positive fraction is less than a prespecified false-positive proportion in healthy people with a 95% confidence level II (preliminary questions) difference between the means and proportions is detected IIA One sample estimate Sample large enough to obtain an estimate of sensitivity (or specificity) with desired precision. IIB difference between the accuracies (sensi- tivity, specificity) is detected IIC III-1 III-2 III-3 IV (randomized trials) IV (observational studies: IV-1) IV (observational studies: IV-2) IV (observational studies: IV-3) Equivalence of a proportion with a known value One sample estimate difference between the proportions of complications in false-negative (positive) patients is detected difference between the proportions of patients with the outcome who are test positive is detected difference between the proportions of events (outcome) is detected difference between the proportions of events (outcome) is detected difference between the proportion of patients with outcome events or adverse events is detected Sample large enough to show that the efficacy of the new test is equivalent to that reported in antecedent studies Sample large enough to show that the observed proportion of patients with rare AEs is less than a prespecified value with a 95% confidence level difference between the proportions of diseased patients among the discordant results is detected Hanley s formula (rule of three)* Usual methods for comparison of means and proportions Sample size derived from anticipated width of confidence intervals Usual methods for comparison of proportions in paired or independent samples Usual methods for comparison of proportions; the additional element to be considered is the incidence of complications in patients with negative (positive) test results. Usual methods for comparison of proportions; the additional element that must be considered is the efficacy of the standard treatment. Usual methods for comparison of proportions; two additional elements must be considered: the proportion of patients with negative test results with subsequent positive reassessment result and the efficacy of the standard treatment. Usual methods for comparison of proportions; the additional element that must be taken into account is the efficacy of the available treatment strategies. Usual methods for comparison of proportions; two additional elements must be considered: the harms and the benefits of the treatment and the expected proportion of complications in patients with the target disease. Methods for equivalence tests for one proportion Hanley s formula (rule of three)* Usual methods for comparison of proportions in paired samples; additional elements that must be anticipated are the proportion of patients with discordant results and the prevalence of target disease in one of the of discordant results. *Source: Eypasch E, Lefering R, Kum CK, Troidl H. Probability of adverse events that have not yet occurred: a statistical reminder. BMJ 1995;311: Source: Obuchowski NA. Sample size calculations in studies of test accuracy. Stat Methods Med Res 1998;7: Source: Altman DG. Practical Statistics for Medical Research. London: Chapman & Hall; Source: Chow SC, Wang H, Shao J. Sample Size Calculations in Clinical Research. New York: Marcel Dekker; Phase II: The Diagnostic Accuracy Phase II studies aim to assess the diagnostic accuracy of the index test. Diagnostic accuracy is a measurement of the agreement between the index test and the reference standard in discriminating diseased from nondiseased participants. The pertinent question is: is the index test able to distinguish between a person with and a person without the target disease among those suspected of having the target disease? The extent to which an index test is able to do so is estimated by the common measures of diagnostic accuracy, such as sensitivity and specificity, positive and negative likelihood ratios, diagnostic odds ratio, and receiver operating characteristic curves. The discriminatory capacity of the test is evaluated in a population where it is clinically sensible to suspect the target disease. Therefore, the design and context of phase II studies are close to actual clinical practice, where tests are used to exclude or confirm a diagnostic hypothesis.

6 HEPATOLOGY, Vol. 60, No. 1, 2014 COLLI, FRAQUELLI, ET AL. 413 Phase II studies may differ in terms of the way patients are selected. Their study design can be crosssectional, nested patient control, or patient control (previously called case control). In cross-sectional studies, participants likely to harbor the target disease are consecutively enrolled without knowing the results of the reference standard test. The participants should simultaneously ( cross-sectionally ) undergo the index test and the reference standard test; the time lapse between these two tests must be as short as possible, particularly in the case of diseases with a fluctuating course. The order does not matter. In the classical design, participants undergo the index test first followed by the reference standard, whereas in a reversedflow study, the index test follows the reference standard. In both cases, the index test and the reference standard test must be blindly assessed with regard to the results of the other test. In a nested patient-control design study, all (or a random sample of) the patients affected by the target disease, from a cohort of participants being suspected of having the target disease, are compared to a random sample of participants from the same cohort of participants who do not have the disease. In the case of a target disease with a very low prevalence and/or an invasive or very costly index test, this selection modality allows to assess the sensitivity of the index test in all affected patients, sparing the amount of index test to assess specificity, without biasing the diagnostic estimates. On the contrary, in the patient-control studies, the patients have to undergo the reference standard before the index test. In fact, people known to be affected by the target disease ( patients ) and people who are not affected ( controls ; i.e., healthy people or people with other diseases than the one we want to discriminate) are selected according to the results of the reference standard. Patient-control studies may be affected by spectrum bias which produces an overestimation of the index test accuracy. This is present when the participants do not represent the real spectrum of the severity of the target disease as well as alternative conditions, particularly if the sickest of the sick ones are compared to the healthiest of the healthy people. Cross-sectional studies on consecutive series of participants are not flawed by spectrum bias. Among the enrolled participants, there should only be participants with symptoms suggesting the target disease, but there should be no patients with full-blown severe manifestations of the target disease and no healthy controls. Hence, index test accuracy is assessed in participants in whom it is sensible to suspect the target disease, which leads to more realistic estimates. However, because the included series of participants might differ from one another, a spectrum variation is possible and highly related to the different settings (e.g., primary or secondary care referrals). When designing and reporting studies on diagnostic test accuracy, one should follow the standards described by the STARD initiative. 10 Once two or more diagnostic test accuracy studies are carried out, they need to be systematically reviewed according to The Cochrane Collaboration recommendations. 11 In this process, the need to evaluate the risks for systematic errors (bias), random errors (play of chance), and extent of applicability of the individual studies arises. QUADAS-2 is a recommended tool for assessment of the individual studies. 12 Examples. Properly designed and conducted studies measured liver stiffness in a series of consecutive patients with a wide spectrum of chronic liver disease severity. All participants underwent both TE and liver biopsy with histological staging. The proportions of false-positive or -negative results were low in diagnosing significant fibrosis and cirrhosis. 9,13 These results were confirmed by a meta-analysis. 14 Besides the evaluation of the diagnostic accuracy of the index test, phase II studies can address other preliminary questions as well. The question, do test results in affected patients differ from those obtained in healthy individuals, relates not only to the distribution of test results among diseased and healthy participants, but also to the degree of their overlap. The lower the degree of overlap, the more accurate the test is considered to be. The interindividual true variability within well-designed, well-conducted phase 0 studies can provide an explanation for the variability of test results in phase II studies. The usual design for such a phase II study is one in which affected and healthy participants are sampled and the obtained test value is measured once (patient-control design). These studies are relatively simple and can readily advise on whether a more profound assessment of the test accuracy should or should not be performed. Another preliminary question is: do test results in patients with target disease differ from those obtained in patients not affected by the target disease, but suspected of having it? In the diagnostic studies designed to answer this question, patients, that is the participants with positive reference standard test results, are compared to participants with some clinical suspect of having the target disease, but with negative results of the reference standard test. Hence, controls are no more healthy individuals. Example. This question is exemplified by a patient-control study on des-gamma carboxy-prothrombin (DCP) for the diagnosis of hepatocellular

7 414 COLLI, FRAQUELLI, ET AL. HEPATOLOGY, July 2014 carcinoma (HCC). 15 Fifty-five patients with histologically proven HCC in liver cirrhosis (patients) and 53 patients with histologically proven liver cirrhosis (controls) were included. The mean serum concentration of DCP and the proportion of patients with a raised DCP value above the 125-mAU/mL cutoff were higher in patients with HCC (target disease). Another question is: are patients with a more advanced stage of the target disease more likely to have abnormal index test values than patients with a less severe target disease? To answer this question, a study has to include patients with different stages of the target disease. The study design is, again, the patient-control study (selection of groups of patients with a known stage of the disease). Patient-control studies will establish whether the index test can discriminate between the affected and nonaffected patients, but they are not yet intended to estimate the true sensitivity and specificity in a clinical context. In fact, the estimates can be flawed by spectrum bias resulting from the inclusion of participants on the basis of the results of the reference standard test. After establishing the actual relationship between the test results and the target disease (and its different stages), the assessment of the specificity of such a relationship is needed. The questions are: does any disease different from the target disease produce abnormal test results, and do the test results in patients with only the target disease differ from those in patients with the target disease plus concurrent diseases? Examples. Different studies answered the first question showing high liver stiffness values in other conditions, such as cholestasis or heart failure with increased hepatic venous pressure. 16 These high values are comparable to those obtained in a patient with severe liver fibrosis (the target disease). For the second question, a pertinent study enrolled 10 patients with chronic hepatitis B and hepatitis exacerbations: Liver stiffness values paralleled those of alanine aminotransferase (ALT), increasing 1.3- to 3-fold during ALT flares, 17 showing that the results of the index test are influenced by the presence of acute hepatitis with inflammation and necrosis (concurrent disease) on chronic hepatitis with fibrosis (the target disease). Phase IIB. Phase IIB studies aim to compare the accuracy of two or more different index tests. The question is, Which of two or more tests is more or most accurate? The answer is possible with two different study designs. The first is the cross-sectional study including consecutive participants who are likely to harbor the target disease and who undergo the index tests as well as the reference standard test. Thus, a direct comparison of sensitivity and specificity between the different index tests is possible. The second is the randomized, clinical trial (RCT) design: Consecutive participants who are likely to harbor the target disease are randomized to the different index tests. This study design allows for only indirect comparison of sensitivity and specificity between the two different index tests in made comparable by randomization. This design should be selected when participants cannot undergo both tests because of their risk of harm or costs or both. Example. In the assessment of liver fibrosis, an RCT compared the accuracy of two different liver biopsy needles (Tru-cut versus Menghini) in the diagnosis of cirrhosis: 1,192 patients were randomized to one of the two needles, and a significant difference in diagnostic accuracy was found. 18 Phase IIC. Phase IIC studies explore the possible consequences of the introduction of the index test into clinical practice: The diagnostic strategy incorporating the index test is compared with the current standard evaluation. The question is: what is the proportion of false results in the two diagnostic strategies? The studies of this phase, also referred to as diagnostic exploratory trials, are conducted on participants in whom it is clinically sensible to suspect the target disease in order to assess any immediate downstream consequence of testing and offering treatment, based on that testing. 19 These studies are designed to compare the new diagnostic-therapeutic strategy incorporating the index test against the current best diagnostic strategy for managing these patients. The studies should mainly address safety aspects by enabling the capturing of possible harm as a consequence of introducing the index test. Harm from diagnosis can result from misclassification with two opposite types of error (false negative or false positive). The risk is not only to overlook the target disease, but also to overdiagnose it. On taking into account the prognosis of an untreated disease and the effectiveness and possible harm of available treatments, one has to define what types of error to avoid. The objective is to minimize the penalty of being wrong. Sometimes, it is better to minimize the number of false-negative results and, other times, that of false-positive results. The proportion of false results (i.e., 1-accuracy) is estimated in phase IIA studies and eventually compared with other existing tests. A direct, or even indirect, comparison between sensitivity and specificity of the index test with existing tests makes it possible to hypothesize the role of the index test in a new strategy to diagnose the

8 HEPATOLOGY, Vol. 60, No. 1, 2014 COLLI, FRAQUELLI, ET AL. 415 target disease: as a replacement for the existing test, as an add-on test after the existing test, or as a triage test before the existing test. 20 In general, a more accurate or safer test can replace the existing ones. Particularly, tests with very high sensitivity can rule out the target disease and could be purposed as triage tests before any further harmful or costly testing, whereas tests with very high specificity can be purposed as add-on tests. Thus, the operative characteristics of the new test and its risk of harm and/or cost suggest a possible role for the test. It is in this role that the new diagnostic test should properly be evaluated in phase IIC and subsequent phase III RCTs that assess the downstream effects of the new strategy. Phase IIB studies may anticipate some preliminary evidence of efficacy of the available treatment by comparing the outcomes of patients randomized to the index tests versus the standard test. The primary safety concerns usually limit its power to establish efficacy and thereby support the necessity to conduct subsequent phase III RCTs. Example. An example of a diagnostic exploratory trial is the pilot study conducted to estimate the risk of false-positive results in lung cancer screening, where the cumulative incidence of false-positive results is compared in smokers randomly assigned to annual low-dose computed tomography (CT) versus chest radiography. The false diagnosis of lung cancer (i.e., overdiagnosis) is identified as a possible major harm. In fact, the use of a more sensitive test (CT) requires further confirmatory procedures, both risky and costly, and possibly induces negative psychological effects. 21 Phase III Studies: The Diagnostic Benefits and Harms Phase III studies aim to answer the question: do tested patients fare better than comparable patients tested by a standard method or not tested at all? No diagnostic test can, by itself, ever be of benefit for a patient. However, through a diagnostic test, we can reach a decision about which treatment to offer patients. Therefore, the question posed in phase III RCTs not only deals with test accuracy, but also with the benefits and harms of any therapy decided on the basis of the test results. The appropriate study design is, again, an RCT or, more appropriately, an RCT on diagnostic plus therapeutic strategies. 3 The most important methodological issues that is, allocation sequence generation, allocation concealment, blinding, follow-up, reporting of all outcomes, and transparency regarding conflicts of interest that are central in a therapeutic RCT should also be considered for the diagnostic-therapeutic RCT to secure internal and external validity. However, ensuring an adequate methodology may not be easy. The assessed outcomes should involve more than the conventional ones, i.e. mortality, morbidity, quality of life, diseasefree survival, and adverse events (AEs). Given the possible relevant consequences of labeling a patient with a positive diagnosis, diagnostic test-therapeutic RCTs should also measure short-term outcomes occurring between diagnosis and treatment. 19 Furthermore, the critical comparison between the new diagnostic-therapeutic strategy and the current strategy should be identified. If the index test is intended to improve the sensitivity of the diagnostic strategy, the benefits and harms of the treatment in the extra positive patients should be assessed. Provided that previous trials have shown the efficacy of the treatment among patients detected by the current standard test, the benefits and harms of the treatment in the adjunctive patients detected by the index tests may not be the same and this needs to be evaluated. If the new test is intended to improve the specificity of the diagnostic strategy, then the possible benefit of fewer false-positive results has to be assessed. Furthermore, an improvement in specificity usually entails impairment in sensitivity and vice versa. A more sensitive test is expected to be less specific, and the consequences of this possible trade-off should be carefully explored. The downstream effects of the additional false-negative or -positive results have to be properly assessed. Finally, the new test might be regarded as equally sensitive and specific, because the existent test and the new strategy incorporating the index test is intended to be safer or cheaper. In this case, the effects of the new strategy on the results of treatment and the harmful events associated with testing should be addressed. Diagnostic test-therapeutic trials are appropriate to solve the problem of the target diseases for which there is no reference standard or the one available is imperfect. In this case, the index test results classified as false may actually be true. In fact, a result of the index test named false positive might be a true positive and a false negative of the imperfect (less sensitive) reference test. On the other hand, a false negative of the index test might be a true negative as a consequence of a more specific index test. RCTs that compare the downstream effect of the application of the index test versus the test regarded as the current reference test are able to estimate which test is more effective and thus indirectly more accurate.

9 416 COLLI, FRAQUELLI, ET AL. HEPATOLOGY, July 2014 Fig. 2. (A) Conventional RCT design 1. It answers the question of whether patients who test positive with test A and receive treatment fare better than patients who test positive with test B and receive treatment. (B) Conventional RCT design 2. It answers the question of whether patients diagnosed with test A fare better than patients diagnosed with test B, given that the same treatment strategy is employed for the two tests, including any reassessment of negative results with the possible treatment in case of subsequent positivity. (C) Conventional RCT design 3. It answers the question of whether patients who receive test A as the subsequent test fare better than patients who receive test B as the subsequent test. To answer this question, only those patients with the target disease who are going to be treated and receive subsequent testing in order to ascertain any need for possible treatment modification are included. Three study designs are available for phase III diagnostic test-therapeutic trials. The diagnostic test-therapeutic RCT design 1 answers the question of whether patients that test positive with test A and receive treatment fare better than patients that test positive with test B and receive the same treatment (Fig. 2A). This design is effective in informing which test optimizes the diagnostic testtreatment strategy and also provides evidence on how well patients with negative test results fare. If the tests are used to rule the target disease out, important outcomes from patients excluded from the treatment on the basis of negative results (A or B) must be compared. Thus, this design mainly allows us to answer the question for the group of patients with one of the two test results (positive or negative). The diagnostic test-therapeutic RCT design 2 answers the question of whether patients with positive test A fare better than patients with positive test B, given that the same treatment strategy is employed for the positives, including any reassessment of negative results with the possible treatment in case of subsequent positivity (Fig. 2B). The advantage of this design is that it provides evidence of outcome in all the included patients, regardless of the initial test results, actually resembling sensible clinical practice. However, the reassessment and long-term follow-up of patients with an initially negative test might lead to unmanageable cost increases. The diagnostic test-therapeutic RCT design 3 includes patients with the target disease, who are going to be treated and receive subsequent testing in order to ascertain any need for possible treatment modification. This design answers the question of whether patients that receive test A as a subsequent test fare better than patients who receive test B (Fig. 2C). RCTs design 3 offer valuable information on whether patients still have the condition and whether their treatment should be modified or discontinued. Examples. A desirable step would be to undertake RCTs in which the efficacy of a given diagnostic pathway is compared by randomizing consecutive patients to liver stiffness measurement versus liver biopsy. Given that the treatment is the same in both groups, after a few years of follow-up it will be possible to compare the clinically relevant outcomes in the two groups (i.e., death, liver decompensation, liver transplantation, and so on) while controlling the role of possible confounders. To our knowledge, such trials have not been conducted to date. Examples of RCTs include trials evaluating procalcitonin as a biomarker of bacterial infection, particularly some RCTs assessing the role of procalcitonin-based algorithms for guiding antibiotic use in different settings Phase IV Studies Phase IV studies encompass either large RCTs or cohort studies of patients tested with the index test. The phase IV RCTs are conducted after the index test has been introduced into clinical practice to reassess the benefits and harms of the index-test treatment versus the reference-test treatment strategy. This can be relevant in a number of instances, for example, when the validity of the previous phase III RCTs is questioned, new triage tests are introduced, or changes to the treatment occur. The phase IV observational studies include large cohorts of consecutive participants tested with the

10 HEPATOLOGY, Vol. 60, No. 1, 2014 COLLI, FRAQUELLI, ET AL. 417 index test. These studies should aim to answer the following questions: (1) Is the diagnostic test-therapeutic outcome in consecutive patient series in clinical practice in accord with the results from RCTs? and (2) What is the proportion of patients with rare AEs resulting from the diagnostic test-therapeutic strategy? Furthermore, such cohort studies and RCTs play a role in redefining the accuracy of the index test. This may be required in case of an imperfect reference standard and when one intends to verify discordant results and answer the question: in case of disagreement between the index test and the reference standard test, which are the right results? Through planning an adequate follow-up of the patients with different results for the index test and the reference standard test (i.e., classified as false positive or false negative), these longitudinal studies would allow detecting the actual appearance of the target disease or its complications. Example. Patients with elevated liver stiffness in the range of cirrhosis, but without histological evidence of cirrhosis, might be classified as false positive being actually true positive and a false negative of the liver biopsy. In this respect, a large French study 25 has very recently compared the ability of liver stiffness and liver biopsy to predict overall survival or decompensation. Comment We propose a five-phase development of a new diagnostic test from the definition of its properties and operative characteristics to the demonstration of its value in clinical practice. The logical sequence of diagnostic studies should follow the order defined by the explicit diagnostic questions that such studies are designed to answer. Early phases are necessary for the correct design of later phases, hopefully having progressive increase in the strength of evidence in favor of the index test. However, even if logically consecutive, the progression of diagnostic research may be nonlinear. An earlier phase may be initially skipped or its answer is to be provided by later-phase studies. This would prompt for (re)assessment in an earlier-phase design. This categorization is a development of previous descriptions of the diagnostic architecture. 1-3 Our present categorization carries the advantage of being more comparable to the phases of therapeutic research (e.g., regarding drugs or medical devices). 3 It is essential that the diagnostic accuracy assessment should not be perceived as the final objective of diagnostic research, but only as a necessary step in the introduction of a test into clinical practice. In fact, for most tests, the clinical consequences of their implementation in clinical practice are not sufficiently obvious from the definition of their sensitivity and specificity. To address this question, we require diagnostic-treatment RCTs, even if very few trials have been conducted to date. 19 When two or more diagnostic-treatment RCTs have been completed, systematic reviews, possibly with meta-analyses and trial sequential analyses, are warranted 26 and should be conducted before the new tests are introduced into clinical practice. Then, large cohort surveillance studies would allow for the determination of the actual effects. These final studies are very important because they represent a unique opportunity to assess the real effect of the test on the clinical practice and because such studies can reliably measure its actual benefits and harm as well as identify rare instances of harm that may not be captured through RCTs. As shown in Appendix 1 in the Supporting Information, the sample size for both observational studies and RCTs assessing diagnostic tests may become very large indeed, posing problems with feasibility in times of austerity. References 1. Feinstein AR. Clinical Epidemiology. The architecture of Clinical Research. Philadelphia: WB Saunders; Sackett D, Haynes RB. The architecture of diagnostic research. BMJ 2002;324: Gluud C, Gluud LL. Evidence based diagnostics. BMJ 2005;330: Pepe MS, Etzioni R, Feng Z, Potter JD, Thompson ML, Thornquist M, et al. Phase of biomarker development for early detection of cancer. J Natl Cancer Inst 2001;93: Carmel R, Agrawal YP. Failures of cobalamin assays in pernicious anemia. N Engl J Med 2012;367: Sandrin L, Tanter M, Gennisson JL, Catheline S, Fink M. Shear elasticity probe for soft tissues with 1-D transient elastography. IEEE Trans Ultrason Ferroelectr Freq Control 2002;49: Sandrin L, Fourquet B, Hasquenoph JM, Yon S, Fournier C, Mal F, et al. Transient elastography: a new non-invasive method for assessment of hepatic fibrosis. Ultrasound Med Biol 2003;29: Corpechot C, El Naggar A, Poujol-Robert A, Ziol M, Wendum D, Chazouillères O, et al. Assessment of biliary fibrosis by transient elastography in patients with PBC and PSC. HEPATOLOGY 2006;43: Roulot D, Czernichow S, Le Clesiau H, Costes JL, Vergnaud AC, Beaugrand M. Liver stiffness values in apparently healthy subjects: influence of gender and metabolic syndrome. J Hepatol 2008;48: Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, et al.;standards for Reporting of Diagnostic Accuracy. Toward complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Ann Intern Med 2003:138: Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions Version (updated February 2008). The Cochrane Collaboration, Available from

11 418 COLLI, FRAQUELLI, ET AL. HEPATOLOGY, July Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al.;quadas-2 Group. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155: Ganne-Carrie N, Ziol M, de Ledinghen V, Douvin C, Marcellin P, Castera L, et al. Accuracy of liver stiffness measurement for the diagnosis of cirrhosis in patients with chronic liver diseases. HEPATOLOGY 2006;44: Friedrich-Rust M, Ong MF, Martens S, Sarrazin C, Bojunga J, Zeuzem S, Herrmann E. Performance of transient elastography for the staging of liver fibrosis: a meta-analysis. Gastroenterology 2008;134: Marrero JA, Su GL, Wei W, Emick D, Conjeevaram HS, Fontana RJ, Lok AS. Des-gamma carboxyprothrombin can differentiate hepatocellular carcinoma from nonmalignant chronic liver disease in American patients. HEPATOLOGY 2003;37: Colli A, Pozzoni P, Berzuini A, Gerosa A, Canovi C, Molteni EE, et al. Decompensated chronic heart failure: increased liver stiffness measured by means of transient elastography. Radiology 2010;257: Sagir A, Erhardt A, Schmitt M, H aussinger D. Transient elastography is unreliable for detection of cirrhosis in patients with acute liver damage. HEPATOLOGY 2008;47: Colombo M, Del Ninno E, de Franchis R, De Fazio C, Festorazzi S, Ronchi G, Tommasini MA. Ultrasound-assisted percutaneous liver biopsy: superiority of the Tru-Cut over the Menghini needle for diagnosis of cirrhosis. Gastroenterology 1988;95: Ferrante di Ruffano L, Hyde CJ, McCaffery KJ, Bossuyt PM, Deeks JJ. Assessing the value of diagnostic tests: a framework for designing and evaluating trials. BMJ 2012;344:e Bossuyt PM, Irwig L, Craig J, Glasziou P. Comparative accuracy: assessing new tests against existing diagnostic pathways. BMJ 2006;332: Croswell JM, Baker SG, Marcus PM, Clapp JD, Kramer BS. Cumulative incidence of false-positive test results in lung cancer screening: a randomized trial. Ann Intern Med 2010;152: , W176-W Christ-Crain M, Jaccard-Stolz D, Bingisser R, Gencay MM, Huber PR, Tamm M, M uller B. Effect of procalcitonin-guided treatment on antibiotic use and outcome in lower respiratory tract infections: clusterrandomised, single-blinded intervention trial. Lancet 2004;363: BrielM,SchuetzP,MuellerB,YoungJ,SchildU,NusbaumerC,etal.Procalcitonin guided antibiotic use vs a standard approach for acute respiratory tract infections in primary care. Arch Intern Med 2008;168: Jensen JU, Hein L, Lundgren B, Bestle MH, Mohr T, Andersen MH, et al. Procalcitonin-guided interventions against infections to increase early appropriate antibiotics and improve survival in the intensive care unit: a randomized trial. Crit Care Med 2011;39: Vergniol J, Foucher J, Terrebonne E, Bernard PH, le Bail B, Merrouche W, et al. Non-invasive tests for fibrosis and liver stiffness predict 5-year outcomes of patients with chronic hepatitis C. Gastroenterology 2011;140: Wetterslev J, Thorlund K, Brok J, Gluud C. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol 2008;61:64-75.