Digital image analysis: a review of reproducibility, stability and basic requirements for optimal results

Transcription

1 APMIS 120: Review Article Ó 2011 The Authors APMIS Ó 2011 APMIS DOI /j x Digital image analysis: a review of reproducibility, stability and basic requirements for optimal results RIKKE RIBER-HANSEN, 1 BEN VAINER 2 and TORBEN STEINICHE 1 1 Institute of Pathology, Aarhus University Hospital, Aarhus; and 2 Department of Pathology, Copenhagen University Hospital, Rigshospitalet, Denmark Riber-Hansen R, Vainer B, Steiniche T. Digital image analysis: a review of reproducibility, stability and basic requirements for optimal results. APMIS 2012; 120: Digital image analysis (DIA) is increasingly implemented in histopathological research to facilitate truly quantitative measurements, decrease inter-observer variation and reduce hands-on time. Originally, efforts were made to enable DIA to reproduce manually obtained results on histological slides optimized for light microscopy and the human eye. With improved technical methods and the acknowledgement that computerized readings are different from analysis by human eye, recognition has been achieved that to really empower DIA, histological slides must be optimized for the digital eye, with reproducible results correlating with clinical findings. In this review, we focus on the basic expectations and requirements for DIA to gain wider use in histopathological research and diagnostics. With a reference to studies that specifically compare DIA with conventional methods, this review discusses reproducibility, application of stereology-based quantitative measurements, time consumption, optimization of histological slides, regions of interest selection and recent developments in staining and imaging techniques. Key words: Image processing; computer-assisted; pathology; immunohistochemistry; reproducibility of results. Rikke Riber-Hansen, Institute of Pathology, Aarhus University Hospital, Noerrebrogade 44, DK-8000 Aarhus C, Denmark. rikrib@rm.dk Digital image analysis (DIA) in histopathology is a process of obtaining digitalized images by a microscope connected to a video camera and a frame-grabber or a digital camera, or by whole slide scanning to conduct computer-based analyses to extract meaningful information from the images. In histopathological research, the motivation for implementing DIA has from its invention been to optimize and standardize diagnostic and prognostic processes. The fast development in technology entailed the expectation that DIA would be able to address the intra- and inter-observer variation inherent to manual pathology, to allow truly quantitative measurements to replace semi-quantitative Received 9 July Accepted 21 November 2011 scores and to reduce the workload for the pathologist. For these hopes to come true, some basic requirements have to be met by the DIA process, covering all steps from removal of the tissue of interest to the final assessment. These requirements include optimization and standardization of tissue fixation, sampling and staining as well as image capturing, region of interest (ROI) selection, algorithm design and output. Generally, comparing digital studies performed over a time span of more than 60 years is complicated by the often rapid technological advancement; however, the basic expectations and requirements are unchanged. In this article, we therefore review studies focused on quality 276

2 IMAGE ANALYSIS IN PATHOLOGY RESEARCH issues and discuss why collection of optimal DIA data begins with optimization of such basic issues as tissue handling and processing in the laboratory. REPRODUCIBILITY One of the strongest arguments for implementation of DIA in histopathological research and diagnostics is that DIA will produce more reliable, reproducible or even more objective results than standard manual microscopy. Although a fixed DIA algorithm applied to the same slide twice will produce highly reproducible results, Tadrous (1) in his recent review on objectivity in DIA in pathology highlights that algorithms are devised and implemented by human beings who make subjective decisions at each stage of the algorithm design and implementation. He also argues that the whole image analysis process from block selection to final result must be included in the evaluation of repeatability. Even though this holds true for any scientific study conducted, it is only rarely applied, even in manual reproducibility studies. While most studies have validated DIA estimates of a given parameter by comparison with manual estimates, other gold standard methods or survival data, only a few studies have evaluated what might be called the inter-algorithm performance, and a full algorithm repeatability test as proposed by Tadrous (1) has not been published. However, different aspects of the DIA process have been examined by several investigators. One set of studies have tested the algorithm design performed by different users. Jagoe et al. (2) performed an inter-algorithm study with repetition of the DIA process from image capturing to final result by different operators, but with the same imaging system. They studied the variability in determining the area of immunohistochemically (IHC)-stained cells in the rat pituitary gland by greyscale thresholding, that is, setting a threshold pixel value to separate stained from unstained areas. Each of six observers, all with different experience levels for IHC interpretation and DIA, thresholded three sequences of 40 still images in one sitting. The study shows significant inter- as well as intraobserver variation, which was not reduced by allowing the two operators experienced in IHC interpretation to view the original microscope image. Interestingly, the intra-observer variation was greater than the inter-observer and the between-field variation, that is, the variation between regions selected for assessment. This shows that variation in thresholding may be a significant source of bias using DIA techniques. In a later study by Haringman et al. (3) on the efficacy of anti-inflammatory treatment in synovial biopsies, image acquisition, threshold levels and calculation of the marker of interest were performed twice by one observer and once by two additional observers. The study demonstrated the anticipated reduction in cell count after treatment without significant differences in the intra- and inter-observer analyses. Similar results were obtained in a study of liver fibrosis estimated by DIA, in which two observers each performed the complete protocol of digitization and thresholding in 25 liver samples, achieving an average inter-class correlation coefficient of 0.98 (4). Another approach adopted to study reproducibility is to examine how different DIA systems quantitate the same IHC slides. In the studies on categorical estimation of oestrogen receptor expression in breast cancer samples by Gokhale et al. (5) and Layfield et al. (6) two different DIA systems were tested with concordant results and with a high agreement with either manual readings (5), or estimates obtained by a biochemical ligand-binding assay (6). Comparable results were obtained in a study in which three treatment centres assessed the inflammation in synovial tissue from 12 patients using three different algorithms and comparing the readings with manual quantitative and semiquantitative methods (7). This study also interestingly showed that manual semi-quantitative reading provided significantly higher correlation coefficients than results by comparing manual quantification with DIA. It is not discussed whether the lower correlation coefficients using DIA was caused by the differences in DIA algorithm per se. Finally, investigation of DIA reproducibility may be performed by repeating both manual and digital results and then compare the difference in intra- and inter-observer variability between the two sets of estimates. This was performed in a study of HER-2 expression in breast cancer, where 14 observers (seven pathologists Ó 2011 The Authors APMIS Ó 2011 APMIS 277

3 RIBER-HANSEN et al. and seven DIA researchers without medical training) read 335 HER-2 stained images from 64 breast cancers on a colour monitor with and without computer-extracted values of membrane staining intensity and membrane staining completeness, along with the range of these feature values for each HER-2 category (8). A significant increase in both intra- and interobserver agreement was obtained in the computer-aided estimates. However, this study was based on already selected areas and not whole slides, and as such does not completely mirror the daily diagnostic routine. Although no studies have performed a fullscale investigation of every aspect of the DIA process, the combined evidence clearly shows that digital image assessment is able to reproduce data at an acceptable level with no more variability than manual assessment using conventional microscopy. IMPLEMENTATION OF STEREOLOGICAL QUANTIFICATION METHODS Many diagnostic and prognostic features in pathology today are assessed by semi-quantitative scoring systems. These measurements are often a compromise between more accurate stereological measurements and the time possible to spend on each case. In other situations, truly quantitative measurements have not proven superior to semi-quantitative estimates. One such example is melanoma tumour volume studied by Bønnelykke-Behrndtz et al. (9). In this study of 60 patients, the two-dimensional estimate of primary tumour thickness (Breslow thickness) surpasses melanoma tumour volume measured by stereological methods, as a prognostic factor. As discussed by the authors the explanation may be that melanomas do not expand by the same proportion in all directions, but are more dependent on downward growth. However, for a number of markers, stereology has proven to be a better predictor of outcome, whereas for others stereology has eliminated the inherent bias present in many semi-quantitative estimates. Measurement of the maximum metastasis diameter in sentinel lymph nodes (SLNs) is an example of this issue (10, 11). The SLN is the first node to receive lymph from a given tumour, and for a number of cancers, for example, breast cancer (12) and melanoma (13) the SLN status (i.e. whether the lymph node contains metastasis or not), together with the size of the metastases present, are some of the most important prognostic markers. We have shown that assessment of the largest diameter of SLN metastases is highly protocol-dependent (14). An SLN submitted for complete sectioning would easily produce 1000 sections per lymph node, which is a daunting task for any pathologist, and therefore sampling is mandatory. Consequently, typical protocols include sectioning from three to five levels, often from the central part of the SLNs. As metastases are located in all levels of the SLNs, this approach not only misses a substantial number of metastases (15), but also subjects the diameter measurement to a significant bias. We have previously shown that 20% of patients with metastasis detected in the central sections will experience systematic up-staging if examination of the peripheral parts of the SLNs is included. No patients are downstaged (14). By definition, the maximum metastasis diameter can only stay constant or increase when additional steps are examined. This is in fact true for all other parameters assessing either maximum or minimum values, including highest number of mitoses, smallest distances or highest grades of malignancy. Using digitally assisted manual stereology methods, we have previously shown the total metastatic volume of melanoma SLN metastases to be a significant prognostic factor in line with the largest metastasis diameter, but without the same inherent bias of protocol dependency (11). However, manual volume measurements of this scale are very time-consuming and not manageable in daily diagnostic routines. Fortunately, DIA allows volume measurements to be digitalized (Fig. 1) and we have been able to demonstrate significant agreement between manual and digital volume measurements along with prognostic significance of DIA volume measurements in multivariate analysis (16). WORKLOAD REDUCTION One of the most appealing aspects of DIA is the anticipation of implementing accurate estimates 278 Ó 2011 The Authors APMIS Ó 2011 APMIS

4 IMAGE ANALYSIS IN PATHOLOGY RESEARCH A Three studies from the last decade have compared manual counting of inflammatory cells with DIA. They showed a time reduction of approximately 70 85% without affecting the quality of the readings (17 19). One of the studies included time for image capturing (17), but none exploited the benefit of whole slide scanning. With the emergence of fast, high-volume whole slide scanners able to digitalize a whole slide in a matter of minutes, Wang et al. (20) has developed high-performance computing using multiple blade servers to speed up analysis time in digital tissue microarray (TMA) evaluation. With this technology the authors were able to cut 95% of the process time by analysing one TMA slide containing 230 patient samples in 1 min and with simultaneous digital analysis of 90 TMAs. It is reasonable to believe that progress in technology will facilitate the achievement of even faster results by DIA. REQUIREMENTS FOR OPTIMAL DIA B Fig. 1. Digital image analysis (DIA) of metastatic tumour volume in melanoma sentinel lymph nodes. (A) A lymph node with metastases stained with MART-1 antibody (brown) and counterstained with haematoxylin. DIA segments the metastases (red) from the lymphatic tissue (blue), and post-processing ensures that processing artefacts such as dark appearing folds are not interpreted as tumour cells (B) (MART-1; original magnification 1; Visiomorph, Visiopharm). at reduced workload. However in most cases, DIA studies do not report on time consumption, presumably because DIA as of yet cannot compete with manual semi-quantitative estimation. However, as discussed previously, DIA now offers the opportunity to re-introduce stereologically sound methods of estimations, methods which are normally compromised by a heavy workload. Sampling Sampling is an important step in any quantification process and is performed multiple times during the DIA procedure, initially during tissue selection for preparation and later in ROI selection for further analysis. The ROI selection may be time-consuming and will to a certain degree depend on a subjective evaluation of the scanned areas. In an interesting study, Bedossa et al. (21) investigated the required length of liver biopsy material in order for a correct classification (METAVIR fibrosis score) to be obtained by a DIA system, showing that sampling is just as important for DIA as for ordinary manual light microscopy assessment. In an attempt to circumvent manual ROI selection, Oger et al. (22) described an automated ROI retrieval method, separating epithelial regions from stromal areas in breast biopsies using spectral analysis. In other studies, special DIA algorithm designs developed for HER-2 quantification based on the connectivity of the membranes (23) or based on a computer-targeted mathematical colour model (24) also succeeded in distinguishing stromal from epithelial tissues. However, a skilled user is still required Ó 2011 The Authors APMIS Ó 2011 APMIS 279

5 RIBER-HANSEN et al. for selecting well-preserved tissue of, for example, invasive cancer tissue as opposed to carcinoma in situ changes. Such selection is problematic for two reasons: it is laborious and time-consuming and it is prone to subjectivity which optimally should be avoided in scientific research. Digital solutions to avoid this manual selection step are therefore warranted. Tissue and staining optimization To perform a valid and reproducible DIA algorithm, care must be taken to optimize all steps in the procedure including tissue preparation and staining. Basically, this is not different from manual IHC quantification or molecular pathology, but in fact DIA is probably more dependent on optimal stains than the human eye, as the human brain unlike most computer software is able to compensate efficiently for variations in staining intensity. Optimization procedures generally accepted for tissue ischaemia time and formalin fixation time must be followed to ensure valid antigenic epitope presentation for IHC staining. As an example, it is well acknowledged that fixation length plays a vital role for presentation of nuclear antigens such as the oestrogen receptor, importantly affecting the expression assessment in breast carcinoma (25). Specifically for DIA procedures, sections optimal for digital assessment should be carefully produced to avoid folds and sections of unequal thickness, which can cause great problems in DIA segmentation. Probably most importantly, the staining procedure in general and in IHC in particular must be optimized and standardized, using the best antibodies and staining protocols. This can be ensured by following the recommendations of one of the existing IHC quality programmes, such as NORDIQC (26) or UK-NEQAS (27). Standardizing IHC is complex, because every step in the staining protocol will influence the final result. It has been documented that antigen retrieval is an important factor in inter-institutional IHC variation as well as variation in antibody dilution, differences in antibodies directed at the same antigen and differences in detection systems (25). Effects of staining variation on results obtained by DIA were investigated by Rexhepaj et al. (24) and Tuominin et al. (28). Rexhepaj et al. (24) reported a newly developed DIA protocol for oestrogen and progesterone receptor expression in breast cancer tissue, based on application of a specific mathematical colour model called CIELUV instead of the traditional Red Green Blue (RGB) model. Whereas the RGB model closely resembles perception by human eye, it does not reproduce the whole spectrum of visible colours and it is not perceptually uniform, that is, the same small change in colour in different parts of the colour range will be perceived as unequally sized. To counter these problems the International Commission on Illumination (CIE) adopted the CIELUV in 1976 to ensure easy computation and perceptual uniformity, and today this colour space is widely used in computer graphics. Applied by Rexhepaj et al. (24), this model was robust against variation in haematoxylin intensity. In the second study, Tuominen et al. (28) designed a free, internet-based web application for DIA of the oestrogen and progesterone receptor expression together with the proliferation Ki-67 index in breast cancer tissue. To test the robustness of their application to IHC staining variations, the Ki-67 stain was reproduced using suboptimal antibody dilution and haematoxylin counterstaining. The results show this application to be robust to twofold changes in antibody dilution but dependent on an optimal haematoxylin counterstaining. In their system, weak counterstaining causes suboptimal nuclear detection by the DIA, whereas an inappropriately strong counterstaining leads to cytoplasm falsely interpreted as nuclei. Scanning and digital storage Technological development in DIA has been immense, from the initial black-and-white photos obtained by an analogue video camera and converted to digital images by a frame-grabber, over digitalized cameras and motorized microscope stages to whole slide scanning. The recent advancement of whole slide scanning now allows full slides to be quantitated if need be instead of selected areas. A recent article by Rojo et al. (29) reviewed 31 different digital microscopy systems. Their analysis concludes that the most crucial elements of the systems are camera quality, 280 Ó 2011 The Authors APMIS Ó 2011 APMIS

6 IMAGE ANALYSIS IN PATHOLOGY RESEARCH illumination, digitization speed, image stitching, maximal daily scanning capacity and file formats. Most of the DIA systems evaluated were able to process a whole slide using an 40 objective within 1 h. In our experience, some of the problems encountered using whole slide scanners are the requirement by most systems for optimal glass slides without cracks or glue on the sides of the slides. In addition, at least some systems are unable to scan tissue located close to the slide edge. File storage presents an additional problem with digital pathology. Depending on the scanning system, a whole slide scan runs up to 1 GB, and for routine digital pathology this adds up to TB annually in a larger pathology unit. In research setting where the amount of glass slides is limited to a specific research protocol, the challenge of file size and storage is less important, but will still make demands on hardware speed. Fortunately, recent studies show promising results using lower levels of magnification for analysing digital images (28, 30), even for IHC quantification on JPEG compressed images (28, 31, 32). Algorithm design DIA algorithm design is complex, often requiring skilled engineers and experienced pathologists for optimal results. The principle steps in a DIA algorithm setup are: pre-processing the image to enhance the structures of interest and to suppress noise, segmentation of the image into its relevant components, post-processing to refine the segmentation and generation of output (33). Absolute standardization of DIA algorithms for specific tasks would be optimal, but even for diagnostic purposes DIA algorithms must be flexible to encounter inter-institutional and day-to-day variations in tissue staining. Theoretically, a solution to this problem might be an automatic adjustment of the DIA protocol to small series of control sections with known expression and staining intensity of tissue components. The need for standardized specimens exhibiting, for example, a given percentage of positive cells may be hampered by tissue heterogeneity, which would render the selection of such controls difficult. We suggest that some kind of artificial tissue or artificially grown tissue components might be used for this purpose. DIA VALIDATION As a newly developed technology, DIA research in histopathology has mainly focused on validation according to the generally accepted methods of tissue or cell assessment. The validation has been performed by comparing digital results with manual estimates, either quantitative or semi-quantitative; by comparing DIA with another form of gold standard, for example, fluorescence in situ hybridization (FISH); or by comparing DIA with clinical (often prognostic) information. When reviewing the data on DIA validation, one must recognize the extremely rapid technological development, and that techniques validated a few years ago may have been surpassed by technological advancements. Comparison with manual estimates As semi-quantitative estimates are the gold standard in many areas of present pathology, numerous studies have compared DIA with results obtained by manual reading of histopathological specimens. Most of these studies have been performed using digital video cameras and analysis of select areas chosen by one of the investigators and have employed a variable array of DIA algorithms and systems. Histopathological evaluation of liver biopsies is essential not only for the diagnosis but also for assessing inflammatory activity and chronicity (stage) of liver diseases. Various scoring systems of liver fibrosis based on architectural alterations of the normal lobular structure are available to the pathologist. Although founded on well-characterized structural changes, these scoring systems are still exposed to subjectivity of the pathologist s judgement. Fibrosis is a result of a healing process with scar tissue formation. Collagen is the major component of this fibrosis, and cirrhotic liver contains approximately 30 mg g collagen compared with 5.5 mg g in the normal liver (34). In a study of the effect of interferon-c1b in patients with chronic hepatitis C, Goodman et al. (35) showed that digital quantification of the collagen content in Sirius red-stained sections is Ó 2011 The Authors APMIS Ó 2011 APMIS 281

7 RIBER-HANSEN et al. more sensitive than histological staging to demonstrate the progression of fibrosis. This was corroborated by a research team headed by Burroughs and Dhillon, investigating specimens from patients who were liver transplanted for hepatitis C virus-induced cirrhosis (36, 37). This group have recently published a line of studies addressing the translation of DIA into clinical parameters in chronic liver disease (36 38). A comparison with fibrosis scoring systems demonstrated a correlation between the relative collagen area (designated as the collagen proportionate area, CPA) and the Ishak stage score, but it more precisely correlated with the hepatic venous pressure gradient believed to be the best clinical parameter of liver fibrosis (36). The CPA technique presents an easily applicable method to assess liver fibrosis, although automatic steps for deselecting unwanted areas were still needed, such as larger venous lumina. An example of DIA in liver fibrosis is presented in Fig. 2. The introduction of IHC in the 1980s opened up a new area of research and diagnostics in histopathology and was adopted by DIA from an early point. IHC staining segments positive from negative cells, and accordingly cellular markers such as lymphocytic surface antigens and the nuclear marker Ki-67 have been obvious targets for development into DIA technology. The interest in digital inflammatory cell quantification has most likely been spurred by the labour and time required for manual cell counting. While initially primarily investigated in inflammatory diseases, contemporary inflammatory cell quantification has spread to cancer research because of the growing evidence for the importance of interaction between cancer cells and the microenviroment (39). A B C D Fig. 2. Digital image analysis of liver fibrosis. To avoid areas outside the liver tissue to be included in the calculations, the region of interest (ROI) is outlined (in this case with a green dotted line; A). A software algorithm recognizes the red pixels (selected by the software on the basis of wavelengths) and applies an overlayer with adjustable transparency, allowing confirmation of the selected areas (B, green). The collagen proportionate area is subsequently calculated as the area of the collagen divided by the total ROI area. This procedure rather precisely selects only the collagen fibres (C, D), and it is therefore possible to avoid including structures not involved in the increased stiffness of the liver such as proliferating bile ducts (full-line arrow) or inflammatory cells (dotted arrow; D). Electronical subtraction of empty spaces, for example, the larger vessel lumina, from the total area is also possible (Sirius red; cropped screen dumps from Visiomorph, Visiopharm). 282 Ó 2011 The Authors APMIS Ó 2011 APMIS

8 IMAGE ANALYSIS IN PATHOLOGY RESEARCH Several studies have shown excellent correlation between manual cell counting and DIA (17, 18, 40, 41). However, one of the main challenges in DIA cell counting is separating individual cells because of indistinct cell borders. A solution to this issue has been proposed by measuring the area fraction of cell infiltrates compared with the total tissue area, and in a study of central nervous system inflammation excellent correlation with profile counting techniques was demonstrated (19). In contrast, a lymphoma study found only poor correlation between the percentage of Ki-67-positive nuclei counted separately and the fraction of the area occupied by Ki-67-positive malignant cells (42). We are currently investigating DIA in melanoma Ki-67 expression, and we have found no significant difference in cell counting and Ki-67 area fractions (43). Based on our experience and the study by Rexhepaj et al. (24), DIA cell counting can be enhanced by optimization of the DIA algorithm. In several cancer types the proliferation marker Ki-67 has been implemented in routine diagnostic pathology as either a diagnostic or a prognostic marker, and thus it has been an obvious target for DIA development. Most studies have shown excellent correlation between visual and digital estimates for Ki-67, including studies of oligodendrogliomas (44), meningeomas (45), Hodgkin lymphomas (46) and breast cancers (28). In disagreement, Klapper et al. (42) compared stringent Ki-67 counting using light microscopy and DIA and found poor agreement between these reading methods in mantle cell lymphoma. However, the authors did not comment on whether the DIA technique could safely distinguish tumour cells from the surrounding reactive lymphocytes outnumbering tumour cells in mantle cell lymphoma. In addition, the IHC was performed by four different institutions using two different antibody clones and four individual staining procedures, but a discussion of whether the differences in staining methods between the four different institutions played a role is not included. Comparison with other gold standards As a result of its prognostic role and its ability to predict response to trastuzumab (Herceptin) treatment, assessment of HER-2 expression in breast cancer tissue has been implemented in the diagnostic work-up over the last few years. FISH analysis of HER-2 amplification is considered the gold standard technique for identifying patients who will benefit from trastuzumab treatment. This is, however, resource demanding, and IHC is therefore used to categorize patients to treatment (IHC score: 3+), no treatment (IHC score: 0 or 1+) or supplementary FISH examination (IHC score: 2+) for definite treatment decision. This makes HER-2 FISH results excellent for comparison with DIA of IHC results. In a recent TMA study of 219 consecutive breast cancers, Skaland et al. (47) showed DIA to correlate with manual consensus scores by three expert pathologists. In addition, DIA was superior to manual readings in classifying IHC3 + FISH + cases. In another larger TMA study (n = 616), apparently using the same US Food and Drug Administration approved DIA system, kappa values for HER-2 FISH and digital IHC assessment were lower than the ones for FISH and manual readings (48), although both showed good agreement (digital reading: ; manual reading: ). In this study, two different operators (one experienced and one inexperienced) scanned and processed all slides using the same DIA protocol on two identical DIA-processing machines. The weakest agreement between FISH and DIA was demonstrated for the inexperienced investigator. As discussed by the authors, the differences between the two machines may be explained by inexperience in operating the system, but differences in scanner settings, for example, calibration and white balance may also play a role. In addition, DIA resulted in 2 3 times more 2+ cases, leading to substantially more FISH analyses to be performed if initially assessed by DIA alone compared with manual estimates. These results are in conflict with a smaller previous study using a different DIA system (49), which showed a better distinction of 2+ cases by DIA than by manual assessment. The referred studies all used staining intensity as the parameter for DIA analysis. Recently, a new DIA system for examining connectivity and size distribution of stained membranes has been published, showing a sensitivity of 99.2% and a specificity of 100% compared with HER-2 Ó 2011 The Authors APMIS Ó 2011 APMIS 283

9 RIBER-HANSEN et al. FISH in a cohort of 43 breast cancer patients (23). This study shows a promising trend to exploit the special capabilities of computer algorithms to optimize DIA performance instead of relying primarily on staining intensity. It also opens possibilities to include more parameters from the human eye assessment into stabile, high-valid computer algorithms. Comparison with clinical data Several studies have compared marker estimates obtained by DIA techniques with clinical data, such as disease-free and overall survival, but like many manually assessed markers, most of these have been hypothesis-generating investigations without subsequent validation (6, 50 52). A few studies have, however, validated DIA results by algorithm repetition in a new cohort with subsequent comparison with clinical follow-up data (53 55). Rexhepaj et al. and Skaland et al. successfully validated the digital assessment of survivin (53) and phosphohistone H3 (54) as prognostic markers in breast cancer as a follow-up on hypothesis-generating studies conducted earlier by the same research groups (56, 57). Interestingly, Skaland et al. established phosphohistone H3 as the most important prognostic factor in lymph node-negative breast cancer patients less than 71 years of age (54). In both studies, the validation was carried out using the original staining protocols on tissue from comparable patient cohorts, but significant changes were made to the initial DIA procedures. An additional step differentiating tumour cells from stroma was applied in the study by Rexhepaj et al. (53), whereas Skaland et al. replaced the originally applied Image-J DIA software program (National Institutes of Health, Washington, USA; with the fully automated VIS analysis system (Visiopharm, Hørsholm, Denmark), however, using the same image-processing principles established in their first study (54). These changes in DIA procedures did show any obvious effect on outcome variables. A third study by Alexander et al. (55) investigating a new 4-antibody model of DNA repair proteins segregated high- and low-risk groups of breast cancer patients negative for oestrogen receptor, progesterone receptor and HER-2 protein expression in both univariate and multivariate survival analyses. FUTURE PERSPECTIVES AND CONCLUSIONS In the paper The Analysis of Cell Images published in 1966, Prewitt and Mendelsohn stated that It would be a serious mistake to presuppose that mechanical perception must mimic the human s perceptual apparatus. The recent developments of digital techniques have emphasized the veracity of this statement, and it becomes still more evident that we need to focus on areas in which the human eye and the human brain distort reality or in which the human assessment in reality is mechanical. Today, most DIA systems are based on RGB images, where each colour is described by the amount of red, green and blue light beams. Unfortunately, histopathological stains including IHC may show overlapping colour spectra in RGB images posing difficulties to automatic readings. To circumvent this problem several suggestions have been made to enhance colour discrimination by replacing standard staining techniques with others. For example, chromogen combinations of blue or black with red, as suggested in a report from a workshop of DIA in lung tissue from 2001 (58), or by using lighter shades of haematoxylin to allow good distinction from 3,3 -diaminobenzidine (DAB) (44). A new promising tool to enhance segmentation is IHC double staining, allowing simultaneous detection of different antigens present in the cells of interest. To our knowledge, no study using this technique has yet been published, but studies of Ki-67 and cytokeratin double staining are currently being conducted (Fig. 3). We have recently shown that in a cohort of 128 melanocytic lesions manual estimation of IHC double labelling with antibodies directed against MART1 and Ki-67 had excellent discriminative powers between nevi and melanomas (59), because proliferating lymphocytes and stromal cells in the surrounding infiltrate can be filtered. We are currently completing a series of DIA studies of this issue, and the results appear promising (43). Until now, flourochrome labelling has been the method of choice in evaluating the co-localization of two or more proteins in a tissue 284 Ó 2011 The Authors APMIS Ó 2011 APMIS

10 IMAGE ANALYSIS IN PATHOLOGY RESEARCH Fig. 3. Immunohistochemically double-stained breast cancer tissue with red-stained cytoplasm (cytokeratin KL-1; asterisk) and brown-stained nuclei (proliferation marker Ki-67; arrow) to segment proliferating cancer cells from proliferating stromal cells (arrow head; A). The digital image analysis algorithm detects and marks brown-stained nuclei yellow (B). Image (C) was obtained after elimination of brown nuclei not completely surrounded by red cytoplasm, segmentation of cytoplasm (red), Ki-67-negative (blue) and -positive (green) tumour nuclei and post-processing to eliminate unspecific brown staining (Kl-1 and Ki-67, original magnification 20; Visiomorph, Visiopharm). Courtesy of Patricia Switten Nielsen, MSc, Institute of Pathology, Aarhus University Hospital, Denmark. A B C section. However, spectral overlap between flourochromes and the presence of autofluorescence may complicate this analysis, and therefore various unmixing computer protocols exist based on the spectral properties of the flourochromes. The major problem using fluorescence in general is that identifying cells or tissue morphology is seriously hampered by the need for dark field microscopy. Yet, new and fascinating developments have emerged to encounter both spectral mixing problems and the lack of morphology. One of these developing techniques was demonstrated in an elaborate study using so-called multi-epitope-ligand cartography (MELC) with the possibility of labelling up to 100 different epitopes in the same section by bleaching each dye completely after labelling and imaging (60). The epitope images are superimposed on an image obtained by phase contrast, which provides a relatively good morphology and allows a large number of components to be digitally co-localized in fixed cells and tissue. Another emerging technology applicable for digital visualization are quantum dots, which are nearly spherical and highly fluorescent nanocrystals with a strictly defined emission spectrum and a strong emission signal, along with minimal photobleaching and long fluorescence lifetimes. These features allow multiple spectra to be readily distinguished from each other, facilitating multiple marker labelling (61). Application of quantum dots has been described in several studies on human tissue, including evaluation of the heterogeneity of prostate cancer (62), assessment of prognostic markers in Ó 2011 The Authors APMIS Ó 2011 APMIS 285

11 RIBER-HANSEN et al. prostate cancer (63) and identification of a new subtype of breast cancer (64). A study by Hikage et al. (65) has suggested that the quantum dot technology will be able to detect the exact location of metastases in SLNs by injection in the tumour of interest. Like in MELC, applying multiple quantum dot flourochromes to a single cell may be of help in discrimination of individual cells. However, if additional morphological information is needed, digitalization and alignment of a haematoxylin and eosinstained tissue section adjacent to the fluorescence section can be done. To gain maximum information from quantum dot labelling, the imaging technique of spectral image analysis must be applied. Spectral images are obtained by illuminating the histological slide at different wavelengths and obtaining precise optical spectra for each pixel, allowing dyes with partially overlapping spectra to be separated (66). As an example, green autofluorescence can be separated from green flourochromes (67), and DAB can be distinguished from melanin pigment. Besides being used in combination with quantum dots, spectral image analysis has been used for cytology (68, 69), breast cancer SLN metastases (70) and proliferation and mitosis measurements (71) with promising results. Traditional approaches to quantification in histological tissues have been semi-quantitative visualization by light microscopy or cumbersome manual stereology. This approach is still used in today s patient care where treatment is decided using semi-quantitative measurements in cytology or tissue sections, often flawed by significant measurement bias or inter-observer variation. Today, DIA offers solutions to advance truly quantitative measurements in sections correctly sampled for the benefit of both research and patient care. Although much remains to be learned about the reproducibility, stability and optimal handling of tissues for DIA, ongoing research is constantly breaking new grounds for digital quantitative pathology. REFERENCES 1. Tadrous PJ. On the concept of objectivity in digital image analysis in pathology. Pathology 2010; 42: Jagoe R, Steel JH, Vucicevic V, Alexander N, van NS, Wootton R, et al. Observer variation in quantification of immunocytochemistry by image analysis. Histochem J 1991;23: Haringman JJ, Vinkenoog M, Gerlag DM, Smeets TJ, Zwinderman AH, Tak PP. Reliability of computerized image analysis for the evaluation of serial synovial biopsies in randomized controlled trials in rheumatoid arthritis. Arthritis Res Ther 2005;7:R Lazzarini AL, Levine RA, Ploutz-Snyder RJ, Sanderson SO. Advances in digital quantification technique enhance discrimination between mild and advanced liver fibrosis in chronic hepatitis C. Liver Int 2005;25: Gokhale S, Rosen D, Sneige N, Diaz LK, Resetkova E, Sahin A, et al. Assessment of two automated imaging systems in evaluating estrogen receptor status in breast carcinoma. Appl Immunohistochem Mol Morphol 2007;15: Layfield LJ, Saria EA, Conlon DH, Kerns BJ. Estrogen and progesterone receptor status determined by the Ventana ES 320 automated immunohistochemical stainer and the CAS 200 image analyzer in 236 early-stage breast carcinomas: prognostic significance. J Surg Oncol 1996;61: Rooney T, Bresnihan B, Andersson U, Gogarty M, Kraan M, Schumacher HR, et al. Microscopic measurement of inflammation in synovial tissue: inter-observer agreement for manual quantitative, semiquantitative and computerised digital image analysis. Ann Rheum Dis 2007;66: Gavrielides MA, Gallas BD, Lenz P, Badano A, Hewitt SM. Observer variability in the interpretation of HER2 neu immunohistochemical expression with unaided and computer-aided digital microscopy. Arch Pathol Lab Med 2011;135: Bonnelykke-Behrndtz ML, Sorensen FB, Damsgaard TE. Stereological quantification of tumor volume, mean nuclear volume and total number of melanoma cells correlated with morbidity and mortality. APMIS 2008;116: van der Ploeg AP, van Akkooi AC, Schmitz PI, Koljenovic S, Verhoef C, Eggermont AM. EORTC Melanoma Group sentinel node protocol identifies high rate of submicrometastases according to Rotterdam Criteria. Eur J Cancer 2010;46: Riber-Hansen R, Nyengaard JR, Hamilton- Dutoit SJ, Sjoegren P, Steiniche T. Metastatic melanoma volume in sentinel nodes: objective stereology-based measurement predicts disease recurrence and survival. Histopathology 2009;54: Carter CL, Allen C, Henson DE. Relation of tumor size, lymph node status, and survival in 24,740 breast cancer cases. Cancer 1989;63: Ó 2011 The Authors APMIS Ó 2011 APMIS

12 IMAGE ANALYSIS IN PATHOLOGY RESEARCH 13. Balch CM, Gershenwald JE, Soong SJ, Thompson JF, Atkins MB, Byrd DR, et al. Final version of 2009 AJCC Melanoma Staging and Classification. J Clin Oncol 2009;27: Riber-Hansen R, Nyengaard JR, Hamilton- Dutoit SJ, Steiniche T. Stage migration after minor changes in histologic estimation of tumor burden in sentinel lymph nodes: the protocol trap. Cancer 2009;115: Riber-Hansen R, Nyengaard JR, Hamilton- Dutoit SJ, Steiniche T. The nodal location of metastases in melanoma sentinel lymph nodes. Am J Surg Pathol 2009;33: Riber-Hansen R, Nyengaard JR, Hamilton- Dutoit S, Sjoegren P, Steiniche T. Automated digital volume measurement of melanoma metastases in sentinel nodes predicts disease recurrence and survival. Histopathology 2011;59: Goedkoop AY, de Rie MA, Teunissen MB, Picavet DI, van der Hall PO, Bos JD, et al. Digital image analysis for the evaluation of the inflammatory infiltrate in psoriasis. Arch Dermatol Res 2005;297: Kraan MC, Haringman JJ, Ahern MJ, Breedveld FC, Smith MD, Tak PP. Quantification of the cell infiltrate in synovial tissue by digital image analysis. Rheumatology (Oxford) 2000;39: Donnelly DJ, Gensel JC, Ankeny DP, van RN, Popovich PG. An efficient and reproducible method for quantifying macrophages in different experimental models of central nervous system pathology. J Neurosci Methods 2009;181: Wang Y, McCleary D, Wang CW, Kelly P, James J, Fennell DA, et al. Ultra-fast processing of gigapixel tissue MicroArray images using high performance computing. Anal Cell Pathol (Amst) 2010;33: Bedossa P, Dargere D, Paradis V. Sampling variability of liver fibrosis in chronic hepatitis C. Hepatology 2003;38: Oger M, Belhomme P, Klossa J, Michels JJ, Elmoataz A. Automated region of interest retrieval and classification using spectral analysis. Diagn Pathol 2008;3(Suppl 1):S17; Brugmann A, Eld M, Lelkaitis G, Nielsen S, Grunkin M, Hansen JD, et al. Digital image analysis of membrane connectivity is a robust measure of HER2 immunostains. Breast Cancer Res Treat 2011 [Epub ahead of print]. 24. Rexhepaj E, Brennan DJ, Holloway P, Kay EW, McCann AH, Landberg G, et al. Novel image analysis approach for quantifying expression of nuclear proteins assessed by immunohistochemistry: application to measurement of oestrogen and progesterone receptor levels in breast cancer. Breast Cancer Res 2008;10:R89; Walker RA. Quantification of immunohistochemistry issues concerning methods, utility and semiquantitative assessment I. Histopathology 2006;49: NORDIQC Website. (accessed 9 July 2011). 27. UK-NEQAS Website. uk/content/pageserver.asp?s= &c= 1252&Type=N&AID=16&SID=47 (accessed 9 July 2011). 28. Tuominen VJ, Ruotoistenmaki S, Viitanen A, Jumppanen M, Isola J. ImmunoRatio: a publicly available web application for quantitative image analysis of estrogen receptor (ER), progesterone receptor (PR), and Ki-67. Breast Cancer Res 2010;12:R56; Rojo MG, Garcia GB, Mateos CP, Garcia JG, Vicente MC. Critical comparison of 31 commercially available digital slide systems in pathology. Int J Surg Pathol 2006;14: Johansson AC, Visse E, Widegren B, Sjogren HO, Siesjo P. Computerized image analysis as a tool to quantify infiltrating leukocytes: a comparison between high- and low-magnification images. J Histochem Cytochem 2001;49: Lopez C, Jaen MJ, Lejeune M, Escriva P, Salvado MT, Pons LE, et al. Roundness variation in JPEG images affects the automated process of nuclear immunohistochemical quantification: correction with a linear regression model. Histochem Cell Biol 2009;132: Lopez C, Lejeune M, Escriva P, Bosch R, Salvado MT, Pons LE, et al. Effects of image compression on automatic count of immunohistochemically stained nuclei in digital images. J Am Med Inform Assoc 2008;15: Grunkin M, Raundahl J, Foged NT. Practical considerations of image analysis and quantification of signal transduction IHC staining. Methods Mol Biol 2011;717: Rojkind M, Ponce-Noyola P. The extracellular matrix of the liver. Coll Relat Res 1982;2: Goodman ZD, Becker RL Jr, Pockros PJ, Afdhal NH. Progression of fibrosis in advanced chronic hepatitis C: evaluation by morphometric image analysis. Hepatology 2007;45: Calvaruso V, Burroughs AK, Standish R, Manousou P, Grillo F, Leandro G, et al. Computer-assisted image analysis of liver collagen: relationship to Ishak scoring and hepatic venous pressure gradient. Hepatology 2009;49: Manousou P, Dhillon AP, Isgro G, Calvaruso V, Luong TV, Tsochatzis E, et al. Digital image analysis of liver collagen predicts clinical outcome of recurrent hepatitis C virus 1 year after liver transplantation. Liver Transpl 2011;17: Germani G, Burroughs AK, Dhillon AP. The relationship between liver disease stage and liver fibrosis: a tangled web. Histopathology 2010;57: Ó 2011 The Authors APMIS Ó 2011 APMIS 287

13 RIBER-HANSEN et al. 39. Hanahan D, Weinberg RA. Hallmarks of cancer: the next generation. Cell 2011;144: Noutsias M, Pauschinger M, Ostermann K, Escher F, Blohm JH, Schultheiss H, et al. Digital image analysis system for the quantification of infiltrates and cell adhesion molecules in inflammatory cardiomyopathy. Med Sci Monit 2002;8: MT Sont JK, De Boer WI, van Schadewijk WA, Grunberg K, van Krieken JH, Hiemstra PS, et al. Fully automated assessment of inflammatory cell counts and cytokine expression in bronchial tissue. Am J Respir Crit Care Med 2003;167: Klapper W, Hoster E, Determann O, Oschlies I, van der Laak J, Berger F, et al. Ki-67 as a prognostic marker in mantle cell lymphoma consensus guidelines of the pathology panel of the European MCL Network. J Hematol 2009;2: Nielsen PS, Riber-Hansen R, Raundahl J, Steiniche T. Automated quantification of MART1- verified Ki67 indices by digital image analysis in melanocytic lesions. Arch Pathol Lab Med (in press). 44. Coleman KE, Brat DJ, Cotsonis GA, Lawson D, Cohen C. Proliferation (MIB-1 expression) in oligodendrogliomas: assessment of quantitative methods and prognostic significance. Appl Immunohistochem Mol Morphol 2006;14: Grala B, Markiewicz T, Kozlowski W, Osowski S, Slodkowska J, Papierz W. New automated image analysis method for the assessment of Ki-67 labeling index in meningiomas. Folia Histochem Cytobiol 2009;47: Lejeune M, Jaen J, Pons L, Lopez C, Salvado MT, Bosch R, et al. Quantification of diverse subcellular immunohistochemical markers with clinicobiological relevancies: validation of a new computer-assisted image analysis procedure. J Anat 2008;212: Skaland I, Ovestad I, Janssen EA, Klos J, Kjellevold KH, Helliesen T, et al. Digital image analysis improves the quality of subjective HER-2 expression scoring in breast cancer. Appl Immunohistochem Mol Morphol 2008;16: Turashvili G, Leung S, Turbin D, Montgomery K, Gilks B, West R, et al. Inter-observer reproducibility of HER2 immunohistochemical assessment and concordance with fluorescent in situ hybridization (FISH): pathologist assessment compared to quantitative image analysis. BMC Cancer 2009;9:165; Ellis CM, Dyson MJ, Stephenson TJ, Maltby EL. HER2 amplification status in breast cancer: a comparison between immunohistochemical staining and fluorescence in situ hybridisation using manual and automated quantitative image analysis scoring techniques. J Clin Pathol 2005;58: Cheng L, Pisansky TM, Sebo TJ, Leibovich BC, Ramnani DM, Weaver AL, et al. Cell proliferation in prostate cancer patients with lymph node metastasis: a marker for progression. Clin Cancer Res 1999;5: Stanton KJ, Sidner RA, Miller GA, Cummings OW, Schmidt CM, Howard TJ, et al. Analysis of Ki-67 antigen expression, DNA proliferative fraction, and survival in resected cancer of the pancreas. Am J Surg 2003;186: Levy MJ, Clain JE, Clayton A, Halling KC, Kipp BR, Rajan E, et al. Preliminary experience comparing routine cytology results with the composite results of digital image analysis and fluorescence in situ hybridization in patients undergoing EUSguided FNA. Gastrointest Endosc 2007;66: Rexhepaj E, Jirstrom K, O connor DP, O Brien SL, Landberg G, Duffy MJ, et al. Validation of cytoplasmic-to-nuclear ratio of survivin as an indicator of improved prognosis in breast cancer. BMC Cancer 2010;10:639; Skaland I, Janssen EA, Gudlaugsson E, Klos J, Kjellevold KH, Soiland H, et al. Validating the prognostic value of proliferation measured by phosphohistone H3 (PPH3) in invasive lymph node-negative breast cancer patients less than 71 years of age. Breast Cancer Res Treat 2009; 114: Alexander BM, Sprott K, Farrow DA, Wang X, D Andrea AD, Schnitt SJ, et al. DNA repair protein biomarkers associated with time to recurrence in triple-negative breast cancer. Clin Cancer Res 2010;16: Skaland I, Janssen EA, Gudlaugsson E, Klos J, Kjellevold KH, Soiland H, et al. Phosphohistone H3 expression has much stronger prognostic value than classical prognosticators in invasive lymph node-negative breast cancer patients less than 55 years of age. Mod Pathol 2007;20: Brennan DJ, Rexhepaj E, O Brien SL, McSherry E, O connor DP, Fagan A, et al. Altered cytoplasmic-to-nuclear ratio of survivin is a prognostic indicator in breast cancer. Clin Cancer Res 2008;14: De Boer WI, Hiemstra PS, Sont JK, De HE, Rabe KF, van Krieken JH, et al. Image analysis and quantification in lung tissue. Clin Exp Allergy 2001;31: Nielsen PS, Riber-Hansen R, Steiniche TD. Immunohistochemical double stains against Ki67 MART1 and HMB45 MITF: promising diagnostic tools in melanocytic lesions. Am J Dermatopathol 2011;33: Ó 2011 The Authors APMIS Ó 2011 APMIS