OBJECTIVE BAKE ASSESSMENT USING IMAGE ANALYSIS AND ARTIFICIAL INTELLIGENCE

Transcription

1 OBJECTIVE BAKE ASSESSMENT USING IMAGE ANALYSIS AND ARTIFICIAL INTELLIGENCE L. G. C. Hamey 1,2, J. C-H. Yeh 1,2 and C. Ng 1,3 1 Cooperative Research Centre for International Food Manufacture and Packaging Science, Hawthorn, Vic 3122 Australia 2 Macquarie University, Department of Computing, NSW 2109 Australia 3 Arnott s Biscuits Ltd, Research Centre, Homebush, NSW 2140 Australia Abstract Bake assessment in food manufacture is currently performed by trained human inspectors. The subjective nature of human assessment introduces short-term variation and long-term drift of bake standards due to inconsistencies in human performance. A durable, repeatable and transferable assessment method is desirable to ensure long-term consistency of product bake and to facilitate product migration between manufacturing sites. These goals are successfully met by employing computer analysis of colour digital images to assess product bake. Our studies have shown the existence of a baking curve, unique to each product, which describes the colour development of that product during baking. Based upon this discovery, a system has been developed employing Artificial Intelligence techniques, specifically Artificial Neural Networks, to match computer bake assessment to one or more expert bake assessors. The process of training the system for a particular product is driven by these assessors who select and classify product samples. By this means, the system can be deployed across the product range with minimal involvement of image analysis and computing technologists. The system has been shown to provide significant performance improvements in the bake quality assessment task. 1. Introduction The colour of baked goods is important to the baking industry as it affects their appearance and visual appeal, and is also related to flavour, texture and moisture content. Off-colour food is likely to be rejected even if it has good flavour and texture whereas food products with high colour quality have higher market value (Ling et al, 1996). The baking industry has, until recently, relied upon human experts to assess product quality parameters such as bake colour. However, human expert assessments are subjective and vary depending upon environment and experience. Samples selected to represent desired bake colour have limited useful life and their colour changes with aging. Photographs also fade with age, and the difficulties of achieving accurate colour rendition have prevented their use as a standard for bake colour. Food colour is often measured using tristimulus colorimetry techniques (Hunter, 1975), in which an instrument measures the reflectance of a patch of the surface of an object in three colour bands. From these measurements, the colour of the object can be determined. Tristimulus colorimetry has been used in numerous food applications including the evaluation of colour in young red wines (Almela et al, 1995) and peanut butter (Pattee, Giesbrecht and Young, 1991) and to study the colour evolution in 13 varieties of paprika pepper (de Guevara and Pardo-González, 1996). However, a colorimeter is only able to measure the colour of one surface patch at a time. Similar limitations apply to spectrophotometry (Mabon, 1993). Thus,

2 as stated aptly in (Ling et al, 1996), a sensing technology which can rapidly monitor the change in colour during processing is needed. Colour machine vision is the technique that can fill such a role, due to its higher spatial resolution and the ability to measure colour at many locations simultaneously. In addition, machine vision can be used to evaluate non-colour aspects such as shape and texture. Grenander and Manbeck (1993) use machine vision for defect detection in potatoes, based on both shape and colour. A digital imaging system consists of a video camera and illumination sources, a computer with image capture hardware installed, and suitable software to operate the equipment. Digital images may be preserved immutably on computer media such as compact disc, and may be interpreted consistently provided that the imaging equipment is calibrated against laboratory standards. The present paper describes a digital imaging system for assessment of bake colour in baked goods. 2. The Baking Curve In order to determine the relationship between product bake and colour development, we studied the colour changes during baking in an industrial oven. We imaged samples of product from each zone of the oven, projecting the colours of individual picture elements (pixels) into the camera s red-green-blue colour space. The results demonstrate that the colour development of biscuits follows a clearly defined path through colour space. We call this the baking curve for a product (RayChaudhuri et al, 1995). White Green Baked product Raw dough Blue Red Black Figure 1: Baking curve for a plain cracker biscuit. Figure 1 illustrates the baking curve for a plain cracker product. Similar characteristic curves have been found for a wide variety of biscuit and cookie products. For crackers in particular, the presence of blisters on the product means that some portions of the product sample develop in colour more quickly than other portions. However, the colour development path

3 followed remains the same. This observation prompts us to hypothesise that the chemical processes involved in colour development are very similar whether the colour development proceeds quickly or slowly. This observation is important for automatic bake colour assessment since it allows the bake colour development to be assessed along the baking curve, irrespective of the speed of the colour development process. 3. Image Analysis System Our experiments were conducted using an image analysis system consisting of a PC computer, a JVC KYF55-BE 3-chip colour CCD camera, Data Translation DT2871 frame capture board and associated software. Illumination is provided by daylight colour balanced fluorescent lamps as illustrated in figure 2. The image analysis software includes camera calibration procedures supported by the use of reflectance calibration targets traceable to international standards. 4. Bake Assessment Approach Figure 2: Image capture system: camera and lighting. The assessment of bake colour can be conceived as a transformation of measurements of product colour along the baking curve to corresponding bake colour assessments as produced by an expert assessor. Our assessment system employs Artificial Neural Networks, a form of flexible modeling procedure, to estimate the relationship between the colours present in a product sample and the expert assessment of the bake colour (Westcott and Hamey, 1995). The trained networks may then be deployed to assess previously unseen product samples with high consistency and precision (Yeh et al, 1995). The procedure for training the assessment system (figure 3) commences with sample collection. A representative variety of product bake samples are collected, including both acceptable and unacceptable product. The samples are then digitally imaged and assessed by an expert. Assessments are on a nine-point scale as detailed in table 1. In order to assess the consistency of expert assessors, every sample is graded in two separate passes through the sample set. The expert assessments and the digital images provide the training data for the neural network system. The trained neural network system can then be loaded into an image analysis PC to perform product assessment.

4 9 5 1 Over baked Under baked Wed 6 Sep 1995, 12:00:35 Bake Level Trend Current Bake 1. Samples 2. Expert Assessment 3. Computer Imaging 4. System Training 5. Assessment System Arnott s Milk Coffee 6 Figure 3: Training the bake assessment system. Table 1: The nine-point assessment scale. Scale Interpretation Point 1 Extremely under baked 3 Just acceptable under bake 5 Ideal bake colour 7 Just acceptable over bake 9 Extreme over bake 5. Performance Results Assessment systems have been trained for ten products, covering a range of crackers and other types of biscuits. During training, a portion of the data is held out for final testing. The output of the trained neural network is compared with the average of the expert assessments on the hold-out set to determine assessment performance. We measured the RMS prediction error as the RMS difference between the average expert assessment and the neural network s prediction. We also measured the expert s consistency as the RMS of the standard deviations of the paired assessments obtained for each product sample. On the ten products considered to date, the expert s consistency ranged from a maximum of 1.1 (corresponding to a confidence interval of ±2.2 points) to a minimum of 0.5. On the same products, the RMS prediction error ranged from a maximum of 0.8 (corresponding to a confidence interval of ±1.6 points) to a minimum of 0.4. However, much of the prediction error resulted from the residual expert inconsistency in the training data. When this residual error was compensated for, the RMS prediction error ranged from a maximum of 0.4 to a minimum of 0.1. Since the range of acceptable product is four points on the nine-point scale, it is desirable for an assessment system to be accurate to within ±1 point on the scale. The results indicate that our system has achieved this goal.

5 6. Conclusion Computer image analysis is becoming widely used in the food industry for quality assurance. We have presented a system comprising colour digital imaging and artificial neural networks which is capable of assessing the bake colour of baked goods. The system achieves a precision significantly greater than that achieved by human experts. References Almela, L., S. Javaloy, J. A. Fernandez-Lopez, and J. M. Lopez-Roca. Comparison between the tristimulus measurements Yxy and L* a* b* to evaluate the colour of young red wines. Food Chemistry, 53: , Grenander, U., and K. M. Manbeck. A stochastic shape and color model for defect detection in potatoes. J. Comp. Graph. Stat., 2: , Guevara, R. G.-L., and J. E. Pardo-González. Evolution of color during the ripening of selected varieties of paprika pepper (Capsicum annuum L.). J. Agric. Food Chem., 44: , Hunter, R. S. The Measurement of Appearance. Wiley-Interscience, Ling, P. P., V. N. Ruzhitsky, A. N. Kapanidis, and T.-C. Lee. Measuring the color of food. Chemtech, 26:46-53, Mabon, T. J. Color measurement of food. Cereal Foods World, 38:21-25, Pattee, H. E., F. G. Giesbrecht, and C. T. Young. Comparison of peanut butter color determination by CIELAB L* a* b* and Hunter color-difference methods and the relationship of roasted peanut color to roasted peanut flavor response. J. Agric. Food Chem., 39: , RayChaudhuri, T., J. C.-H. Yeh, L. G. C. Hamey, and C. T. Westcott. Baked product classification with the use of a self-organising map. Proc. Australian Conf. Neural Networks, pp , Yeh, J. C.-H., L. G. C. Hamey, C. T. Westcott, and S. K. Y. Sung. Colour bake inspection system using hybrid artificial neural networks. Proc. IEEE Int. Conf. Neural Networks, pp , Westcott, C. T., and L. G. C. Hamey. Data Recognition System. Patent specification PCT/AU95/00813, Arnott s Biscuits Ltd, 1995.