RESPONSE OF 10 PULSE OXIMETERS TO AN IN VITRO TEST SYSTEM

Transcription

1 British Journal of Anaesthesia 1992; 68: RESPONSE OF 10 PULSE OXIMETERS TO AN IN VITRO TEST SYSTEM K. J. REYNOLDS, J. T. B. MOYLE, M. K. SYKES AND C. E. W. HAHN SUMMARY Pulse oximeters are often used in situations in which severe hypoxaemia may occur. We have developed an in vitro system to test the accuracy of pulse oximeter calibration. The probe of10 different oximeters was attached to a model finger in an in vitro blood circuit, and pulse oximeter readings (Sp Oi ) were compared with multi-wavelength in vitro oximeter readings (SoJ over a range of So 2 values from to 100%. The oximeters tested varied widely in their accuracy and linearity. We conclude that the system can test the accuracy, reproducibility and linearity of response of pulse oximeter readings at low oxyhaemoglobin saturations. KEY WORDS Equipment: pulse oximeters. Measurement techniques, pulse oximetry Pulse oximeters determine the oxygen saturation of arterial blood by a non-invasive optical technique. They are calibrated using data obtained by inducing hypoxaemia in healthy adult subjects. This is achieved easily at high saturations, but at saturations less than approximately 75 %, the acquisition of data becomes more difficult and possibly unethical. Data are limited, therefore, at these low saturations, and calibration is mainly by extrapolation. As calibration is performed individually by each manufacturer for his own instrument, the data used for calibration have been obtained from several subject population types and sizes. It is likely, therefore, that accuracy differs widely between different instruments, especially at low saturations. This has been confirmed in extensive in vivo studies by Severinghaus and colleagues [1,2]. We have developed an in vitro system for testing the performance of pulse oximeters over a wide range of saturations. This study determined the response of a range of different pulse oximeters to the in vitro test system. circuit by a computer-controlled peristaltic pump. The desired waveform and pulse rate of blood in the circuit are fed into the microcomputer, and appropriate control pulses are applied to a stepper motor driving the pumphead. Blood is pumped in pulsatile fashion through a cuvette acting as a model finger (fig. 2) and the probe of the pulse oximeter being tested is positioned across the finger. As it is impossible to construct a model finger reproducing exactly the anatomical features of a real finger, we aimed to keep the design of the cuvette as simple as possible (fig. 2). Blood enters from one end of the finger and flows in a thin Computer controlled pumphead Pulse oximeter Pulse oximeter probe FtG. 1. The pulse oximeter test system. Gas supply Membrane oxygenator METHODS Test system An in vitro test system was designed capable of producing signals similar to those obtained from pulsatile arterial blood in the finger (fig. 1). Whole blood obtained from adult subjects and anticoagulated with heparin is pumped through the K. J. REYNOLDS*, B.A., M.SC. ; J. T. B. MOYLE, M.B., B.S., I.ENG., M.INST.M.C, M.I.ELEC.I.E., F.C.ANAES.; M. K. SYKES, M.B., B.CHIR., F.C.ANAES., F.F.A.R.A.C.S. (HON.), F.F.A.(S.A-) (HON.); C. E. W. HAHN, M.A., B.SC, M.sc. D.PHiL.; Nuffield Department of Anaesthetics, John Raddiffe Hospital, Oxford. Accepted for Publication: November 2, Present address, for correspondence: c/o Physiology Department, University of Leicester, University Road, Leicester LEI 7RH.

2 366 BRITISH JOURNAL OF ANAESTHESIA Light from pulse oximeter probe TABLB I. Pulse oximeters tested, probe used, data points measured and number of units of blood used Pulse oximeter Type of probe No. data points Blood used (units) Black Clear Perspex Perspex Blood in Pulsating blood channels Silicon rubber membranes Datex Satlite (DA) Nellcor N-200 (NE) Ohmeda Biox 3700 (OH) Physiocontrol Lifestat 1600 (PH) Radiometer OX1 (RA) SiMed S-100 (SI) MiniOX IV (MI) Novametrix 515A (NO) Kontron 7840 (KO) Hewlett-Packard Monitor (HP) Blood out FIG. 2. The model finger used in the test system. Top: side view; below: face view. (1-mm) layer through the cuvette, over the finger tip end and back in a similar fashion along the underside. Both inlet and outlet are tapered to prevent flow separation. The silicone rubber membranes (0.5 mm thick) allow blood pulsations within the model finger. The a.c. :d.c. signal ratio may be altered by adjusting the gate clamps at points A and B in figure 1. This ratio should be in the physiological range (1:100 to 5:100 [3]). This design enables flow within the model finger to be developed fully within a short distance from each end of the cuvette, and the pulsations produced in the finger to be consistent along its length. This is necessary because the position of the light sources relative to the "fingertip " end of the oximeter probe varies between makes of oximeter, and so for a valid comparison to be made between different oximeters, the pulsations must be consistent at all positions along the cuvette. A Datex Satlite Versalite (ear) probe was positioned in turn at several places along the length of the cuvette, and the saturation value recorded by the pulse oximeter was found to be repeatable at all positions. The overall dimensions of the model finger were similar to those of a real finger, to ensure that all oximeter finger probes fitted snugly over the finger. To provide some scattering of the incident light, a piece of translucent draughtsman's paper was inserted between the light sources in the probe and the model finger. This paper, together with the silicon rubber membranes and the red blood cells, causes diffusion of the incident light. The saturation of blood flowing through the circuit is controlled by a small parallel plate membrane oxygenator. The blood flows between two parallel microporous polypropylene membranes, each with an effective area of 6.4 x 16.0 cm 2. Nitrogen, oxygen and carbon dioxide pass by diffusion across the membranes. The composition of the gas mixture passing through the membrane oxygenator is controlled by flowmeters. The percentage of carbon dioxide is maintained at 6 %, while the percentage of oxygen is varied; the balance of the gas mixture is nitrogen. Finally, the blood enters a small reservoir which damps the pulse wave completely before it returns to the pump. The blood in the reservoir is exposed to the same mixture of gases as that passing through the membrane oxygenator in order to maintain equilibrium. Experimental methods Ten pulse oximeters from different manufacturers (table I) were used to test the in vitro system. The probe of each pulse oximeter was placed over the model finger. Blood in the circuit was desaturated to approximately % by passing a mixture of 2 % oxygen, 6% carbon dioxide, balance nitrogen, through the membrane oxygenator. When equilibrium was reached and the pulse oximeter reading became stable, measurements displayed by the pulse oximeter (5p Ot ) were compared with blood samples withdrawn simultaneously from a sampling port immediately next to the outlet of the model finger. The oxygen saturation (So 2 ) of the blood samples was measured immediately using a co-oximeter (OSM3 Hemoximeter, Radiometer Ltd). The Hemoximeter uses six wavelengths of light to measure SOj directly, and is capable, therefore, of distinguishing the abnormal haemoglobins, carboxyhaemoglobin and methaemoglobin. The concentration of oxygen in the gas mixture was increased in 1 % steps, and 5p o, and So t readings recorded when equilibrium was reached at each saturation. A least-squares linear regression analysis was performed on the data from each pulse oximeter and the correlation coefficients calculated. As pulse oximeters are generally calibrated from empirical data for saturations greater than approximately 75 % and by extrapolation for values less than 75%, we decided also to analyse these two data ranges separately. Another useful method of analysis of the data is by calculation of mean bias, (Sp Oj So,,), and precision, which is the SD of the bias. Mean bias and precision were calculated for each oximeter over all three data ranges.

3 PULSE OXIMETER TEST SYSTEM 367 TABLE II. Linear regression analysis of Sp Oj on So,for the oximeters tested over the saturation ranges, and %. (See table I for details of oximeters.) CI = Confidence interval; SEE = standard error of estimate; n = number of readings Pulse oximeter and probe DA NE OH PH RA SI MI NO KO HP So, range (%) n Total haemoglobin concentration (thb) of the blood was adjusted to g %. Data were obtained also using die Datex Satlite for thb of 8 g % and 14 g%, to see if haemoglobin concentration affected the results. The SD of haemoximeter readings was estimated by analysing 10 samples from 1 ml of blood in a 1-ml syringe. The syringe was capped and placed in ice between readings to prevent the So 2 changing. The measurements were repeated on two further 1-ml samples of blood. Several in vivo measurements were made on hypoxaemic patients in an attempt to confirm the in vitro results. The same pulse oximeters and probes as used for the in vitro measurements were placed at random on the fingers of each patient, and Sp Oj values recorded. Simultaneously, an arterial blood sample was obtained from each patient and analysed in die OSM3 Hemoximeter to obtain So 2. RESULTS Table I shows the number of data points and number of different units of blood used for each pulse oximeter. In all blood samples, the carboxyhaemoglobin and methaemoglobin concentrations were < 1.5%. These small concentrations of abnormal haemoglobins should not affect the accuracy of the pulse oximeter readings. Linear regression, correlation coefficient, mean bias and precision are given in table II for the three saturation ranges %, % and % for each pulse oximeter. Slope (±95% CI) 0.80 (0.04) (0.09) 0.71 (0.12) 0.94 (0.09) 0.94 (0.26) 1.07(0.26) 0.84 (0.04) 0.88 (0.09) 0.82(0.13) (0.06) 0.82(0.16) 1.08(0.21) 0.59 (0.03) 0.72 (0.06) 0.47(0.11) 0. (0.04) 0.68 (0.08) 0.35(0.11) 0.59 (0.04) 0.57 (0.07) 0.62(0.13) 0.65 (0.04) 0.66 (0.08) 0.69(0.13) 0.58 (0.04) 0.76(0.10) 0.46(0.12) 0.49 (0.04) 0.69 (0.05) 0.32(0.10) X 2 (0 Q SEE n Intercept r Bias (mean (SD)) 2.7 (3.7) 1.0(2.0) 5.4 (4.2) 0.2(3.1) -0.5 (3.0) 1.4(2.9) 1.8(3.4) -0.2 (2.0) 3.7 (3.3) 3.5 (2.7) 2.9 (2.2) 4.2 (3.0) 7.6 (6.5) 2.3 (2.5) 13.1 (4.5) 10.2 (7.3) 4.2(3.1) 16.1 (5.0) 11.3(5.9) 6.6 (3.7) 16.0 (3.5) 9.7 (5.0) 5.6 (3.0) 13.7 (2.8) 6.0 (6.2) 0.9 (2.9) 10.7 (4.4) 9.5 (7.6) 3.1 (2.4) 15.8 (5.4) Hemoximeter SOj (%) FIG. 3. Comparison between Datex Satlite pulse oximeter readings (Sp Ot ) obtained from the test system, and Hemoximeter readings (So,). Linear regression lines and data points:, data set So, > 75 %; O, So, < 75 %; indicates identity. In vivo data points: A > 75 %, A < 75 %. The data points for three makes of oximeter are plotted in figures 3-5, to show the variation in response between different oximeters. In vivo results obtained from hypoxic patients are plotted on the same figures to allow comparison with the results obtained from the test system. On each figure, regression lines are drawn for test system data from

4 368 BRITISH JOURNAL OF ANAESTHESIA 100 -i o (D 1 70H Hemoximeter So, (%) FIG. 4. Comparison between SiMed S-100 pulse oximeter readings (5p Ot ) obtained rora the test system, and Hemoximeter readings (So,). (See figure 3 for explanation of symbols.) O > o z 100 -i Hemoximeter So, (%) FIG. 5. Comparison between Novametrix 515A pulse oximeter readings (5p o J obtained from the test system, and Hemoximeter readings (So,). (See figure 3 for explanation of symbols.) the two saturation ranges % (dashed line) and % (solid line). These two lines are useful to show the linearity of results between the two data ranges. All pulse oximeters tended to over-read; this effect was greater at low saturations (table II). The most extreme results were produced by the SiMed (fig. 4) and the HP Monitor, both of which produced readings of approximately 75 % for a true saturation of %. The correlation coefficient over the entire range measured ( %) was Ss 0.94 for all instruments. The correlation coefficient over the range % for nine of the 10 instruments was ^. Over this range, the Nellcor produced a correlation coefficient of This smaller value was attributable to the smaller number of points in the analysis, and one outlying point, rather than to any lesser reproducibility of the results. Correlation at lesser saturation was less good, with six pulse oximeters producing correlation coefficients of < 0.90 in the % range. In all cases (except the Nellcor), the residual variance (or scatter) was greater in the lesser saturation range. Each pulse oximeter was tested using at least five different units of blood on different days (table I). The results for each oximeter were consistent in repeated sets, which is a necessary requirement for a performance test system. No significant difference was found between results using the Datex Satlite from blood samples with haemoglobin concentrations of 8 g % and 14 g%. However, scatter of the results can be attributed partly to the SD of 5o 2 measurement, which was 0.4 %. DISCUSSION All the pulse oximeters showed a tendency to over-read at low saturations. This is confirmed by the results of the regression analysis, with regression coefficients for the % So i range varying between 0.49 for the HP Monitor and for the Physiocontrol, and positive values for the mean bias for all pulse oximeters over the range %. This tendency to over-read was noticed also by Sidi and colleagues using an Ohmeda Biox 3700 and a Nellcor N-100 on hypoxic dogs [4]. In contrast, in vivo measurements made by Severinghaus on healthy adults subjected to profound hypoxia, gave values for regression coefficients between 0.86 and 1.42, and negative biases on several occasions [2], It was not possible for us to obtain a complete set of in vivo results to confirm our in vitro results. However, a small number of readings were obtained from hypoxaemic patients with arterial saturations as little as 56 %. These in vivo points are plotted with the in vitro results for the three oximeters in figures 3 5. The results confirm our in vitro results, with all oximeters over-reading to some degree at low saturations. For example, a range of pulse oximeter readings from 69% (Ohmeda) to 84% (SiMed) (fig. 4) was obtained from a patient with an arterial saturation of 68.0 %. As oximeters are calibrated mainly by extrapolation of data at low saturations, calibration in this range is speculative. For this reason, the two ranges greater and less than 75 % were analysed separately. In order to assess the linearity of response of the pulse oximeters, regression coefficients of the entire saturation range (%), the "calibrated" range (%), and the "extrapolated" range ( %) were compared, with 95 % confidence intervals for the coefficients (table II). For six of the 10 pulse oximeters, all three coefficients coincided within 95 % confidence limits. Four pulse oximeters, however, (RA, SI, KO, HP) had a significant difference between the regression coefficient in the % So, range and that in the % range. This suggests non-linearity between the calibrated range and the uncalibrated range obtained by extrapol-

5 PULSE OXIMETER TEST SYSTEM 369 ation. An example of this non-linearity of results is shown in figure 4 for the SiMed pulse oximeter. In contrast, results presented in figures 3 and 5 for two other pulse oximeters showed no significant nonlinearity. The increase in scatter at low saturations is in agreement with in vivo observations [2, 5]. A possible explanation for this increasing scatter at low saturations may be as follows. The relatively large extinction coefficient of reduced haemoglobin at 660 nm implies that, as saturation decreases, less red light is transmitted through blood, and it appears darker. Any errors in the transmitted light signal at the red wavelength are therefore increased relative to the red signal at low saturations, which leads to a greater variance in 5p o, at these saturations. Indeed, we found that, if the saturation of the blood in the circuit was reduced to much less than %, in some cases the pulse oximeter would register a "noisy signal" warning, suggesting that the errors in the red signal were becoming large relative to the magnitude of the signal itself. The use of a system such as this may enable manufacturers to reassess and improve the calibration algorithms for their oximeters. However, the results presented in this paper are intended to be used as an assessment of the test system, and should not be extrapolated to a validation of the pulse oximeters themselves, as only one unit from each manufacturer was used. (Severinghaus [2], for example, found large variations between different sets of results using different instruments from the same manufacturer.) Also, die calibration algorithms are continually being updated by the manufacturers, and die oximeters used in our study are not necessarily the latest models and software versions. Kagle and colleagues demonstrated a significant difference between results obtained from an Ohmeda 3700 Version J software and diose from the same make of oximeter but containing modified Version XJ1 software [6]. These studies investigated die use of an in vitro system to compare pulse oximeter accuracy. The test system was used under laboratory conditions. Pulse oximeters are known to be affected by a large number of external factors, for example response time [1], accuracy at low perfusion [7, 8], interference by external electromagnetic sources [9, 10] and effect of motion [11]. The test system is potentially able to test the response of pulse oximeters to all diese artefacts. ACKNOWLEDGEMENTS We thank the Department of Health for generous support, and the manufacturers for providing the pulse oximeters. REFERENCES 1. Severinghaus JW, Naif eh KH. Accuracy of response of six pulse oximeters to profound hypoxia. Anesthesiology 1987; 67: Severinghaus JW, Naifeh KH, Koh SO. Errors in 14 pulse oximeters during profound hypoxia. Journal of Clinical Monitoring 1989; S: Huch A, Huch R, K6nig V, Neuman MR, Parker D, Yount J, Lubbers D. Limitations of pulse oximetry. Lancet 1988; 1: Sidi A, Rush W, Gravenstcin N, Ruiz B, Paulus DA, Davis RF. Pulse oximetry fails to accurately detect low levels of arterial hemoglobin oxygen saturation in dogs. Journal of Clinical Monitoring 1987; 3: Ridley SA. A comparison of two pulse oximeters. Assessment of accuracy at low arterial saturation in paediatric surgical patients. Anaesthesia 1988; 43: Kagle DM, Alexander CM, Berko RS, Giuffre M, Gross JB. Evaluation of the Ohmeda 3700 pulse oximeter: steady-state and transient response characteristics. Anesthesiology 1987; 66: Clayton DG, Webb RK, Ralston AC, Duthie D, Runciman WB. A comparison of the performance of 20 pulse oximeters under conditions of poor perfusion. Anaesthesia 1991; 46: Falconer RJ, Robinson BJ. Comparison of pulse oximeters: accuracy at low arterial pressure in volunteers. British Journal of Anaesthesia 1990; 65: Costarino AT, Davis DA, Keon TP. Falsely normal saturation reading with the pulse oximeter. Anesthesiology 1987; 67: Block FE. Interference in a pulse oximeter from a fiberoptic light source. Journal of Clinical Monitoring 1987; 3: Langton JA, Hanning CD. Effect of motion artefact on pulse oximeters: evaluation of four instruments and finger probes. British Journal of Anaesthesia 1990; 65: