Clinical Performance of CGMS in Type 1 Diabetic Patients Treated by Continuous Subcutaneous Insulin Infusion Using Insulin Analogs

The study was carried out on 18 adult type 1 diabetic patients, diagnosed according to ADA criteria (8), who were C-peptide negative (<0.3 nmol/l, 6 min after a 1-mg intravenous injection of glucagon). Patients selected for the study had been coming to our outpatient clinic. They were being treated for over 1 year by external pump (Medtronic infusor MMT 506, 507c, and 508; Medtronic Technologies, Northridge, CA) using an insulin analog (Humalog, U-100; Lilly France, Saint-Cloud, France) and a deconnectable catheter (Tender set, Disetronic Medical Systems AG, Burgdorf, Switzerland; Sofset QR, Medtronic Technologies). Insulin was infused into the abdomen and the infusion site was changed every 3 days. Patients aged <18 years and who had a BMI >30 kg/m² and lack of compliance to SMBG were not included in the study.

All the patients recruited to this study were eligible for participation because their blood glucose control had not been obtained (HbA_1c >7.5%) despite intensive insulin therapy using an external pump and a conventional SMBG several times a day. The objective of their hospitalization was to improve glycemic control by adjusting insulin doses and/or diet regimens. The patients had been taught to measure their capillary blood glucose (including preprandial, postprandial, and bedtime periods) using a GlucoTouch memory meter (Lifescan, Roissy, France) to calculate mean blood glucose levels, mean standard deviation of blood glucose, postprandial glucose levels, and incidence of hypoglycemia. Blood glucose concentrations of <55 mg/dl (3 mmol/l) were considered to be indicative of hypoglycemia.

Patients arrived at the hospital, and a 2-h training session was conducted to educate the patients about CGMS. After sensor insertion and initial calibration of the sensor according to Medtronic procedures, the patients were instructed to enter event markers into the monitor (meals, insulin boluses, exercise, and hypoglycemic episodes) for a period of 72 h. Among the 18 type 1 diabetic patients, 9 had one sensor inserted into the abdominal site, while the other 9 patients had two sensors inserted simultaneously into the abdominal site (one on the left, the other on the right). For these nine patients, the two sensors were carefully calibrated at the same time in order to compare their values and to evaluate the reproducibility of the device. The pump catheter that delivered insulin was inserted at least 10 cm away from the abdominal site where the sensor had been inserted. Insulin doses were adjusted during hospitalization. Therapy adjustments by algorithms were based on meter data, but were also partly based on CGMS data downloaded every 24 h.

During continuous glucose monitoring, patients were instructed to measure and enter at least four fingerstick blood glucose values a day to maintain the calibration of the CGMS. During each period of CGMS recording, capillary whole blood sample was collected to check capillary plasma glucose levels with glucose meter at fasting, at each preprandial (before lunch and dinner) and postprandial (1 and 2 h after the meals) period, and at bedtime. Additional blood derived from the same sample was used to determine simultaneously plasma glucose levels with reference glucose oxidase method (see “Biochemical determinations”). The times at which measurements were made were noted precisely so that the corresponding value given by the CGMS could be evaluated. In cases in which symptoms of hypoglycemia appeared, additional measures were performed. Capillary plasma glucose concentrations were obtained using the same glucose meter for all the patients (GlucoTouch; Lifescan). At the end of the CGMS recording period, the data were downloaded from the monitor and the results were analyzed (Medtronic Download program 1.7A).

Blood samples were taken after a 12-h overnight fast. Glycated hemoglobin (HbA_1c) was measured by high-performance liquid chromatography on Biorex resins (BioRad, Richmond, CA; normal range 4–6%). All the hematocrits of patients’ blood samples were analyzed. Capillary samples were collected into a microtube containing fluoride and then immediately centrifuged (5 min at 18,000g). Plasma glucose concentrations from capillary whole blood samples were assayed using a reagent based on the glucose oxidase and peroxidase enzymatic method (PAP 7500; BioMérieux SA) using a Beckman analyzer (Beckman Glucose Analyzer; Beckman, Fullerton, CA). The samples were taken at fasting, at each preprandial and postprandial period, and at bedtime. The coefficient of variation of this reference method was <3%. SMBG determinations using glucose meter (GlucoTouch) were realized simultaneously (nine times/day).

The performance of the CGMS was evaluated by comparing its readings to those obtained at the same time by the glucose oxidase method using the Beckman analyzer and by comparing them to the SMBG readings (GlucoTouch) performed by the patients. Different methods of agreement were used, as follows:

1) The criteria of optimal accuracy defined by Medtronic Technology have been established as follows (7): a correlation between the sensor and meter readings (GlucoTouch) of at least 0.79 and a mean absolute error of no more than 28%. For each day of data collection, the correlation coefficient between the meter readings (GlucoTouch) and the glucose sensor values was calculated. The mean absolute error was calculated by taking the absolute difference between the meter value (GlucoTouch) and the glucose sensor value, and then by dividing by the meter value and averaging all the pairs of data.

2) Based on an ADA consensus statement (1), we used the recommended performance goal of SMBG: a total analytic error of <10% (present ADA criteria) or <5% (future ADA criteria) (9) at glucose concentrations of 30–400 mg/dl.

3) To assess agreement between the monitors and the reference method (Beckman analyzer), we used the method of residuals, referred to as the Bland and Altman method (10). This is defined as the mean, over all data pairs, of the absolute value of the difference between the CGMS and reference glucose (Beckman analyzer), divided by the reference glucose. The mean ± 1.96 SD represented the 95% CI.

4) Comparisons were also carried out between the reference method (Beckman analyzer) and CGMS and between the reference method and glucose meters (GlucoTouch). In this way, a Clarke Error Grid analysis was performed to assess the clinical impact of treatment decisions that could be based on the results of CGMS readings in comparison with reference results obtained using the Beckman analyzer. The same approach was used to analyze results from the GlucoTouch glucose meter in comparison with reference results obtained using the Beckman analyzer. The Clarke Error Grid separates a typical scatter plot into five zones of clinical significance (11). The zones are defined according to the presence and severity of a treatment error based on the blood glucose assay being evaluated. Zone A represents the absence of treatment error, and zone B represents cases in which the two methods disagree by >20% but still do not lead to a treatment error. Zones C, D, and E represent increasingly large and potentially harmful discrepancies between the evaluation and reference methods. If the new method has a high percentage (>95%) of its pairs in zones A and B, then it is considered clinically acceptable.

5) In terms of accuracy of this system, we analyzed the results by calculating the specificity and sensitivity of the CGMS to correctly detect incidences of hypoglycemia with respect to the comparative plasma blood glucose level determined by the glucose oxidase reference method (Beckman analyzer). Defining hypoglycemia as a value of blood glucose <55 mg/dl, we attributed to each of the CGMS measurements the term of true positive (TP), false positive (FP), true negative (TN), or false negative (FN), according to the simultaneously measured blood glucose value above or below 55 mg/dl. Sensitivity and specificity were defined by the following equations: Sensitivity = TP/(TP + FN), and specificity = TN/(TN + FP).

6) Reproducibility was calculated for sensor values obtained from pairs of two glucose sensors simultaneously inserted and calibrated in 9 of the 18 type 1 diabetic patients.

Data are expressed as means ± SD. The data in figures are shown as means ± SEM. The distribution of variables was tested for approximation to a Gaussian distribution (normality) using the Kurtosis and Skewness tests. Correlation coefficients were used to compare sensor readings when they were simultaneously placed for nine patients. Statistical significance is implied by a value of P <0.05. Statistical analyses were performed using the Statview program (Statview V; BrainPower, Calabasas, CA).

The main clinical characteristics of the type 1 diabetic patients are shown in Table 1. All the hematocrits of the patients were in the normal range from 35.6 to 44.1%, which is within the operative specifications of the glucose meters. Among the 33 glucose sensors, 6 (18%) were nonfunctional in terms of their ISIG (sensor current) value being below 10 nA. Data from a total of 27 glucose sensors was thus downloaded and analyzed corresponding to the nine type 1 diabetic patients who used only one glucose sensor, and the other nine patients who used simultaneously two glucose sensors. One glucose sensor was defective after only 25 h of recording. The mean duration of CGMS recording was 63 ± 12 h (range 25–74). We detected some problems of disconnection between the monitor and the cable (n = 2) and between the cable and the glucose sensor (n = 3). The subcutaneous tolerance of the entire glucose sensor was excellent, without any side effects reported at the site of sensor implantation.

Based on Medtronic performance goals, the agreement between the sensor glucose readings and the meter readings was good. Of the entire 692 pairs of data for glucose meter readings and CGMS, the coefficients of correlation ranged from 0.87 to 0.92 and the mean absolute error ranged from 12.8 to 15.7% (Table 2). The coefficients of correlation and the mean absolute errors were not different for the 3 consecutive days of recording. These paired data were obtained when the values of CGMS ranged from 40 to 400 mg/dl. The mean amount of time per day for which each patient experienced hypoglycemia (<55 mg/dl) was calculated to be 86 ± 62 min.

The percentage of values within ±10% of the reference value allowed classification of the CGMS and glucose meter according to their accuracy. Only 39% of the CGMS values satisfied the ADA precision criteria to within ±10%, while 19% of these values satisfied the future ADA precision criteria for accuracy to within ±5% (for glucose concentrations from 30 to 400 mg/dl). In contrast, 71 and 43% of the meter values satisfied ADA criteria by being within ±10% and ±5% (present and future precision criteria, respectively) of the reference value.

Figure 1 shows the means of differences for the CGMS versus glucose reference methods. This analysis confirmed that the CGMS slightly underestimated the real glucose value as compared with the reference method, which is indicated by the negative mean of differences (mean = −12 mg/dl). In the same way, the GlucoTouch meter underestimated the values of the glucose reference by 14 mg/dl. However, the dispersion of the values (expressed as mean ± 2 SD) around the mean of differences was lower for the glucose meter (34 mg/dl) than with CGMS (84 mg/dl) (Fig. 1).

Error grid analysis of data obtained from the 18 patients showed that 94% of the measurements performed with the glucose meter (n = 175) were in zone A of the error grid and 100% in zones A and B (Fig. 2). In contrast, only 77% of those performed with the glucose sensor (n = 276) were in zone A, and 98.9% were in zones A and B. Furthermore, we found that two values fell in zone C and a single value in zone D with the glucose sensor. No values in zone E were detected.

The values were subdivided into three different clinical targets of glycemia: within target (80–180 mg/dl), below target (<80 mg/dl), and above target (>180 mg/dl). Among these three targets, there was a good correlation between CGMS and the reference glucose value (r = 0.43–0.75, P = 0.02–0.0001). We found that glucose meter values were in accordance with the glucose reference values more frequently than values obtained from the glucose sensor (90.7 vs. 75.6%, P = 0.0003).

We recorded 276 paired values of blood plasma glucose to analyze the sensitivity and specificity of the CGMS for hypoglycemia detection. When compared with plasma blood glucose, with the limit of 55 mg/dl as the definition of hypoglycemia, we divided results into four quadrants (see Fig. 3) and reported 260 TN, 4 FN, 10 FP, and 2 TP (see research design and methods). From these data, values for sensitivity and specificity of CGMS for hypoglycemia detection were 0.33 and 0.96, respectively. Hence, when the low-glucose alert was set at 55 mg/dl, the CGMS correctly identified 33% (2/[2 + 4]) of the hypoglycemic events. In addition, 96% (260/[260 + 10]) of the time the CGMS correctly identified a value ≥55 mg/dl in the absence of hypoglycemia.

The inter-sensor coefficient of variation was evaluated by the two simultaneous CGMS sensors placed in nine patients. A total of 6,666 paired values were recorded with a coefficient of correlation of 0.84 between the paired sensors (Fig. 4). The coefficient of variation was evaluated to be 8.25%.

We have evaluated the clinical performance of CGMS in patients under intensive therapy for diabetes, which requires the use of an accurate and reproducible glucose-measuring device for therapy adjustments. We have shown that the CGMS gives much more information than intermittent SMBG; however, CGMS cannot be used as a replacement for glucose meters because it does not satisfy conventional goal performance criteria for in vitro glucose measurements and can lead to clinically incorrect treatment decisions. Our study is the first to analyze the performance of CGMS in comparison with a reference method over several days of real-life conditions.

In our study, technical results and the performance criteria as outlined by Medtronic were in accordance with those of previous studies (5,12,13). As defined in these performance criteria, CGMS appears to be an accurate device for measuring glucose levels on a continual basis. However, there is some controversy over what methods should be used to analyze the accuracy of these systems; we performed an evaluation of CGMS using different and complementary methods. We have intentionally not used the Spearman’s correlation test because the r coefficient measures the extent to which two sets of data fit a linear relationship, not the consistency between data (14,15). According to the ADA consensus statement, the performance goal of the CGMS was very low and inferior to that of glucose meters. Graphical presentation of the results using the Bland and Altman method showed that the dispersion of values against a reference method was greater with CGMS than with the glucose meter. The Error Grid Analysis elaborated by Clarke and colleagues is the sole method that directly affects patients; this method categorizes individual blood glucose values by possible clinical consequences of inaccuracy in the reported values (16,17). As shown by Clarke Error Grid analyses, only 77% of the values of CGMS were in zone A, whereas 94% of the pairs of reference method glucose values fell in the same zone. However, CGMS satisfies the criterion of having at least 95% of the values in zones A and B, due probably to the low rate of cases of hypoglycemia recorded during the study. Finally, the sensitivity of CGMS to detect hypoglycemia was low, as was the reproducibility of the measurements.

The lack of reproducibility of the CGMS suggests difficulties to perform successively two or more recordings by the CGMS. The reproducibility of glucose measurements using the CGMS has been recently analyzed by comparing data provided by two sensors worn simultaneously by 11 diabetic patients (13). However, this study included not only type 1 diabetic patients, but also type 2 diabetic patients and healthy control subjects. Moreover, according to the number of patients included, >3,370 data pairs were recorded in which two sensors were worn simultaneously. In our study, with only nine patients, we recorded twice as many data pairs, suggesting poor accuracy and/or many technical problems associated with analysis of the reproducibility of the CGMS in the study of Metzger et al. (13). Another explanation is that we took advantage of recent improvements in CGMS software, which has been upgraded to resolve the problem of the “midnight shift,” whereas this was not the case in the study by Metzger.

Other devices for invasive or noninvasive glucose measurements have also been recently analyzed. Concerning the detection of hypoglycemia, the Glucowatch Biographer does not appear as an accurate device according to the ADA criteria (18). In a recent study, the sensitivity of Glucowatch to detect hypoglycemic episodes was only 24% when considering 3.9 mmol/l as the threshold of hypoglycemia, whereas specificity was 99% (19). In contrast, a study carried out to evaluate the accuracy of Glucoday, a microdialysis fiber inserted subcutaneously, showed that 97% of the 381 data pairs fell in the A and B regions of the error grid analysis (20). It is worth noting that this study compared the new subcutaneous glucose sensor to the glucose oxidase (Beckman) reference method. The results obtained “online” by the microdialysis method demonstrated that the system was extremely sensitive in identifying clinical hypoglycemia.

It has been demonstrated that the performance of CGMS also depends on subgroups within the studied population (21). CGMS performance statistics can be strongly influenced by the level and range of the standard blood glucose meter values used to calculate them, independently of the quality of recording or of potential technical or local problems with the sensors used. Specifically, the magnitude of the correlation coefficient is directly related to variations in the meter values. In general, the coefficients of variation were excellent between CGMS data and glucose meter data in type 1 diabetic patients. Patients with type 2 diabetes and pregnant patients have significantly less variable glucose levels when compared with patients with type 1 diabetes (21). This could explain the differences in CGMS performance statistics for these population subgroups. Consequently, it appears inappropriate to simultaneously analyze the performance of CGMS for these different groups.

We demonstrated that CGMS underestimated the plasma glucose values when compared with the reference glucose oxidase method using a Beckman analyzer. This is an important point because several studies have emphasized the clinical accuracy of this device, particularly in identifying the incidence of hypoglycemic episodes while maintaining fair blood glucose control (22,23). The fact that CGMS underestimates plasma values is not surprising because it has been recently demonstrated that the concentration of glucose in the interstitial fluid does not accurately reflect blood glucose levels. Several studies revealed frequent and prolonged periods of asymptomatic hypoglycemia (sometimes unawareness) in insulin-treated patients when they were monitored for 3 or more days by subcutaneous glucose sensor (23,24). They reported a long and prolonged period of hypoglycemia, especially during the night, of 2 h or more per night in almost 70% of children (23). But, it is difficult to explain how these patients with a recent history of diabetes could experience such long periods of asymptomatic hypoglycemia. In our study, we found that the diabetic patients had a mean period of hypoglycemia recorded by CGMS of 86 min per day.

The concentration of glucose in the interstitial fluid does not always accurately reflect blood glucose levels due to the possibility of a significant time lag between the two (6). Venous plasma glucose is close to interstitial fluid glucose under steady-state conditions in healthy subjects (25); however, the time lag between interstitial fluid and plasma glucose depends on the rate of glucose variations. During oral glucose tolerance tests (OGTT) and hyperglycemic conditions, the lag time was greater than 8 min. Delays in interstitial fluid equilibration observed in the study of Rebrin et al. (6) were typically <10 min. Under basal conditions, the differences between interstitial fluid glucose from adipose tissue and plasma were ∼23 mg/dl, while in hyperglycemic clamp experiments, the difference was found to be ∼47 mg/dl.

During a hyperinsulinemic-euglycemiac clamp, the performance of CGMS remained accurate during the phase of euglycemia and then of hypoglycemia (3.1 mmol/l), but remained 20% lower than plasma glucose levels (26). In addition, during recovery from hypoglycemia, sensor readings lagged behind increases in plasma glucose but remained 15% lower than plasma glucose levels, and never returned to the steady state level of 8.6 mmol/l even 60 min after recovering from hypoglycemia.

Cheyne et al. (27) reported on the performance of CGMS during and in recovery from a controlled hyperinsulinemic-hypoglycemic clamp (2.5 mmol/l for 60 min) experiment in eight healthy volunteers. The time taken for the sensor to achieve a reading >4 mmol/l (threshold of euglycemia for this study) after hypoglycemia was delayed by an average of 30 min (range 0–55), suggesting that CGMS may overestimate the degree of and duration of hypoglycemia. Other authors confirm that during the recovery from hypoglycemia, subcutaneous glucose concentrations remain low for a prolonged period of time (28). Our results are in accordance with this phenomenon of underestimation of glucose levels using the glucose sensor.

CGMS might, however, be useful in routine clinical practice to gauge the glucose profile during specific clinical situations. For example, a pilot study conducted in pediatric type 1 diabetic subjects has shown that CGMS was useful to detect abnormal patterns of glycemia (24). In this study, the data provided by CGMS for 3 days gave some information for optimizing glucose control and, consequently, to reduce HbA_1c levels in these patients for a 3- to 6-month time period. Another study conducted in pediatric patients with type 1 diabetes demonstrated the limitations of conventional SMBG (23). In particular, these authors showed that despite an acceptable glucose control observed in these patients, according to the HbA_1c level and target of blood glucose, some of the patients often experienced hypoglycemic episodes. However, no randomized prospective study has proved an efficacy of CGMS in the improvement of glycemic control.

In summary, whatever the method used, the accuracy of CGMS appears too low. CGMS does not satisfy the ADA criteria for in vitro glucose measurements, and it can lead to clinically incorrect treatment decisions, as shown here by Clarke Error Grid analyses. In particular, the use of a glucose sensor for monitoring the insulin treatment of type 1 diabetic patients must take into account the performance limitations of this method, because this device appears to overestimate hypoglycemic episodes. Finally, its reproducibility should be improved in the future when successive monitoring is performed on the same patient. The improvement remains insufficient with regard to most of the official performance goals. However, the CGMS in routine clinical practice is useful for analysis of the glucose profile under real-life conditions, as well as during the nocturnal period to detect dawn phenomenon and unawareness hypoglycemic episodes, as well as postprandial blood glucose excursions, in gauging the influence of physical activity on the glucose profile, and to analyze pharmacological studies (insulin analogs, hypoglycemic agents).

This study was presented as an oral communication at the last “Hypoglycemia meeting,” Assisi, Italy, May 2001.

This article is dedicated to Professor Pierre Drouin, deceased 21 October 2002.