The Accuracy of the FreeStyle Navigator Continuous Glucose Monitoring System in Children With Type 1 Diabetes

The study was conducted by the Diabetes Research in Children Network (DirecNet) at five clinical centers. A data and safety monitoring board and the institutional review boards at each center approved the study protocol, consent form, and assent form. A parent or guardian and each subject aged ≥7 years gave written consent and assent, respectively.

Eligible subjects were between 3 and 18 years old with a clinical diagnosis of type 1 diabetes of ≥1-year duration. Subjects initially used a Navigator that was blinded so that glucose values could not be seen for approximately 1 week at home. During this time, glucose levels were checked with the built-in FreeStyle blood glucose meter (FreeStyle meter) at least four times per day, including the three meter measurements during the first 24 h that were used for calibration of the Navigator.

Each subject was then hospitalized for 24 h in a clinical research center (CRC), where a second Navigator sensor was inserted. Venous blood samples for laboratory serum glucose concentration determinations were taken every 30 min during the hospitalization. Additional samples were taken every 20 min during a session in which subjects >7 years of age exercised on a treadmill for four 15-min sessions of moderate intensity interspersed with three 5-min rest periods (75 min total) and every 10 min following breakfast (during which time the breakfast insulin dose was delayed) for subjects whose age and weight permitted additional blood samples. Serum glucose concentrations from these samples were measured at the DirecNet Central Biochemistry Laboratory at the University of Minnesota using a hexokinase enzymatic method (5).

Following the CRC admission, the Navigator was used at home for 13 weeks. Subjects were instructed to use the Navigator continuously. During the first 2 weeks, subjects were to check the glucose level with the FreeStyle meter whenever the Navigator alarmed; thereafter, meter glucose checks were at the discretion of the subject and parent. Navigator and FreeStyle glucose data were downloaded weekly to the subject’s home computer and e-mailed to the study coordinating center.

Accuracy analyses were performed separately for inpatient and outpatient Navigator use. For the inpatient analysis, accuracy was evaluated separately for the half-hour measurements and for the exercise and breakfast testing. Laboratory serum glucose values were used as the reference during the inpatient CRC visit, and FreeStyle glucose measurements (excluding those used to calibrate the Navigator) were used as the reference during home use. In a prior study, we found that the FreeStyle meter had a high degree of accuracy (6). Each reference glucose value was paired to the closest Navigator reading within ±5 min. The following were computed for each Navigator reference pair: difference (Navigator value minus reference value), absolute difference (AD; absolute value of difference), and relative absolute difference (RAD; AD divided by reference value, expressed as a percentage). The difference measure incorporates the direction of the error so that pairs with the sensor reading “high ” cancel out pairs with the sensor reading “low.” The median difference therefore evaluates whether there is any bias for the sensor to read systematically high or low. The AD and RAD values use the absolute value of the difference between the sensor value and the reference value, ignoring the direction of the error. These measures reflect the magnitude of the error without regard to whether the sensor value was higher or lower than the reference value. Each pair was also evaluated to determine whether the sensor value met the International Organisation for Standardization (ISO) criteria for home glucose meters (for reference glucose value ≤75 mg/dl, meter value within ±15 mg/dl and for reference glucose value >75 mg/dl, meter value within ±20%, hereafter referred to as the ISO criteria) (7). Summary statistics (e.g., median and percentages) were calculated by pooling all paired values. Median values were reported instead of means because of the skewed distribution. The bootstrap technique (resampling subjects with replacement) (8) was used to account for the within-subject correlation in the statistical comparisons and calculation of CIs.

Since point-to-point measures of accuracy do not capture the temporal dimension of near-continuous data, we supplemented these with “event-based” analyses. The glucose excursion during the exercise session (drop from baseline to nadir) for the laboratory reference values and the Navigator were compared. The Navigator nadir glucose was defined as the lowest glucose value from baseline until 30 min following the laboratory nadir (to allow for a possible lag). The rate of change was defined from baseline until the nadir. An analogous analysis was not performed for the postbreakfast period because many subjects still had not reached their peak glucose when reference sampling was stopped 1 h after breakfast.

The average age of the 30 subjects was 11.2 ± 4.1 years (range 4–17), 40% were female, and 93% were Caucasian, 3% Hispanic, and 3% Asian. The mean duration of diabetes was 5.8 ± 3.0 years, and the mean A1C was 7.1 ± 0.6%.

During the CRC admission, there were 1,811 half-hour laboratory glucose measurements paired with glucose measurements from 58 Navigator sensors (not including the exercise session or the postbreakfast assessment). The median number of paired values per subject was 66 (interquartile range 53–69) ranging from 25 to 79. As shown in Table 1, there was no tendency for the Navigator to read systematically higher or lower than the reference glucose (median difference −2 mg/dl [95% CI −7 to +5]; P = 0.34). Overall, the median AD was 17 mg/dl (interquartile range 8–31; 90th percentile, 53 mg/dl), and the median RAD was 12% (6–21%; 90th percentile, 33%), with 74% of sensor values meeting ISO home glucose meter criteria. When the reference glucose was ≤70 mg/dl, the median absolute difference was 14 mg/dl (8–22; 90th percentile, 44 mg/dl). When the reference glucose was 71–180 mg/dl, the median RAD was 13% (6–22%; 90th percentile, 36%). When the reference glucose was >180 mg/dl, the median RAD was 10% (5–18%; 90th percentile, 25%). As would be expected, the AD was greater at higher glucose levels and the RAD was greater at lower glucose levels. Accuracy measures improved slightly when incorporating a 10-min sensor lag (P < 0.001; Table 1). Among the 53 sensors with at least 10 navigator reference pairs, 19 (36%) had a median RAD ≤10%, 23 (43%) a median RAD of 10.1–15%, 6 (11%) a median RAD of 15.1–20%, and 5 (9%) a median RAD of >20%.

Before starting exercise, the median Navigator glucose concentration was 161 mg/dl (interquartile range 118–220) and median reference glucose concentration was 172 mg/dl (122–239). As shown in Fig. 1A, the Navigator accurately measured the magnitude of glucose falls during exercise. The median fall in reference glucose was 91 mg/dl (51–129), and the median absolute difference in this fall between the Navigator and reference was 16 mg/dl (8–29). Although the drop in glucose was generally well tracked by the Navigator, the sensor glucose values lagged behind the reference values causing the Navigator to underestimate the rate of change, particularly in subjects with a rapid fall in glucose during exercise (Fig. 1B). The median time to the nadir was 100 min for the Navigator and 78 min for the reference (Table 2). The reference glucose fell to ≤70 mg/dl during exercise for four subjects (lab values 56, 60, 68, and 70 mg/dl) with corresponding sensor glucose nadirs of 70, 146, 71, and 62 mg/dl, respectively. The sensor with the glucose nadir of 146 mg/dl tracked the drop in the glucose level during exercise, but it provided erroneously high glucose readings throughout. Point-to-point accuracy showed a median (25th–75th percentiles) RAD of 17% (9–27) during exercise (Table 1), which improved to 11% (7–22) when a 10-min lag was assumed.

At home, subjects averaged 137 ± 30 h of Navigator use per week. Excluding calibration values, there were 8,639 paired sensor–FreeStyle meter values from 607 Navigator sensors. As shown in Table 1, the outpatient accuracy results were similar to the inpatient results. Among the 347 sensors with at least 10 navigator reference paired values, 68 (20%) had a median RAD ≤10%, 115 (33%) a median RAD of 10.1–15%, 99 (29%) a median RAD of 15.1–20%, and 65 (19%) a median RAD of >20%.

Accuracy was fairly consistent over 5 days of use in both the inpatient and outpatient settings. Accuracy did not substantially vary according to insertion site (location data only available in inpatient setting) or sex (Table 1). After adjustment for glucose level, accuracy was significantly better at night for both inpatient (RAD P < 0.001; ISO P = 0.002) and outpatient (RAD and ISO P < 0.001) settings. Accuracy was also significantly better for children 14–18 years of age during home use (RAD P = 0.005; ISO P = 0.01) but was not impacted by age during the inpatient visit (RAD P = 0.28; ISO P = 0.16).

During the CRC admission, subjects simultaneously used two Navigator sensors resulting in 1,971 Navigator-Navigator pairs. The median (25th–75th percentiles) RAD between two simultaneous (within ±5 min) Navigator measurements was 13% (interquartile range 6–21), 15% for values ≤70 mg/dl (average of the two Navigator values), 13% for values 71–180 mg/dl, and 13% for values >180 mg/dl.

We found that the overall RAD between Navigator measurements of interstitial glucose concentrations and reference serum glucose levels was similar during inpatient and outpatient assessments with median values of 12 and 14%, respectively. The AD was greater at higher glucose levels, and the RAD was greater at lower glucose levels.

In previous studies, we evaluated the accuracy of the GlucoWatch and CGMS during inpatient use. The median RAD with the GlucoWatch was 16 and 60% of values meeting ISO criteria (3). With the CGMS, the median RAD was 19% with the original sensor and 11% with a newer modified sensor; 53 and 72% of values, respectively, met ISO criteria (4). It is particularly noteworthy that the Navigator, which gives values in real time, was as accurate as the CGMS, which calculates values retrospectively. All other things being equal, retrospective analysis of sensor data generally enhances accuracy, since there are a larger number of meter values to use for calibration than with rolling, real-time calibration algorithms. Studies of other real-time sensors in adults reported a median RAD of 17% during outpatient use for the Guardian real-time CGMS (Medtronic Minimed) (9) and 16% (inpatient and outpatient data pooled) (10) to 20% (inpatient and outpatient data pooled) (11) for the DexCom STS CGMS (DexCom, San Diego, CA).

The Navigator system tracked the drop in blood glucose induced by exercise well, especially with respect to the magnitude of fall in glucose concentration. However, sensor glucose levels lagged behind the blood glucose levels, causing the device to underestimate the true rate of fall in glucose during exercise in some subjects. Navigator readings were as accurate on the 5th day of use as they were on the 1st day. Exploratory analyses within subgroups controlling for the reference glucose concentration found that the Navigator was more accurate at night than during the day. In past studies, we and others (4,12,13) observed that the CGMS, in contrast, was less accurate during the night than during the day, leading to overestimation of the frequency of nocturnal hypoglycemia. This might be due to the potential for biofouling to occur when there is decreased movement or to changes in the subcutaneous circulation or oxygen availability overnight. The Navigator does not appear to have the same susceptibility to low glucose readings overnight, which could be due to differences in the sensor chemistry (the Navigator is less oxygen dependent) or to difference in the biocompatibility or diffusion characteristics of the membranes coating the sensor. During the day, a large number of glucose values were obtained when there were rapid rates of change of the blood glucose, which makes point-to-point comparisons less accurate because of the physiologic lag between interstitial and blood glucose levels. At night there was a much slower rate of change of blood glucose levels, which allowed the interstitial glucose levels to be equilibrated with blood glucose levels at the time the point-to-point comparisons were determined.

The Navigator’s accuracy does not yet approach the accuracy of current-generation home glucose meters but is sufficient to believe that the device has the potential to be an important adjunct to treatment of youth with type 1 diabetes. Clinical trials are needed, however, to truly demonstrate the clinical utility of the Navigator as well as other glucose sensors.

(Listed in alphabetical order with clinical center name, city, and state. Personnel are listed as principal investigator [PI], coinvestigator [I], and coordinators [C].) 1) The Barbara Davis Center for Childhood Diabetes, University of Colorado, Denver, CO: P.C. (PI), Rosanna Fiallo-Scharer (I), Laurel Messer (C), and Barbara Tallant (C); 2) The Department of Pediatrics, University of Iowa Carver College of Medicine, Iowa City, IA: E.T. (PI), Michael J. Tansey (I), Linda F. Larson (C), Julie Coffey (C), and Joanne Cabbage (C); 3) Nemours Children’s Clinic, Jacksonville, FL: Tim Wysocki (PI), Nelly Mauras (I), Larry A. Fox (I), Keisha Bird (C), and Kim Englert (C); 4) The Division of Pediatric Endocrinology and Diabetes, Stanford University, Stanford, CA: Bruce A. Buckingham (PI), D.M.W. (I), Jennifer M. Block (C), Paula Clinton (C), and Kimberly Caswell; 5) The Department of Pediatrics, Yale University School of Medicine, New Haven, CT: S.A.W. (PI), W.V.T. (I), Elizabeth A. Doyle (C), Heather Mokotoff (C), and Amy Steffen (C).

Jaeb Center for Health Research, Tampa, FL: R.W.B., K.J.R., C.K., Dongyuan Xing, M.J.D., Cynthia R. Stockdale, and Judy Jackson.

Michael W. Steffes, Jean M. Bucksa, Maren L. Nowicki, Carol A. Van Hale, and Vicky Makky.

Gilman D. Grave, Mary Horlick, Karen Teff, and Karen K. Winer.

Dorothy M. Becker, Patricia Cleary, Christopher M. Ryan, Neil H. White, and Perrin C. White.

This research was supported by the following National Institutes of Health/National Institute of Child Health and Human Development Grants: HD041919-01, HD041915-01, HD041890, HD041918-01, HD041908-01, and HD041906-01. Clinical centers also received funding through the following General Clinical Research Center Grants: M01 RR00069, RR00059, RR 06022, and RR00070-41. Abbott Diabetes Care, Alameda, California, provided the FreeStyle Navigator CGMS and the FreeStyle blood glucose meter test strips.

Appreciation is expressed for the work performed by the CRC Nurses at the five clinical centers.