OBJECTIVE—The metabolic syndrome has been promoted as a method for identifying high-risk individuals for type 2 diabetes and cardiovascular disease (CVD). We therefore sought to compare this syndrome, as defined by the National Cholesterol Education Program, to the Diabetes Predicting Model and the Framingham Risk Score as predictors of type 2 diabetes and CVD, respectively.
RESEARCH DESIGN AND METHODS—A population-based sample of 1,709 initially nondiabetic San Antonio Heart Study (SAHS) participants were followed for 7.5 years, 195 of whom developed type 2 diabetes. Over the same time interval, 156 of 2,570 SAHS participants experienced a cardiovascular event. A population-based sample of 1,353 initially nondiabetic Mexico City Diabetes Study (MCDS) participants were followed for 6.5 years, 125 of whom developed type 2 diabetes. Baseline measurements included medical history, age, sex, ethnicity, smoking status, BMI, blood pressure, fasting and 2-h plasma glucose levels, and fasting serum total and HDL cholesterol and triglycerides.
RESULTS—The sensitivities for predicting diabetes with the metabolic syndrome were 66.2 and 62.4% in the SAHS and the MCDS, respectively, and the false-positive rates were 27.8 and 38.7%, respectively. The sensitivity and false-positive rates for predicting CVD with the metabolic syndrome in the SAHS were 67.3 and 34.2%, respectively. At corresponding false-positive rates, the two predicting models had significantly higher sensitivities and, at corresponding sensitivities, significantly lower false-positive rates than the metabolic syndrome for both end points. Combining the metabolic syndrome with either predicting model did not improve the prediction of either end point.
CONCLUSIONS—The metabolic syndrome is inferior to established predicting models for either type 2 diabetes or CVD.
The metabolic syndrome has been promoted recently as a method of identifying individuals at increased risk of both type 2 diabetes and cardiovascular disease (CVD). This syndrome, first described in 1988 by Reaven (1), who called it Syndrome X, consists of obesity (especially abdominal obesity), insulin resistance, impaired glucose metabolism, dyslipidemia of the high triglyceride/low HDL cholesterol type, and elevated blood pressure. Although the syndrome is of considerable importance in understanding the pathophysiology and biochemistry of an interrelated cluster of diabetes and cardiovascular risk factors, recent attempts to inject it into clinical practice may be premature. The metabolic syndrome is an asymptomatic disorder. Thus, its clinical significance is presumably due to its ability to identify individuals for preventive treatments that they might otherwise not receive. The question then arises whether the metabolic syndrome represents an improvement over currently available methods of identifying such individuals.
Recently, two definitions of the metabolic syndrome have been proposed, one by the National Cholesterol Education Program Adult Treatment Panel III (NCEP ATP-III) (2) and the other by the World Health Organization (WHO) (3,4). The NCEP ATP-III definition is more frequently used because, unlike the WHO definition, it requires neither an oral glucose tolerance test nor measurements of fasting insulin or microalbuminuria. In this work, we compare the ability of the metabolic syndrome, as defined by the NCEP ATP-III criteria, to predict type 2 diabetes and CVD with two established predicting models: the Diabetes Predicting Model (5) and the Framingham Risk Score (6). The Diabetes Predicting Model estimates the 7- to 8-year likelihood of developing type 2 diabetes based on a subject’s age, sex, ethnicity (Hispanic or non-Hispanic white), fasting glucose, systolic blood pressure, HDL cholesterol, and history of a parent or sibling with diabetes (5). The Framingham Risk Score estimates the 10-year likelihood of developing coronary heart disease based on a subject’s age, sex, total cholesterol, cigarette smoking status, HDL cholesterol, systolic blood pressure, and presence or absence of diabetes (6).
RESEARCH DESIGN AND METHODS
Subjects were categorized as having the metabolic syndrome if they met at least three of the NCEP ATP-III criteria: waist circumference >102 cm (>40 in) in men and >88 cm (>35 in) in women, triglyceride concentration ≥1.70 mmol/l (150 mg/dl), HDL cholesterol <1.03 mmol/l (<40 mg/dl) in men and <1.29 mmol/l (<50 mg/dl) in women, blood pressure ≥130/≥85 mmHg or on antihypertensive medication, and fasting glucose ≥6.1 mmol/l (≥110 mg/dl). Because the Diabetes Predicting Model was developed in the San Antonio Heart Study (SAHS) (5) and predicting models usually perform better in the dataset in which they were developed, this model was also compared with the metabolic syndrome in an independent dataset, the Mexico City Diabetes Study (MCDS) (7). The Framingham Risk Score was tested relative to the metabolic syndrome in the SAHS because cardiovascular end points as well as diabetes were documented in this study. Both studies were approved by the institutional review board of the University of Texas Health Science Center at San Antonio and the MCDS was, in addition, approved by the institutional review board of the Centro de Estudios en Diabetes in Mexico City. All subjects signed informed consents.
SAHS
The SAHS is a population-based study of 3,301 Mexican Americans and 1,857 non-Hispanic whites, 25–64 years of age at baseline, randomly selected from three types of neighborhoods in San Antonio, Texas: a low-income “barrio,” a middle-income neighborhood, and a high-income suburb (5,8). The total of 5,158 subjects, representing a response rate of 65% of all eligible participants from selected households, were enrolled in two phases, the first from 1979 to 1982 and the second from 1984 to 1988. Among the 4,998 surviving participants, 3,682 (74%) came to a follow-up examination 7 to 8 years after the baseline examination. Since waist circumference, which is needed to define the metabolic syndrome according to the NCEP ATP-III criteria (2), was measured only in the second phase of the study, the analyses presented here are confined to the 2,941 individuals enrolled in that phase. Of these individuals, 2,569 were confirmed to be free of diabetes at baseline (defined as fasting plasma glucose ≥7.0 mmol/l [≥126 mg/dl] or plasma glucose 2 h after a standardized oral glucose load ≥11.1 mmol/l [≥200 mg/dl] [3] or receiving antidiabetic medication at the time of the clinic visit). Of these, 39 were lacking one or more variables needed to define either the metabolic syndrome or to calculate the Diabetes Predicting Model score, leaving 2,530 subjects. Diabetes status after 7–8 years of follow-up was known for 1,709 of these individuals, 195 of whom met the above-mentioned criteria for diabetes.
To examine CVD incidence in the 2,941 individuals who participated in the phase 2 baseline examination, we excluded 95 individuals who were confirmed to have CVD at baseline (defined as self-reported physician diagnosis of heart attack, revascularization procedure, or stroke) and 88 who were lacking one or more variables needed to define either the metabolic syndrome or the Framingham Risk Score, leaving 2,758 subjects. Incident CVD, defined as the above plus CVD death (ICDA codes of 390–459 on the death certificate), was ascertained after 7–8 years of follow-up on 2,570 of these individuals through mortality surveillance and, for nonfatal events, either a home interview or a follow-up medical examination in a clinic (8). One hundred fifty-six of these 2,570 individuals experienced a CVD event during the follow-up period.
MCDS
The MCDS was carried out in six Mexico City “áreas geoestadisticas básicas” (equivalent to U.S. census tracts) (7). A complete enumeration of these areas identified 3,326 study-eligible individuals defined as men and nonpregnant women 35–64 years of age. Of these, 2,813 (85%) completed a home interview and 2,282 (69%) completed a baseline medical examination in a clinic from 1990 to 1992. Of these, 1,922 were confirmed to be free of diabetes, defined as above, at baseline, 43 of whom were lacking one or more variables needed to define the metabolic syndrome or to calculate the Diabetes Predicting Model score, leaving 1,879 subjects. Of these, 1,353 individuals participated in a follow-up examination an average of 6.3 years later. One hundred twenty-five of these individuals developed diabetes during the follow-up period.
Statistical methods
We computed the sensitivities, false-positive rates, and odds ratios for predicting diabetes with the metabolic syndrome as defined by NCEP ATP-III in the SAHS and the MCDS and for predicting CVD in the SAHS. Using the diabetes predicting equation given in our prior publication (5) and the Framingham predicting equation reported in the appendix in the article by Wilson et al. (6), we computed the logit of diabetes risk (logarithm of the odds of developing diabetes) for the SAHS and MCDS participants and the logit of coronary heart disease risk for the SAHS participants. In the remainder of this work, we will refer to these logit values as the Diabetes Risk Score and the Framingham Risk Score. Multiple logistic regression analyses were used to compute two additional scores for models that combined either the Diabetes Risk Score or the Framingham Risk Score with the metabolic syndrome. Since, unlike the metabolic syndrome, these four scores are not dichotomous, we constructed their receiver operating characteristic (ROC) curves and computed the areas under these curves (aROCs). Using these ROC curves, we compared the sensitivities of the various scores with the sensitivity of the metabolic syndrome at the false-positive rate of the latter and the false-positive rates of the scores with the false-positive rate of the metabolic syndrome at the sensitivity of the latter. These sensitivities and false-positive rates were computed for each end point in the relevant datasets.
Odds ratios were computed for the risk scores for a 2.0-unit increment in the logit of risk. This increment produces odds ratios that, we believe, represent a fair comparison with the odds ratios for the dichotomous metabolic syndrome variable. A 2.0-unit increment in risk is close to the interquartile range (2.00 for the Framingham logit and 2.07 for diabetes logit). It is also close to the difference between the Framingham logit of risk for the 932 subjects (chosen to equal the prevalence of the metabolic syndrome) with the highest Framingham Risk Scores and the remaining 1,638 individuals in the CVD dataset (difference in means, 2.22, and difference in medians, 1.82). For the diabetes dataset, the mean difference in logit of risk for the 550 subjects (again chosen to equal the prevalence of the metabolic syndrome) with the highest Diabetes Risk Scores versus the remaining 1,159 subjects was 2.11 (difference in medians, 2.52).
Our analyses were performed using SAS version 8.0. The SAS program developed by Delong et al. (9) was used to compare aROCs. McNemar’s test was used to compare sensitivities and false-positive rates. Statistical significance was defined as P < 0.05.
RESULTS
In the SAHS dataset used to compute diabetes incidence, the prevalence of the NCEP ATP-III-defined metabolic syndrome at baseline was 32.2% (550 of 1,709), with a prevalence of 27.1% (156 of 576) in non-Hispanic whites and 34.8% (394 of 1,133) in Mexican Americans. In the dataset used to compute CVD incidence, the metabolic syndrome prevalence at baseline was 36.3% (932 of 2,570), with a prevalence of 27.6% (228 of 827) in non-Hispanic whites and 40.4% (704 of 1,743) in Mexican Americans. In the MCDS, the metabolic syndrome prevalence at baseline was 40.9% (553 of 1,353).
Table 1 shows that in the SAHS the metabolic syndrome predicts type 2 diabetes with a sensitivity of 66.2% and a false-positive rate of 27.8%. Table 1 also shows that at the same false-positive rate, the Diabetes Risk Score has a significantly higher sensitivity (75.9%) and, at the same sensitivity, has a significantly lower false-positive rate (19.2%). Moreover, when the metabolic syndrome is used in combination with the Diabetes Risk Score, the sensitivity and false-positive rates of the combined model are not significantly improved over the Diabetes Risk Score alone. Also, the aROCs for the Diabetes Risk Score and the Diabetes Risk Score combined with the metabolic syndrome (shown in Fig. 1A) are not significantly different (P = 0.13).
Table 1 also shows analogous data for the MCDS. Again, the sensitivity at fixed false-positive rate and the false-positive rate at fixed sensitivity are significantly better for the Diabetes Risk Score than for the metabolic syndrome. When the two are combined, there is no significant improvement in the sensitivity, false-positive rate, or aROC (Fig. 1B).
Table 1 also shows analogous data for predicting CVD in the SAHS. Once again, at the same false-positive rate, the Framingham Risk Score has significantly higher sensitivity and, at the same sensitivity, a significantly better false-positive rate than the metabolic syndrome. When the metabolic syndrome is used in combination with the Framingham Risk Score, neither the sensitivity nor the false-positive rate is significantly better than the Framingham Risk Score used alone. Also, the aROCs for the Framingham Risk Score and the Framingham Risk Score combined with the metabolic syndrome (Fig. 1C) are nearly identical.
Table 2 shows the univariate and multivariate odds ratios for the metabolic syndrome and the Diabetes Risk Score for predicting diabetes in the SAHS and the MCDS and the corresponding odds ratios for the metabolic syndrome and the Framingham Risk Score for predicting CVD in the SAHS. All six univariate odds ratios are statistically significant. However, when both predictors are used in the same model, the odds ratios for the metabolic syndrome drop sharply and, in two of the three cases, become statistically nonsignificant. In the third case (predicting diabetes in the SAHS), the multivariate odds ratios for the metabolic syndrome, although much reduced, remains statistically significant. By contrast, the multivariate odds ratios for the Diabetes Risk Score and the Framingham Risk Score drop only minimally and remain statistically significant when either is combined with the metabolic syndrome.
CONCLUSIONS
From a clinical or public health perspective, having a definition of the metabolic syndrome is useful only if it identifies individuals at high risk of disease and particularly if it identifies individuals who are candidates for a specific treatment that they would not otherwise receive. The present results indicate that the metabolic syndrome as defined by the NCEP ATP-III criteria are less effective at predicting diabetes or CVD than the established predicting models designed specifically for these purposes. Moreover, at the present time, there is no specific treatment recommended for the metabolic syndrome other than the treatment of its various components, for which established guidelines already exist (10). For example, weight loss and exercise are advocated as the mainstays in the treatment of the metabolic syndrome (2). But, they are also the mainstays in the treatment of most of its components. Likewise, there are established criteria for the pharmacological treatment of elevated blood pressure, dyslipidemia, etc., regardless of whether these conditions occur in isolation or in concert with other features of the metabolic syndrome.
Clearly, one reason the metabolic syndrome, as defined by the NCEP ATP-III criteria, is inferior to the Framingham Risk Score at predicting CVD is because, unlike the latter, it does not contain several well-established, potent, cardiovascular risk factors, e.g., age, sex, total cholesterol, and cigarette smoking. These omissions are consistent with the NCEP ATP-III perspective that the metabolic syndrome is a “secondary target of therapy” for CVD prevention (2). But even as a secondary target, our data indicate that the metabolic syndrome adds little, if anything, to identifying a target population for intervention, as shown by the combined model that includes both it and the Framingham Risk Score.
The NCEP ATP-III-defined metabolic syndrome also lacks a potent risk factor for diabetes, namely, family history of diabetes. This omission may contribute to its inferior prediction of diabetes when compared with the Diabetes Risk Score. Another possible explanation for the superior predicting ability of the models is that the risk factors are treated as continuous variables and not dichotomized as in the NCEP ATP-III-defined metabolic syndrome. Apart from the loss of information that attends dichotomization of continuous variables, the chosen cut points may be population specific. Such cut points are likely to perform less well when applied to populations in which the mean levels of the risk factors differ from those in the population in which the cut points were initially developed.
From Table 2 it is evident that, taken individually, both the metabolic syndrome and the two predicting models have statistically significant odds ratios for predicting either diabetes or CVD. But when the metabolic syndrome is combined with either of the two predicting models, its odds ratios fall sharply, whereas the odds ratios for the predicting models are relatively preserved. In one of the three cases (prediction of diabetes in the SAHS), the multivariate odds ratio for the metabolic syndrome, although much reduced in magnitude, remains statistically significant. But the sensitivities, false-positive rates, and aROCs presented in Table 1 and Fig. 1A demonstrate that even though a variable may retain a statistically significant odds ratio in multivariate analysis, it may nevertheless add little to predicting future health outcomes.
The question arises, however, that if the WHO definition of the metabolic syndrome had been used, would it have performed better as a predictor of diabetes and CVD than the NCEP ATP-III definition. We have examined this issue and the results are presented in the online appendix (Tables 1A and 2A [available from http://care.diabetesjournals.org]). It should be noted that the WHO definition requires that an oral glucose tolerance test be performed because it includes as one of its criteria impaired glucose tolerance (IGT, defined as a plasma glucose between 7.8 and 11.1 mmol/l [140 and 200 mg/dl] 2-h after a standardized oral glucose load [3,4]). Since IGT is, by itself, a potent predictor of diabetes (11), it is not surprising that, as a predictor of diabetes, the WHO-defined metabolic syndrome outperforms the NCEP ATP-III-defined metabolic syndrome that does not include IGT. However, when IGT is excluded from the WHO-defined metabolic syndrome, the latter’s performance as a predictor of diabetes is similar to that of the NCEP ATP-III-defined metabolic syndrome and inferior to the Diabetes Predicting Model. As we have noted previously, an advantage of the Diabetes Predicting Model is that it does not require an oral glucose tolerance test, which is costly and inconvenient to perform (5,8). Finally, like the NCEP ATP-III-defined metabolic syndrome, the WHO-defined metabolic syndrome is inferior to the Framingham Risk Score at predicting CVD.
A number studies have examined the impact of the metabolic syndrome, as defined by either the WHO (12–15) or the NCEP (13–16) criteria, on the development of either diabetes (14,15) or CVD (12,13,16). None of these studies, however, has formally compared the metabolic syndrome with other established methods of predicting these same outcomes. Also, these studies relied heavily, if not exclusively, on odds ratios (or relative risks or hazard ratios) to assess the significance of the metabolic syndrome. Although useful in etiologic studies, a recent report by Pepe et al. (17) has highlighted the limitations of the odds ratio (or relative risks or hazard ratios) as a method of assessing the importance of a potential new risk factor and has emphasized the necessity of examining sensitivities, false-positive rates, and aROCs to form a more comprehensive picture of the clinical and public health relevance of any new risk factor.
As a final point, an advantage of using continuous scores is the flexibility they provide compared with categorical definitions such as the NCEP ATP-III definition of the metabolic syndrome. With a continuous score, the cut point for designating an individual as being at high risk can be chosen with a view to either maximizing the sensitivity or minimizing the false-positive rate depending on program needs. The cut point can also be chosen with a view toward calibrating the number of high-risk individuals identified in order to conform these numbers to the resources available for managing these individuals. By contrast, for the metabolic syndrome the sensitivity, false-positive rate, and number at risk are fixed by the NCEP ATP-III definition and not subject to the control of the investigator or program director. On the other hand, the use of model-derived score typically requires the user to solve an equation, usually a logistic regression equation. Although this can be readily accomplished with the aid of a personal computer or personal digital assistant, there is some resistance on the part of clinicians to use these scores. It is also the case that logistic regression equations can usually be transformed into paper-and-pencil scores with minimal loss of predicting power, and this may facilitate their acceptance by clinicians.
In conclusion, although the metabolic syndrome as defined by the NCEP ATP-III criteria can predict the future development of diabetes and CVD, it predicts less effectively than established predicting models such as the Diabetes Risk Score and the Framingham Risk Score, which were designed for that purpose. Moreover, combining the metabolic syndrome with these established predicting models does not enhance their performance.
. | aROC . | Sensitivity (%) . | False-positive rate (%) . |
---|---|---|---|
Prediction of diabetes | |||
SAHS | |||
Metabolic syndrome | — | 66.2 | 27.8 |
Diabetes Risk Score | 0.819 | 75.9 (P = 0.015) | Fixed at 27.8 |
Diabetes Risk Score and metabolic syndrome | 0.824 (P = 0.13) | 75.9 (P = 1.00) | Fixed at 27.8 |
Diabetes Risk Score | — | Fixed at 66.2 | 19.2 (P < 0.0001) |
Diabetes Risk Score and metabolic syndrome | — | Fixed at 66.2 | 19.9 (P = 0.22) |
MCDS | |||
Metabolic syndrome | — | 62.4 | 38.7 |
Diabetes Risk Score | 0.765 | 76.0 (P = 0.004) | Fixed at 38.7 |
Diabetes Risk Score and metabolic syndrome | 0.768 (P = 0.22) | 74.4 (P = 0.32) | Fixed at 38.7 |
Diabetes Risk Score | — | Fixed at 62.4 | 23.0 (P < 0.0001) |
Diabetes Risk Score and metabolic syndrome | — | Fixed at 62.4 | 24.7 (P < 0.32) |
Prediction of CVD in the SAHS | |||
Metabolic syndrome | — | 67.3 | 34.2 |
Framingham Risk Score | 0.816 | 81.4 (P = 0.0002) | Fixed at 34.2 |
Framingham Risk Score and metabolic syndrome | 0.811 (P = 0.10) | 81.4 (P = 1.00) | Fixed at 34.2 |
Framingham Risk Score | — | Fixed at 67.3 | 20.0 (P < 0.0001) |
Framingham Risk Score and metabolic syndrome | — | Fixed at 67.3 | 19.7 (P = 0.41) |
. | aROC . | Sensitivity (%) . | False-positive rate (%) . |
---|---|---|---|
Prediction of diabetes | |||
SAHS | |||
Metabolic syndrome | — | 66.2 | 27.8 |
Diabetes Risk Score | 0.819 | 75.9 (P = 0.015) | Fixed at 27.8 |
Diabetes Risk Score and metabolic syndrome | 0.824 (P = 0.13) | 75.9 (P = 1.00) | Fixed at 27.8 |
Diabetes Risk Score | — | Fixed at 66.2 | 19.2 (P < 0.0001) |
Diabetes Risk Score and metabolic syndrome | — | Fixed at 66.2 | 19.9 (P = 0.22) |
MCDS | |||
Metabolic syndrome | — | 62.4 | 38.7 |
Diabetes Risk Score | 0.765 | 76.0 (P = 0.004) | Fixed at 38.7 |
Diabetes Risk Score and metabolic syndrome | 0.768 (P = 0.22) | 74.4 (P = 0.32) | Fixed at 38.7 |
Diabetes Risk Score | — | Fixed at 62.4 | 23.0 (P < 0.0001) |
Diabetes Risk Score and metabolic syndrome | — | Fixed at 62.4 | 24.7 (P < 0.32) |
Prediction of CVD in the SAHS | |||
Metabolic syndrome | — | 67.3 | 34.2 |
Framingham Risk Score | 0.816 | 81.4 (P = 0.0002) | Fixed at 34.2 |
Framingham Risk Score and metabolic syndrome | 0.811 (P = 0.10) | 81.4 (P = 1.00) | Fixed at 34.2 |
Framingham Risk Score | — | Fixed at 67.3 | 20.0 (P < 0.0001) |
Framingham Risk Score and metabolic syndrome | — | Fixed at 67.3 | 19.7 (P = 0.41) |
P for comparison with the row immediately above.
. | Univariate . | Multivariate† . |
---|---|---|
Prediction of diabetes in the SAHS | ||
Metabolic syndrome | 5.08 (3.70–6.97) | 1.64 (1.14–2.38) |
Diabetes Risk Score | 6.46 (4.97–8.40) | 5.50 (4.13–7.33) |
Prediction of diabetes in the MCDS | ||
Metabolic syndrome | 2.63 (1.80–3.85) | 1.15 (0.74–1.77) |
Diabetes Risk Score | 4.22 (3.11–5.72) | 4.03 (2.87–5.65) |
Prediction of CVD in the SAHS | ||
Metabolic Syndrome | 3.95 (2.80–5.58) | 1.14 (0.76–1.71) |
Framingham Risk Score | 9.68 (6.69–14.02) | 9.06 (5.96–13.79) |
. | Univariate . | Multivariate† . |
---|---|---|
Prediction of diabetes in the SAHS | ||
Metabolic syndrome | 5.08 (3.70–6.97) | 1.64 (1.14–2.38) |
Diabetes Risk Score | 6.46 (4.97–8.40) | 5.50 (4.13–7.33) |
Prediction of diabetes in the MCDS | ||
Metabolic syndrome | 2.63 (1.80–3.85) | 1.15 (0.74–1.77) |
Diabetes Risk Score | 4.22 (3.11–5.72) | 4.03 (2.87–5.65) |
Prediction of CVD in the SAHS | ||
Metabolic Syndrome | 3.95 (2.80–5.58) | 1.14 (0.76–1.71) |
Framingham Risk Score | 9.68 (6.69–14.02) | 9.06 (5.96–13.79) |
Data are odds ratio (95% CI).
Odds ratios for multiple regression model-derived scores are based on a 2.0-unit increment in the logit of risk;
Article Information
This work was supported by grants from the National Heart, Lung, and Blood Institute (R01 HL24799 and R01 HL36820).
References
Additional information for this article can be found in an online appendix at http://care.diabetesjournals.org.
A table elsewhere in this issue shows conventional and Système International (SI) units and conversion factors for many substances.
See accompanying editorial, p. 2761.