Magnitude and Pattern of Placebo Response in Clinical Trials of Oral Antihyperglycemic Agents: Data From the U.S. Food and Drug Administration, 1999–2015

The placebo response in clinical trials of diabetes has received minimal attention in clinical trial research. In this context, placebo response is a measurement of any change from baseline that occurs in the placebo treatment arm; these changes may occur as a response to the inactive pill, regression to the mean, spontaneous improvement, contextual factors of the trial itself, lifestyle changes made by patients, and other nonpharmacologic effects.

The concern that placebo response is inadequately low in trials of diabetes medications has fueled debate about the ethics of placebo exposure in patients with type 2 diabetes (1–3). However, such positions on either side of the debate are not based on specific data, as published research on clinical trials of type 2 diabetes is lacking a systematic review of data regarding response to placebo treatment in these trials. Therefore, assumptions about the magnitude and pattern of placebo response in these trials have neither been affirmed nor challenged.

Type 2 diabetes is a complex endocrine disorder with a clear pathophysiologic basis, and, as such, it might be assumed that it would be unlikely to improve with placebo. Nonetheless, research over the past 60 years has supported the contention articulated by Beecher in his classic 1955 paper (4) that substantial improvement with placebo occurs across a wide array of ailments, including those with a clear pathophysiological basis. Additionally, previous findings in other conditions raise the possibility that patients with diabetes may experience a measureable placebo response, potentially related in part to the effects of better behavioral management such as exercise, dieting, and stress reduction.

If such a placebo response were to be present in clinical trials of diabetes, the assumption would be that the response pattern emerging over time would be similar to the pattern observed in trials of other conditions—an increase in the magnitude of placebo response without significant change in the efficacy outcomes (effect size and success rate) of trials for oral antihyperglycemic agents. The reason we assume that this pattern will emerge in diabetes trials is because rising placebo response and stable outcome measures over time appear to be a general phenomenon across placebo-controlled trials of all types, as has been seen in depression (5), attention-deficit hyperactivity disorder (6), hypertension, and epilepsy trials (7,8). Enhanced expectation of improvement may be behind this rising placebo response but other variables may play a role as well.

Furthermore, for several reasons, we consider diabetes trials to be a good sample to investigate this exploratory model of placebo response and clinical trial efficacy outcomes over time. First, it is a chronic condition similar to the other illnesses we have evaluated with trials that are conducted over a period of at least a month. Also, there are many oral antihyperglycemic agents that have been approved over the past couple of decades, and therefore, we could access a large database sufficient for this type of analysis. Last, behavioral, nonpharmacological interventions have a role in the management of diabetes, thus raising the possibility that changes in diabetes care using these interventions may have influenced the magnitude of placebo response.

This study aims to fill the gap in the literature regarding placebo response in clinical trials of type 2 diabetes. We hypothesized that there would be a measureable placebo response in trials for type 2 diabetes and that response to placebo treatment would have increased significantly over the history of these trials. We also hypothesized that despite this increase, trial efficacy outcomes would not be impacted, following the same general pattern we have seen in clinical trials of other conditions.

To investigate this possibility, we examined the medical and statistical reviews submitted to the U.S. Food and Drug Administration (FDA) for the approval of 19 oral antihyperglycemic agents between 1999 and 2015. We recorded the reported change in placebo and active treatments, calculated outcome measures of effect size and success rate, and examined these factors over time. We noted that there are two common trial designs for placebo control in trials of diabetes: exclusive placebo-control monotherapy and augmented placebo add-on (in which all patients are stabilized on a background medication and the control group receives placebo added on top instead of the investigational agent). Due to the major divergence in these two designs, we examined the data for these two groups separately and conducted exploratory analysis to examine any differences based on type of placebo control. Finally, we conducted an exploratory meta-regression analysis of patient and trial design characteristics (mean age, percent female, baseline HbA_1c, duration, and number of treatment arms) to evaluate if any of these variables might have contributed to the magnitude and pattern of treatment response and efficacy outcomes we observed.

We accessed the FDA database (www.accessdata.fda.gov/scripts/cder/daf/index.cfm) (9) to collect type 2 diabetes efficacy trial data presented in New Drug Approval packets. Different from published reports, which are analyzed and written by pharmaceutical company staff, these data are the results of statistical analysis conducted by an unbiased FDA reviewer. The FDA staff has access to all of the trial data collected by the pharmaceutical company regarding the investigational medication and can consider all trials with relevant efficacy end points that were conducted according to sufficient scientific models, regardless of if they favor the investigational medication or not. The trials presented in the review of efficacy may have some selection bias relating to this review process, but are not subject to the publication bias that occurs in published reports (10). Additionally, the statistical treatments and presentation of data in these reviews are of sufficient quality, completeness, and comparability such that we could analyze these efficacy data across different types of oral antihyperglycemic investigational agents.

We selected programs for investigational antihyperglycemic medications if their FDA medical and statistical reviews were available on the FDA database (www.accessdata.fda.gov/scripts/cder/daf/index.cfm) and if they were indicated for the treatment of adults with type 2 diabetes. All agents were oral antihyperglycemic agents intended to lower HbA_1c levels. Combinations (i.e., + metformin) and new formulations (i.e., extended-release formulations) were included if they included data that had not been previously used in approval of prior submissions.

There were 19 oral antihyperglycemic agents that met inclusion for this study: pioglitazone hydrochloride (approved in 1999), rosiglitazone maleate (1999), glyburide + metformin (2000), metformin hydrochloride MR (2000), metformin hydrochloride XT (2004), metformin hydrochloride XR (2005), sitagliptin phosphate (2006), sitagliptin + metformin (2007), bromocriptine mesylate (2009), saxagliptin hydrochloride (2010), linagliptin (2011), linagliptin + metformin (2012), sitagliptin + metformin (2012), alogliptin benzoate (2013), alogliptin + metformin (2013), alogliptin + pioglitazone (2013), dapagliflozin propanediol (2014), empagliflozin (2014), and empagliflozin + metformin (2015).

We included all acute, placebo-controlled trials using approved doses of the investigational oral antihyperglycemic medications that were cited in the integrated review of efficacy for approval. These trials enrolled adult patients (≥18 years) with type 2 diabetes and insufficient glycemic control, typically defined as an HbA_1c level of ∼7–10% inclusive. Patients were otherwise healthy with any other physical conditions under adequate control. Both treatment-naive and previously treated patients with diabetes were typically included under the requirement that their diabetes was still uncontrolled as evidenced by elevated HbA_1c. Because we were evaluating comparisons among orally administered agents, we excluded trials with insulin treatment.

Out of the 61 placebo-controlled efficacy trials cited in the 19 programs, we excluded 4 trials that used insulin as a partial control, 3 long-term trials, and 2 trials with confounding designs, and 1 trial could not be used because the end point analysis was censored in the review. This left a total of 51 unique efficacy trials for evaluation. Out of 103 treatment arms from these 51 trials, we eliminated 7 treatment arms at unapproved doses, leaving a total of 96 treatment arms for analysis.

FDA reviewers conduct independent statistical analysis of efficacy for each treatment arm at different dose levels within a trial. For this reason, we decided to examine treatment arms independently of the trials they were in. Efficacy end point analysis compares symptom reduction between antihyperglycemic agent–treated and placebo control groups on the prespecified primary outcome measure.

New Drug Approval reviews use P < 0.05 to indicate trial arm success for efficacy outcomes. Therefore, we replicated this threshold of the FDA reviewers such that if the reported P value was <0.05, then we recorded this as a successful trial arm, and if it was >0.05, we recorded it as a failed trial arm.

Treatment response was defined as the change in percentage HbA_1c (baseline − end point HbA_1c) as measured by the techniques used in trial protocol. Change scores (reported as “Decrease in HbA_1c” in Supplementary Tables 1 and 2) for placebo and active treatment were taken from the reported mean reductions in HbA_1c percentage. Higher treatment response values indicate greater reduction of HbA_1c, whereas negative scores represent an overall increase in HbA_1c from baseline to final measurement.

We calculated effect size using Hedges’ g formula to account for the possibility of small sample sizes. Hedges’ g is calculated using the following equation: g = t × ⁠. Hedges’ g corrects for small sample size as follows: corrected g = g × ⁠.

Statistical measures were generated with IBM SPSS. We evaluated placebo and drug response as well as effect size over time by using weighted meta-regressions with a random-effects model using maximum likelihood. For more complex analysis of the variability between trials in their patient and trial design characteristics, we used a multifactorial weighted meta-regression. To examine between-group differences between the two types of placebo controls, we used t and χ² tests where appropriate.

As a way to evaluate evidence of selection bias, we created funnel plots and analyzed the data using the conservative method of Orwin’s fail-safe N, which calculated the number of studies required to reduce the overall effect size to a nonsignificant level.

Because there were two distinct types of placebo control used in these trials of oral antihyperglycemic agents, we grouped the trials according to this distinction and examined our hypotheses in both groups separately. The first group shown in Supplementary Table 1 included all 23 of the trials (47 treatment arms) with exclusively placebo-treated controls (often referred to as monotherapy trials).

The second group consisted of the trials in which a background medication was used as a run-in for all patients, with placebo or investigational medication added on top. This is often referred to as an “add-on” trial design; however, for the purposes of this study, we will refer to such trials as having augmented placebo controls. These 27 augmented placebo-controlled trials (49 treatment arms) are presented along with the background medications used in Supplementary Table 2. Out of the 19 programs we evaluated, 8 programs contributed trial data to both types of placebo control.

We considered many potential patient and trial design characteristics of interest for exploratory analysis. The five potential variables for which we were able to collect sufficient information were mean age, placebo/drug baseline HbA_1c, percent female, duration, and the number of treatment arms. The rest of the variables we considered were too sparsely reported.

The mean age for all patients across treatment conditions in the trial was collected for each of the trials that gave sufficient information regarding age. The placebo/drug baseline was the reported mean starting level of percent HbA_1c for patients in each treatment condition (Supplementary Tables 1 and 2). Percentage of patients in the trial who identified as female was recorded for all trials in which it was given. Duration was the time period of observation and data collection from baseline measurement to the end of the trial, measured in weeks (Supplementary Tables 1 and 2). The number of treatment assignments in the trial was recorded as “number of treatment arms.” Active comparator arms were counted in this measure to represent the design of the trial at the time it was conducted. A trial with a placebo arm, two different doses of investigational antihyperglycemic agent, and an active comparator arm would be coded as having four treatment arms for this analysis.

Supplementary Table 1 presents the essential characteristics, including trial duration, baseline HbA_1c values, and raw data for 47 treatment arms from 23 monotherapy trials (1999–2014) for oral antihyperglycemics using exclusive placebo controls. Supplementary Table 2 presents the same information along with the background medications used for 49 treatment arms from 27 trials (1999–2015) for diabetes medications using augmented placebo control. All of the treatment arms had sufficient data to calculate each of our dependent measures.

Meta-regression analysis of placebo response and year of approval was found to be significant, with an R² of 0.31 (P = 0.0013) (Fig. 1A). Placebo response increased by 0.035 for each year following 1999. The predictive value of year of approval and drug response was also significant and positive over time, with an R² of 0.076 (P = 0.0498). Drug response increased by 0.017 every year following 1999. Time did not appear to be predictive of any increase or decrease in the effect size of exclusively placebo-controlled treatment arms (Fig. 1B), with an R² of 0.003 (P = 0.725). The success rate was 100% (47 out of 47 successful) and did not change over time.

Similarly for augmented placebo control, a significant regression was found in which year predicted the magnitude of placebo response, with an R² of 0.407 (P < 0.001) (Fig. 2A). Placebo response increased by 0.33 for each year following 1999. Although it appears that the drug response is also rising slightly over time, this regression did not meet statistical significance (P = 0.207), with an R² of 0.031. The regression for effect size over time was also insignificant (P = 0.158), with an R² of 0.039 (Fig. 2B). The success rate was 100% (49 out of 49 successful) and did not change over time.

Interestingly, augmented placebo-controlled arms had a significantly higher placebo response (0.05 ± 0.23%) than exclusively placebo-controlled arms (−0.19 ± 0.35%; t = 2.8; df = 47; P = 0.008). However, the drug response between the two groups with different types of placebo control did not differ significantly from one another: exclusive placebo control having a mean drug response of 0.75 ± 0.35% and augmented placebo control with a drug response of 0.72 ± 0.19% (t = −0.67; df = 94; P = 0.504). This may contribute to the finding that in addition to a lower placebo response, treatment arms controlled exclusively with placebo had a significantly higher mean effect size (0.96 ± 0.36) compared with augmented placebo (0.68 ± 0.20; t = −4.7; df = 94; P < 0.001).

In terms of reduction of HbA_1c levels, augmented placebo actually reduced HbA_1c from baseline 63% (17 out of 27) of the time, whereas exclusive placebo treatment resulted in HbA_1c reductions from baseline only ∼18% (4 out of 22) of the time (χ² = 9.93; df = 1; P = 0.002).

The percent of female patients, mean age, number of treatment arms, trial duration, and drug arm baseline HbA_1c did not have any significant relationships with effect size, drug response, or placebo response. The multivariate meta-regression revealed that the starting level of HbA_1c for placebo-treated patients significantly predicted the amount of HbA_1c reduction with placebo (exclusive placebo control: β = −0.411, P < 0.001; augmented placebo control: β = −0.199, P = 0.008). When examined independently of the other variables, placebo baseline HbA_1c had an inverse relationship with placebo response (higher baseline HbA_1c was associated with less HbA_1c reduction in the placebo treatment group) in both exclusively placebo-controlled trials (R² = 0.566; β = −0.32; P < 0.001) and augmented placebo-controlled trials (R² = 0.454; β = −0.297; P < 0.001). Additionally, placebo baseline HbA_1c decreased significantly over time in the exclusive (R² = 0.372; β = −0.088; P = 0.002) and augmented group (R² = 0.221; β = −0.055; P = 0.006). Drug baseline HbA_1c had no significant relationship to drug response, and neither drug nor placebo HbA_1c baseline had any significant relationship to effect size in either type of placebo control.

The funnel plots visually depicting the relationship between the SE and the effect size of treatment arms from both exclusively and augmented placebo-controlled studies did not appear to indicate a disproportionate representation of studies with large or small effect sizes (Supplementary Figs. 1 and 2). The effect size for exclusively placebo-controlled studies corresponded to a z-value of 327.0 with a fail-safe N of 39,463. The effect size for augmented placebo-controlled studies corresponded to a z-value of 297.3 with a fail-safe N of 32,618. These findings do not support a bias based on selection in these data.

The aim of this study was to provide a systematic evaluation of the magnitude and pattern of placebo response in clinical trials for investigational oral antihyperglycemic agents and to examine response to treatment conditions and resulting trial efficacy outcomes over time. In support of our hypothesis, both sets of analysis of treatment arms from the two common types of placebo-controlled trial design (exclusively placebo-controlled and augmented placebo-controlled) demonstrated that there is a nontrivial placebo response in clinical trials for type 2 diabetes. Similar to the apparent generalized phenomenon seen in trials of other psychiatric and physical conditions (5–8), the magnitude of placebo response is rising significantly over time, whereas effect sizes and success rates have remained unaffected.

Specifically, in exclusively placebo-controlled monotherapy trials, placebo response has increased by 0.5% HbA_1c reduction over the past 17 years (Fig. 1A). Intriguingly, although older trials show that exclusive placebo treatment exacerbated HbA_1c levels over the course of treatment, in more recent trials, the mean placebo response hovers at ∼0%, indicating that the HbA_1c levels of patients treated with placebo no longer appear to be worsening (Fig. 1A). Likely due to the additive (but proportionately blunted) contribution of the placebo response to the overall measured drug response, the response to investigational oral antihyperglycemic agents has also increased over time, although less steeply. As would be expected, this semiparallel growth appears to have kept the overall placebo-subtracted effect sizes for these investigational agents at a constant 0.96 (± 0.36) over time (Fig. 1B), which equates to a placebo-subtracted drug effect of 0.96% HbA_1c reduction.

Similarly, in augmented placebo-controlled trials, the observed placebo response has shown growth in HbA_1c reduction of ∼0.5 percentage points over time (Fig. 2A). More recently, placebo assignment appears to actually reduce HbA_1c levels by 0.2% in these augmented design trials. Likely also due to the proportional additive effect of placebo response on the overall drug response, the effect size for trials using augmented placebo-control design has remained steady at 0.68 (± 0.20) (Fig. 2B) over the years despite the increase in placebo response, keeping the mean placebo-subtracted drug effect at ∼0.61% HbA_1c reduction. Trials from both placebo-control types support a previous analysis of published trial data, estimating the placebo-subtracted drug effects between 0.5 and 1.25% reduction in HbA_1c (10).

This pattern of a rising placebo response and stable outcome measures is similar to the pattern seen in other chronic illnesses, supporting the assumption that this pattern may represent a generalized phenomenon across placebo-controlled clinical trials. As in the case of antidepressants (5), attention-deficit hyperactivity disorder medications (6), antihypertensives, and antiepileptics (7,8), the significant growth in placebo response in both exclusive and augmented placebo-controlled trials of type 2 diabetes appears not to have had an effect on trial efficacy outcomes. Like other medication trials, this lack of effect on efficacy outcomes appears to be related to the apparent, although not statistically significant, proportional increase in drug response over time (Figs. 1A and 2A), which maintains the same overall treatment effect over placebo. This supports previous findings from published studies that all of the noninsulin oral antihyperglycemic agents appear to have similar placebo-subtracted treatment effects in the context of placebo-controlled trials (11–14), although differences among agents may be more apparent in head-to-head comparator trials (15–18).

These data suggest that the placebo response in diabetes trials represents more than just regression to the mean and passage of time, given that the placebo response has been inconstant and has been rising significantly over time. This growth in placebo response is more likely related to better design and execution of recent diabetes trials compared with earlier.

However, the mechanisms contributing to the placebo response in trials of diabetes are unclear. The only potentially related factor that stood out in our exploratory analysis was the HbA_1c level at baseline measurement for placebo-treated patients. These data suggest that higher starting levels of HbA_1c predicted a lower response to placebo treatment. This suggestion has also been made in the context of antidepressant clinical trials, which have also shown an inverse relationship of placebo response to placebo baseline scores (19). Drug baseline HbA_1c did not show any detectable relationship with drug response, indicating that nonpharmacological effects may have a stronger impact on cases of type 2 diabetes with lower starting levels of HbA_1c, whereas antihyperglycemic drug effects may not be significantly diminished or augmented based on HbA_1c level. Furthermore, all patient HbA_1c baselines have decreased significantly over time (from ∼9% HbA_1c on average to ∼8% HbA_1c), which may have contributed to the observed increase in placebo response, as lower HbA_1c levels appear to predict greater reduction in HbA_1c (higher placebo response). These findings may not align with previous research (20), although differences in methodology may have contributed to this contrast.

However, if the relationship of the starting level of HbA_1c on placebo response is substantiated in future prospective studies, it still only accounts for ∼50% of the variance in placebo response in these data. This suggests that other nonpharmacologic effects that could not be measured in this context may be at play. In psychiatric conditions and hypertension, features of the patient–clinician interaction and the outcome measurement techniques may contribute to the magnitude of placebo response. In diabetes trials, nonpharmacologic effects of patient–clinician interaction may take the form of behavioral interventions such as encouraging better diet, exercise, and stress reduction, which may act as intervening variables. These interventions may have been increasingly incorporated into the treatment for all patients with diabetes participating in clinical trials. Additionally, following the advent of direct-to-consumer marketing of pharmaceuticals, expectations for the effects of medications have likely increased, which may influence the response to placebo observed in diabetes trials and across trials of all conditions (21). All of these prospects need to be further clarified in future studies.

In our exploratory analysis of the impact of the two placebo-control designs, we noted that placebo growth over time appears to be steeper in augmented therapy (R = 0.643) (Fig. 2A) than in exclusive placebo control (R = 0.538) (Fig. 1A). This is interesting because the overall mean drug response is nearly equivalent (∼0.7% reduction) (Figs. 1A and 2A) between the two designs, whereas the overall mean placebo response is much higher in augmented therapy (0.05% reduction) compared with exclusively placebo-controlled arms (−0.19% reduction; P = 0.008).

Furthermore, in augmented placebo-controlled design, placebo treatment reduced HbA_1c from baseline 63% (17 out of 27) of the time, whereas exclusive placebo treatment resulted in HbA_1c reductions from baseline only ∼18% (4 out of 22) of the time (χ² = 9.93; df = 1; P = 0.002). This observation suggests that nonpharmacological effects may have greater potential to benefit the patient when the patient is on an active medication, rather than nonpharmacological effects used in isolation. The estimations of mean effect size from this current analysis may inform the power calculations for future clinical trials. It appears that the majority of these trials are powered correctly given the moderate to large effect sizes observed. It is important to take into consideration that augmented placebo-controlled trials, although more reliably providing HbA_1c reduction for placebo-treated patients, may require more patient exposure from larger sample sizes due to the smaller effect size observed in trials with this design. These differences between these two different methods of placebo control in trials of diabetes are important to consider and should inform the discussions surrounding the use of placebo (1–3).

These preliminary findings should be considered with caution, as this is a first attempt to specifically examine placebo response data in type 2 diabetes clinical trials, and there are no supporting studies examining this phenomenon that have been published. Additionally, this study is post hoc in analysis, and therefore, causal explanations cannot be gleaned from these data, only theorized.

Overall, diabetes clinical trial treatment arms have a success rate of 100% (96 out of 96) and stable, large effect sizes. Currently, exclusive placebo treatment in trials of oral antihyperglycemics appears not to exacerbate HbA_1c levels of patients and in some cases actually provides measurable HbA_1c reduction. In placebo treatment added to active treatment, the benefit is even more consistent and robust. These data suggest that the magnitude of placebo response has increased significantly over time without any impact on trial efficacy outcomes, following the general phenomenon seen across other chronic physical and psychiatric conditions. In conclusion, this analysis highlights the need for more research looking at nonpharmacological as well as pharmacological elements in the design and conduct of future diabetes trials.

Duality of Interest. No potential conflicts of interest relevant to this article were reported.

Author Contributions. A.K. was responsible for methodological design, conceptual development, revisions, and interpretation of data. K.F.M. collected data, performed statistical analysis, drafted and edited the manuscript, and interpreted data. J.S. and W.A.B. provided conceptual input to the introduction, discussion, revisions, and editing, and reviewed the data. A.K. is the guarantor of this work and, as such, had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.