Seeing RED After Making Decisions Based on Observational Analysis:

What Randomization Teaches Us About
the Difference between Not Having Anemia and Treating Anemia

It has been known for many years that among patients with serious illness, those who have anemia have poorer outcomes and functioning than those who don’t. But does that association between higher hemoglobin (Hb) levels and better outcomes mean that treating patients with erythropoiesis-stimulating agents (ESAs) to increase their Hb levels will improve their health? The medical community thought so, for many years. All that has changed, though—thanks to a shift in the quality of research methodology.

Perhaps the most widely publicized change was to kidney disease treatment, in part because a key study, TREAT (Trial to Reduce Cardiovascular Events with Aranesp Therapy), was released shortly before Halloween in 2009, prompting a flood of predictable “Trick or TREAT” headline puns.[1] (Truth be told, Fred Curtiss and I were responsible for one of them in a 2009 JMCP editorial.)[2]

The remarkable sequence of events that “turned the world of anemia management upside down”[3] in kidney disease management was described in a 2008 American Journal of Kidney Diseases commentary by Marc Pfeffer, Principal Investigator of TREAT.[4] Typically, Pfeffer pointed out, drug therapy advances take place in a predictable sequence. First, placebo-controlled trials test the drug in the highest-risk, most severely ill subset of patients with a disorder. For example, the first tests of antihypertensives were made in patients with diastolic (I said diastolic, not systolic) blood pressures averaging 115 to 129 millimeters mercury (mm Hg).[5] Then, additional placebo-controlled trials of the drug are conducted in less severely ill patients. As Pfeffer observed, this sequence was followed in studies of antihypertensive combination therapies, ACE (angiotensin-converting enzyme) inhibitors, and statins.

For ESAs, the path was quite different. Following initial testing of ESAs in a sample of severely anemic dialyzed patients with no control group in 1989,[6] strong associations between higher Hb (or hematocrit) levels and positive patient outcomes in numerous observational studies[7] led to a widespread presumption that treatment with ESAs caused improvement in patient health and quality of life—with no placebo-controlled trials to determine whether the presumption was actually correct. (The placebo-controlled TREAT, initiated in 2004, was the first exception.)

In a discussion that today should serve as a reminder of the potentially serious hazards of standardizing clinical practice without a rigorous evidence base, one of the observational study reports even attributed better outcomes in dialyzed patients to ESA use in accordance with guidelines that had been promulgated in 1999:

The country with the highest median haemoglobin value, Spain …  had relatively high values in the three major categories of anaemia management practice: [recombinant human erythropoietin (rHuEpo)] use (91%), mean rHuEpo dose (114 IU/kg body weight/week) and prevalence of i.v. iron use (66%) … . The survey responses from … medical directors indicated that the majority of Spanish dialysis units had a policy of initiating rHuEpo therapy at relatively high haemoglobin threshold values. The high mean haemoglobin concentration observed for Spanish haemodialysis patients would appear to be largely a consequence of a country-wide practice of initiating rHuEpo use at a relatively high haemoglobin threshold value in conjunction with high use of rHuEpo, at a moderately high dose and maintaining sufficiently high iron levels for haemodialysis patients.  … It is apparent from the findings of the present study that a patient's haemoglobin concentration affects morbidity and mortality. … Although observational studies cannot prove causality, this study suggests that, if the [European Best Practice Guidelines (EBPG)] are followed, a trend for improved outcomes in anaemic CKD patients may be expected. The EBPG on anaemia, based on expert review of evidence, thus receives additional supportive evidence from the present study. Despite the dissemination of the EPBG, many European haemodialysis patients still have haemoglobin concentrations below the minimum recommended level of 11 g/dl. This suggests a major opportunity for improved anaemia management for haemodialysis patients.

Widespread use of ESAs expanded to patients with cancer and nondialyzed kidney disease—again, Pfeffer reported, without supporting evidence from placebo-controlled trials. Remarkably, the benefits of ESAs for patients with anemia in kidney disease were so widely accepted that the ethics of having a placebo group were questioned in the planning stages of TREAT.[4]

The tide began to turn in November 2006 with the publication of the CHOIR (Correction of Hemoglobin and Outcomes In Renal insufficiency) and CREATE (Cardiovascular Risk Reduction by Early Anemia Treatment with Epoetin beta) trials, which randomized nondialyzed patients with kidney disease to higher versus lower Hb targets using ESA treatment (but still no placebo group!).[8] CREATE found that complete anemia correction did not reduce the rate of cardiovascular events but did improve quality-of-life measures of physical functioning and general health. CHOIR found that use of higher Hb targets increased the risk of cardiovascular events without improving quality of life.[8] Shortly thereafter, a series of eight studies provided to the FDA found “more rapid tumor growth or shortened survival when patients with breast, non-small cell lung, head and neck, lymphoid or cervical cancers received ESAs compared to patients who did not receive this treatment,” resulting in “black box warning” additions to the product labels.[9] TREAT, which was conducted in a sample of patients with chronic kidney disease and type 2 diabetes, found no statistical difference between ESA and placebo in the primary outcomes (death or a nonfatal cardiac event; death or end-stage renal disease), but ESA-treated patients were twice as likely to have a stroke.[10] Finally and most recently, the RED-HF (Reduction of Events by Darbopoetin Alfa in Heart Failure) trial, reported in The New England Journal of Medicine in March 2013, found that ESA treatment did not improve the primary study outcome (death or hospitalization from heart failure) or any secondary outcomes in a sample of patients with heart failure and mild-to-moderate anemia, but increased the risk of thromboembolic events.[11]

Practical Takeaway Point: Quality, Not Just Quantity, of Evidence

What can we learn from the nearly 20-year history of treatment with ESAs that, as Pfeffer described it, "greatly outpaced the data" prior to CHOIR, CREATE, TREAT, and now RED-HF? The principal moral of the sequence of events in ESA treatment and research—and so many others like it in health care—is this: much as we wish that our observations told the whole story, they often don’t. Associations are often elucidating but sometimes misleading, and interventions based on them should be subjected to rigorous testing, preferably using experimental designs, before being put into practice.

So, if you read or hear a claim that a proposal or treatment is “evidence-based,” try to find out what type of evidence supports the claim. Most online press articles contain links to the original study report including the abstract, which provides a quick overview of the design and analysis. If the study investigators did not use a control group, or if the control group was not randomized using “allocation concealment” (meaning that the treatment assignment is “concealed” or hidden from investigators or staff who can influence the study results), the quality of evidence should be viewed as relatively weak. It takes a little extra time to make a determination of quality of evidence—but it is time well spent.

[1] Husten L. Halloween trick: don’t TREAT diabetes with ESAs. Cardiobrief. October 30, 2009.

[2] Curtiss FR, Fairman KA. No TREATment with darbepoetin dosed to hemoglobin 13 grams per deciliter in type 2 diabetes with pre-dialysis chronic kidney disease—safety warnings for erythropoiesis-stimulating agents. J Manag Care Pharm. 2009;15(9):759-765.

[3] Singh AK. Does TREAT give the boot to ESAs in the treatment of CKD anemia? J Am Soc Nephrol. 2010;28.

[4] Pfeffer MA. Critical missing data on erythropoiesis-stimulating agents in CKD: first beat placebo. Am J Kidney Dis. 2008;51(3):366-369.

[5] Veterans Administration Cooperative Study Group on Antihypertensive Agents. Effects of treatment on morbidity in hypertension (Results in patients with diastolic blood pressures averaging 115 through 129 mm Hg). JAMA. 1967;202:1028-1034.

[6] Eschbach JW, Abdulhadi MH, Browne JK, et al. Recombinant human erythropoietin in anemic patients with end-stage renal disease. Results of a phase III multicenter clinical trial. Ann Intern Med. 1989;111:992-1000.

[7] Foley RN, Parfrey PS, Harnett JD, Kent GM, Murray DC, Barre PE. The impact of anemia on cardiomyopathy, morbidity, and mortality in end-stage renal disease. Am J Kidney Dis. 1996;28(1):53-61; Tong PC, Kong AP, So WY, et al. Hematocrit, independent of chronic kidney disease, predicts adverse cardiovascular outcomes in Chinese patients with type 2 diabetes. Diabetes Care. 2006;29(11):2439-2444; Locatelli F, Pisoni RL, Combe C, et al. Anaemia in haemodialysis patients of five European countries: association with morbidity and mortality in the Dialysis Outcomes and Practice Patterns Study (DOPPS). Nephrol Dial Transplan. 2004;19(1):121-132; Thorp M, Johnson ES, Yang X, Petrik AF, Platt R, Smith DH. Effect of anaemia on mortality, cardiovascular hospitalizations and end-stage renal disease among patients with chronic kidney disease. Nephrology (Carlton). 2009;14(2):240-246.

[8] Drueke TB, Locatelli F, Clyne N, et al.; CREATE Investigators. Normalization of hemoglobin level in patients with chronic kidney disease and anemia. N Engl J Med. 2006;355(20):2071-2084; Singh AK, Szczech L, Tang KL, et al.; CHOIR Investigators. Correction of anemia with epoetin alfa in chronic kidney disease. N Engl J Med. 2006;355(20:2085-2098.

[9] U.S. Food and Drug Administration. FDA receives new data on risks of anemia drugs consistent with previous data on tumor growth and death. January 3, 2008.

[10] Pfeffer MA, Burdmann EA, Chen CY, et al; TREAT investigators. A trial of darbepoetin alfa in type 2 diabetes and chronic kidney disease. N Engl J Med. 2009;361:2019-2032.

[11] Swedberg K, Young JB, Anand IS, et al.; RED-HF Committees and Investigators. Treatment of anemia with darbepoetin alfa in systolic heart failure. N Engl J Med. 2013 [EPub ahead of print]