STUDY: HOSPITAL RANKINGS MAY RELY ON FAULTY DATA; U.S. NEWS RESPONDS

Summary: “The October 2016 issue of The Joint Commission Journal on Quality and Patient Safety, 

features the article ‘Consumer Rankings and Health Care: Toward Validation and Transparency,’ by 

Bala Hota, MD, MPH, and co-authors, Rush University Medical Center (RUMC), Chicago.” according to 

a press release on EurekAlert. “After RUMC received a lower than expected ranking for patient safety 

in the 2015-16 U.S. News & World Report’s (USNWR) ‘Best Hospitals’ rankings, the authors compared 

the data that USNWR used for their hospital to their own internal data. The authors found that the 

USNWR data showed many more patient safety events, such as pressure ulcers, almost all of which 

had actually been present at the patient’s admission. Suspecting a broader problem, Hota et al. 

analyzed data on a sample of hospitals and found that RUMC was not the only organization with 

discrepancies in data. False-positive event rates were common among high-transfer and high-volume 

hospitals. The authors conclude more transparency and validation is needed for consumer-based 

benchmarking methods. In response to these findings and concerns raised by others, USNWR made 

changes to its methodology and data sources in 2016. In an accompanying editorial, ‘The Quality 

Measurement Crisis: An Urgent Need for Methodological Standards and Transparency,’ David M. 

Shahian, MD, Elizabeth Mort, MD, MPH, and Peter J. Pronovost, MD, PhD, reflect on the Hota et al. 

article to conclude, ‘Just as health care providers have ethical and moral responsibilities to the public 

they serve, rating organizations and journalists that grade providers have similar obligations–in their 

case, to ensure measure validity and methodological transparency.’ RUMC further explored the 

importance of rating organization’s validity and methodological transparency in the following essay, 

‘Hospital Rankings Have Room for Improvement.'” Study: Hospital rankings may rely on faulty 

data,” EurekAlert! Science News. In response, Ben Harder, head of the U.S. News and Word Report 

quality ranking efforts, posted a description of how “quality measurement is a journey,”