Hospital Report Cards an Inaccurate Tool for Assessing Craniotomy Surgery Outcomes
Article ID: 519482
Released: 14-Apr-2006 12:00 PM EDT
Source Newsroom: American Association of Neurological Surgeons (AANS)
Newswise — Modern medicine is under pressure both to increase overall health care quality and to improve patient access to the highest quality care available. An increasingly popular tool in these efforts is the provider-based outcomes report, or "report card" . Typically, report cards contain information about the results a provider has actually achieved in patients with a specified diagnosis or procedure (such as mortality after cardiac bypass surgery) in comparison with expected results for that group of patients, based on a risk prediction model that adjusts for patient characteristics (case mix).
Coronary artery bypass graft (CABG) surgery was the prototype for hospital and surgeon report cards for three primary reasons. First, CABG surgery is common. This means that mortality rates for individual providers can be accurately measured and also that aggregate costs are high, thereby increasing government and insurer interest. Second, mortality rates for CABG surgery vary measurably for different providers. Third, risk-adjustment methods are well developed to account for case mix differences at individual hospitals.
An analysis of report cards for nontraumatic adult craniotomy surgery was recently conducted. The key components of CABG surgery are also common to this procedure: high costs, large caseloads, mortality rates that vary between providers, and available risk-adjustment methods. The results of this study, Hospital "Report Cards" for Mortality after Nontraumatic Adult Craniotomy: Not Ready for Prime Time?, will be presented by Frederick G. Barker, MD, 10:30 to 10:45 a.m. on Wednesday, April 26, 2006, during the 74th Annual Meeting of the American Association of Neurological Surgeons in San Francisco.
Many different types of organizations, including federal and state governments, insurance providers, and coalitions representing corporate purchasers of health care, now mandate or prepare report cards for a variety of different surgical procedures. Often these report cards are made directly available to the public on the Web, with varying degrees of accompanying comment and explanation.
The Agency for Healthcare Research and Quality (AHRQ), a branch of the US Department of Health and Human Services, distributes free software which calculates the expected and actual "mortality after craniotomy" based on administrative data that hospitals collect routinely. The validity of simulated hospital report cards generated from Nationwide Inpatient Sample (NIS) data using methods similar to those promoted by AHRQ is the subject of this study. The NIS is an administrative discharge database that provides a simple description of each inpatient admission for a given year at a sample of hospitals that approximates one-fifth of all inpatient admissions in the United States.
Three states to date have provided Web reports of individual hospital post-craniotomy mortality (including identification of hospitals with unusually high or low mortality rates after the procedure), while at least three other states have used similar data internally. Utilizing 20,301 craniotomy admissions in 2002 at 317 hospitals, a risk-adjustment model paralleling the AHRQ model was developed. The AHRQ model divides craniotomies into a four-stratum risk model that does not explicitly account for potentially important factors such as type of hospital admission (emergency or elective) or specific clinical diagnoses. The parallel model for this study uses very similar risk prediction methods, but employs logistic regression rather than linear regression as used by the AHRQ, thus avoiding negative risk predictions for some centers.
Examination of the NIS data suggests that the simple four-stratum model that parallels the AHRQ model fails to predict risks accurately for important classes of patients, including patients with emergency hospital admissions and those with a diagnosis of subarachnoid hemorrhage (SAH). The following outcomes were noted:
"¢"Moderate-risk" emergency room craniotomy admissions had 8.1 percent mortality vs. 3 percent for "moderate risk" craniotomy admissions that were from routine sources.
"¢"Moderate-risk" SAH craniotomy admissions had 18 percent mortality vs. 3.2 percent for "moderate-risk" craniotomy admissions for other diagnoses.
"¢When the two additional risk factors of emergency room admission and SAH diagnosis were added, this improved the risk prediction model significantly.
To demonstrate that a risk prediction model does not perform fairly in hospital-to-hospital comparisons, it is necessary to show two things: 1) That there are groups of patients whose risk is inadequately described by the model; 2) That these patients are distributed unequally between hospitals. Hospitals varied widely in the fraction of craniotomy admissions that were emergency room and SAH craniotomy admissions. Consequently, the AHRQ-based model without inclusion of these additional two factors appeared to be biased toward lower apparent "quality" (i.e. observed mortality higher than expected mortality) for hospitals with frequent emergency or SAH admissions.
The improved model with the two additional risk factors reduced differences between mean per-hospital observed and predicted mortality (i.e., reduced errors in risk predictions) by 34 percent for the lowest-emergency-admission quartile of hospitals and by 59 percent for the highest-emergency-admission quartile of hospitals.
"Risk-adjustment models available to date, including the model distributed by AHRQ, are probably not adequate for public reporting of hospital 'quality' for craniotomy, and certainly have been little tested," stated Dr. Barker. "Proper risk adjustment is critically necessary in order to avoid unfair assessments of poor quality for providers that tackle high-risk patients. Avoiding this type of disincentive is important in maintaining proper access to care for the sickest patients," concluded Dr. Barker.
Founded in 1931 as the Harvey Cushing Society, the American Association of Neurological Surgeons (AANS) is a scientific and educational association with more than 6,800 members worldwide. The AANS is dedicated to advancing the specialty of neurological surgery in order to provide the highest quality of neurosurgical care to the public. All active members of the AANS are certified by the American Board of Neurological Surgery, the Royal College of Physicians and Surgeons (Neurosurgery) of Canada or the Mexican Council of Neurological Surgery, AC. Neurological surgery is the medical specialty concerned with the prevention, diagnosis, treatment and rehabilitation of disorders that affect the entire nervous system, including the spinal column, spinal cord, brain and peripheral nerves.