Doctors Overestimate Success of Multi-Step Medical Procedures

Newswise — Whether a woman is giving birth or a man is having a cancer biopsy, medical care is rarely straightforward. At any point, an unexpected complication can arise, particularly during treatments that require several steps.

Physicians tend to have unrealistic expectations of the success of multi-step medical procedures, according to a new nationwide study conducted by University of Utah Health scientists and their collaborators. They say inflated estimates of success could adversely influence treatment decisions and lead to unintended harm to patients.

In all, nearly 8 out of 10 physicians surveyed believed the chances of obtaining a desired outcome after a medical procedure were greater than the probability of the success of one or more steps leading to that outcome.

The study, which appears in JAMA Network Open, highlights a serious logical disconnect among physicians who fail to consider that each step in the process has its own risks that can diminish the chances of success for the desired medical outcome, according to Scott Aberegg, M.D., a critical care pulmonologist at U of U Health.

“All too often, doctors act as though the stars align more frequently than they actually do,” Aberegg says. “They tend to focus on the desired outcome rather than the actual chances of success involved in each intermediary step. We can’t continue making medical decisions that way. We need to base them on more realistic expectations.”

The study, conducted by Aberegg; Hal Arkes, Ph.D., of The Ohio State University; and Kevin Arpin, Ph.D., a forensic specialist at Travelers Insurance company in Connecticut, sought to determine how frequently a phenomenon called conjunction fallacy happens in medicine.

Conjunction fallacy occurs when a person believes that a combination of events or statements is more likely than any one of its individual components.

Suppose, for instance, that a doctor notices skin growth on a patient and has 80% suspicion that it is cancerous. There’s also an 80% chance that the pathologist sees cancer on a biopsy specimen in the lab. The false assumption—the conjunction fallacy––would be that there’s more than an 80% chance that the pathologist will see cancer on the patient's biopsy specimen.

In reality, the probability that the pathologist will see cancer on this patient's biopsy is 64%, because first the patient actually has to have cancer and then the pathologist has to see it on the biopsy.

“Many physicians simply aren’t good at calculating probability,” Aberegg says. “As a result, they commonly miss opportunities to make better treatment decisions.”

In their study, Aberegg and colleagues asked 215 obstetricians and pulmonologists to evaluate scenarios that they might encounter while caring for patients.

For example, in one scenario, obstetricians were confronted with a 29-year-old pregnant woman in labor. However, the child is not positioned properly for a vaginal birth. In this case, the doctors were asked to estimate the probability that the child would move into a deliverable position and be born without the need for a C-section.

Overall, 78% of the physicians who evaluated one of three scenarios in the survey estimated that the probability of the desired outcome would be greater than the likelihood of the two individual events required for it to occur. This is a mathematical impossibility, Aberegg says.

“Our study shows that if you poorly estimate the probability of two events needing to happen to get the result you desire, then you could be putting your patients at unnecessary risk,” Aberegg says. “In the case of the childbirth scenario, you could end up waiting around for a long time for that baby and end up having to do a C-section anyway. That delay could be harmful for both mother and child.”

All of the physicians who participated in the surveys had an average of 25 years of experience. Yet this expertise did not appear to prevent them from opting for the conjunction fallacies presented in the study. However, this isn’t too surprising since previous research found that nearly 50% of medical students are prone to these types of probability errors, according to Aberegg.

“There are enormous opportunities in medical education to improve the curriculum in terms of teaching the importance of probability in medical settings,” Aberegg says. “Numbers are the most reliable source of correct decisions in medicine.”

Aberegg urges practicing physicians to not only rely on their experience but also do their best to stay up-to-date on the latest probability research published in medical journals about various conditions and procedures.

Among the study’s limitations is that the participants were asked for written responses that might have been different had they been providing care to real patients.

However, Aberegg believes the study could have broad implications.

“Our results are very strong,” Aberegg says. “We’re confident that they represent a generalized phenomenon in medicine. I’m interested in further cataloging more examples so that the full breadth of this potential problem can be exposed and hopefully resolved.”

                                                            ####

In addition to Aberegg, Hal Arkes, Ph.D., of The Ohio State University; and Kevin Arpin, Ph.D., a forensic specialist at Travelers Insurance company in Connecticut; contributed to this study. 

The study, "Analysis of Physicians' Probability Estimates of a Medical Outcome Based on a Sequence of Events,"appears in JAMA Network Open. The research was self-funded by Aberegg, Arkes and Aprin.