When it comes to patient safety, there’s a problem. I am a certified orthopedic surgeon and we are the reporters who created the #SurgeonScorecard. Ask us anything.

The "200,000 to 400,000 die every year" sounds very scary, but it provides no real way to judge how prevalent medical mistakes are. What is the rate of fatal or debilitating error per 10,000 procedures? Can you put these numbers in some context?

That is a scary number. Those numbers are based on reviews of medical charts that are then extrapolated to the wider population. This OIG study estimated that about 13.5 percent of Medicare beneficiaries suffer an adverse event during a hospitalization. So that's getting a little closer to showing the prevalence of this type of harm. https://oig.hhs.gov/oei/reports/oei-06-09-00090.pdf

Wouldn't a more relevant number be the number of avoidable adverse events? If you perform coronary bypass surgery on 1000 patients, some fraction will not be healthy enough to survive the procedure. But usually such procedures are performed only on those at high risk of death from MI. Its not like a decision to have a botox injection.

On page 22 of that same OIG report the physician reviewers estimated that 44 percent of the harm suffered by patients was clearly or likely preventable.

That is very interesting, thanks. As a follow up, what kind of reporting needs to be implemented to track these incidents?

Dr. David Classen, one of the foremost experts in measuring patient harm (he created the Global Trigger Tool for the Institute of Healthcare Improvement), says it is possible to track harm carefully and quite comprehensively. He works with a company called Pascal Metrics that says they have the technology to track these type of adverse events in electronic medical records. He said that hospitals could be doing this much more effectively than they are. One of the problems, many experts say, is that the medical community has not had the incentive to carefully track things like medical errors or preventable harm. The public reporting of complication rates gives everyone the incentive to improve.

What has your group done to help expose situations whereby incompetent physicians are quietly shuttled between institutions without their incompetence ever truly being exposed? For example, "weak" reference checks and/or a lack of the previous institution's willingness to truly inform the next institution of the physicians problems?

We've had our hands pretty full with this analysis, but have heard of what you're talking about. If you have any examples you'd like to share, please email me at [email protected].

Any reponse to Dr. Davie's column, in which he states that complication rate is a relatively minor aspect of surgical quality in prostatectomy? I am not a surgeon myself, but easy to imagine that the more aggressively the tumor margins are pursued, the greater risk of impotence, incontinence, etc.

I'm not familiar enough with that column to address it directly, but can say that the complications we are looking at are just one way to measure the quality of care. It's important to know the rate of patients who died during the admission for the surgery, or were readmitted with a complication related to the surgery, but those are not the only considerations. The data we used, which is the only kind publicly available nationally for this type of research, can't answer every question, so we've limited our analysis to concrete type of complications that can be identified in the data. So patients would just need to speak to their doctors about the other ways surgical quality is measured.

What studies show that increasing transperancy leads to improved patient outcomes? Many have suggested that the primary effect is to dodge risky patients.

Great question - The National Patient Safety Foundation released a consensus report in January that's a broadside to anyone who opposes transparency. They called transparency the "magic bullet" to improve patient safety, and said the fears about its harms have not been realized. Here's a link to the report: http://www.npsf.org/?shiningalight

My spouse is an RN on a surgery floor. She routinely comes home with frustration about one or two docs in particular who's patients have many more complications than their partners. Sometimes patients would seem too old or sick for surgery and shouldn't have gone in the first place - others just appear to have been screwed up on.

However, unfortunately(?) these docs also have fantastic bedside manner -- and even patients with terrible complications (and even death soon after) leave feeling happy and convinced that their issues are normal, that the doc didn't do anything wrong.

My spouse and her coworkers all agree on this - and talk about how they would never let these docs cut on them - ever.

At the same time, my spouse feels powerless to advocate for change. She can't just tell patients that she thinks they can't walk because the doc is reckless -- and that she sees this every week. Docs are prized commodities in the hospital, she fears for her job if she spoke up too loudly or too the wrong person.

It would seem medical professionals like RNs would be great resources for this kind of safety information - because of what they see on the floor. But isn't that information going to waste without safe ways for them to report and share it?

We hear this type of thing often from nurses and other providers who work in hospitals. When we first started getting our surgeon-specific results we found that medical providers could often guess which surgeons had the highest complication rates. And you're correct that the mechanisms for overseeing surgeons are often broken. Many hospital quality improvement programs don't comprehensively track patient harm, and many peer review programs are undermined by conflicts-of-interest. We delved into some of these issues in the main story that accompanied the Surgeon Scorecard: https://www.propublica.org/article/surgery-risks-patient-safety-surgeon-matters.

Is there a metric for measuring difficulty/complexity of a procedure?

Not all tumors are the same just as not all cholecystectomies are the same. Is there any additional charting to identify these differences?

Between procedures, yes. There is the Resource-Based Relative Value Scale, which attempts to quantify how much more involved treating a boo-boo is compared to open heart surgery. It's the system by which Medicare determines how much to reimburse for services. A complex procedure would have a higher relative value than a simple procedure.

Naturally, not everyone likes the values or the system or the method by which it's maintained, but it exists and is used by every hospital in the US.

But does it evaluate me having 5% complications when my gall bladder removals have tiny pebbles whereas a colleague has the same/slightly higher rate but takes out the ones that are inflamed to the size of a grapefruit like he would one filled with pebbles?

No. For gall bladder removal it looks like the only difference in codes (47600 range) is x-rays/exploration/etc. Would there be a difference in the diagnosis code attached to the procedure?

I had never heard of this project, but I am an analyst at a hospital. Glancing through this AMA, it looks like they were just using billing data? Not clinical data? That's troubling. I'll read their paper tonight and get back with you about their methods.

Here's our full methodology: https://static.propublica.org/projects/patient-safety/methodology/surgeon-level-risk-methodology.pdf

DO you feel patients understand the strengths and weaknesses of the numbers you post?

It depends on the patient, I'm sure. But we do believe that patients are quite smart, and these days they're asked to take on more and more responsibility for their health care decisions. It's our belief that information is complex, but understandable. We're quite open about the limitations of the analysis and have found patients and the medical community to be quite receptive to what we've done.

How can I tell if my doctor is capable of making an error?

One other thing I would add about choosing a doctor or a hospital - I would go to the providers who are transparent about their results and welcome these type of conversations. I found it interesting the way doctors and hospitals responded when we contacted them about our findings. Some, like the Beaumont Health System in Detroit, welcomed us to come visit and were transparent in every way they could be. They did this even though their results were not great in our analysis. Others tried to confuse the issues, avoid the questions, or just refuse to comment at all. We delve into some of the issues here: https://www.propublica.org/article/surgery-risks-patient-safety-surgeon-matters

Any person is capable of making an error, so I'm not sure that's the right standard. But the best providers take exceptional care to ensure that every patient receives the best possible care. When we spoke to the surgeons who performed best in our analysis, we found that they went to exceptional lengths to do the best they could, and were also incredibly responsive and attentive to patients.

Any chance of getting data from non-Medicare patients?

A majority of the orthopedic procedures that are not total joint are private insured, workman's comp or private pay.

Great data though

Most states keep this same type of level for all-payers, but many of them do not make it available to the public for this type of work. Also, where states do make it available they usually do not provide the same type of information that Medicare provides, which restricts its usefulness for this type of analysis.

So you just created a Yelp for surgeons...why not just partner with Yelp and get this info out to the people?

Great idea. We're looking into all sorts of ways to help people find Scorecard and are open to suggestions.

But are you really open to suggestions? You keep mentioning that the next version will incorporate all the feedback you've received, but you've not been clear publicly about what concerns you've really considered valid and are making adjustments to change.

We talked to scores of doctors and surgeons, along with many of the top researchers in the country, when we developed this method. Before we published we also sent our methodology to groups like the American College of Surgeons and the American Hospital Association and American Medical Association, to get their feedback to ensure we were on the right track. As we move forward, we will continue to sort out what we are doing in a thorough manner, so the experts continue to guide our analysis.

Have you ever considered a relationship with insurance payers interested in using your data to create quality-based reimbursement structures with providers?

We have had a handful of insurance providers and other quality improvement groups contact us with interest in our data. That's something that the higher-ups at ProPublica will have to sort through. Above my pay grade.