Performance evaluation process helps surgeons improve surgical practice and teamwork

A performance evaluation process used widely by Fortune 500 companies accurately and effectively assesses surgeons' adherence to core standards of excellence and spurs changes in behavior to improve surgical practice and teamwork, according to a new study published online as an "article in press" in the Journal of the American College of Surgeons (JACS). The study will appear in a print edition of the Journal later this year.

Researchers at Ariadne Labs, Boston, applied multisource feedback methodology in 2011-2013 to query all 385 surgeons who practice in the eight Harvard Medical School-affiliated hospital surgical programs and gathered feedback from 2,928 surgical and nursing colleagues, supervisors, and subordinates so that surgeons could identify gaps between their own performance and desired goals involving not only technical proficiency but interpersonal and communication skills. This study is believed to be the first use of multisource or so-called 360-degree feedback in a large-scale, multi-institutional assessment of surgeons.

"Multisource feedback does not give people grades or try to regulate behavior. It provides tools for identifying how we can be better at what we do every day. In this study we assessed not only how well surgeons perform the technical aspects of surgery but also how they interact with people in the OR who contribute to patient safety and care and the individuals who guide the patient from the physician's office to the hospital, from one part of the hospital to the OR, and from recovery to home," according to lead author and surgical oncologist Alex B. Haynes, MD, MPH, associate director for the Safe Surgery Program at Ariadne Labs.

Multisource feedback is a key method of professional development for many industries; up to 90 percent of Fortune 500 companies use it on a regular basis. In medicine it is used by the American Board of Medical Specialties and the Accreditation Council of Graduate Medical Education as part of their assessment programs and by individual hospitals to measure small groups of physicians. The Harvard Medical School study is the first to apply the method to such a large group of surgeons across multiple facilities, according to William R. Berry, MD, MPH, MPA, FACS, a study coauthor and the chief medical officer at Ariadne Labs.

In the present study multisource feedback evaluated the degree to which surgeons in Harvard-affiliated surgical programs met an institutional code of excellence (COE) that is based on the American College of Surgeons Statements on Principles and defines a minimum standard of conduct in 11 domains: service, respect, teamwork, excellence, ethical discipline, personal responsibility to patients, openness, education, humility, health, and conflict of interest. It was established in 2011 by a group of surgical department heads convened by the Risk Management Foundation (RMF) of Harvard Medical Institutions, the captive malpractice insurer for Harvard facilities and physicians.

To help surgeons integrate the COE in their daily practice, department heads decided to measure performance in all 11 domains by means of multisource feedback. A working group of Harvard surgeons assembled a series of questions that would assess a surgeon's professionalism, communication skills, interpersonal style, leadership, and teamwork approaches. The questions were disseminated by a proprietary online 360-degree review system to every practicing surgeon and to 20-30 individuals who evaluated the performance of each surgeon. Responses produced a composite COE score for each surgeon. A follow-up survey of all surgeons and 1,042 reviewers assessed the effect of 360-degree evaluation on surgeons' behavior and participants' reactions to the methodology.

The study found that a high percentage of surgeons were adhering to COE principles. The mean COE score for all surgeons was 87.6, and the factors that contributed the most to the composite score were service, openness, and humility.

Surgeons nevertheless were altering practice patterns to improve performance. Notably, the follow-up survey indicated that 63 percent of surgeons had changed behavior as a result of the 360-degree evaluation and 60 percent of department heads noted overall improvement in surgeons' behavior, particularly in the areas of communication and professionalism.

The 360-degree evaluation process was well-received by participants. Eighty-seven percent of surgeons and 80 percent of surgical department heads believed the information was accurate; 77 percent of surgeons, 80 percent of department heads, and 85 percent of reviewers would participate in a similar evaluation process in the future.

"Many professionals do not get as much feedback about their performance as they would like. This study shows that people are energized by the opportunity to turn a mirror on themselves and that 360-degree feedback is an effective tool for providing opportunities for self-evaluation and improvement," Dr. Berry said.