Is competency assessment at the specialist level achievable?
The celebrated surgeon and founder of Johns Hopkins Hospital William Stewart Halsted once commented: the advent of anaesthesia has made it so that any idiot can become a surgeon. After the introduction of antiseptic surgery in the late 19th century, the practice of surgery was poorly regulated and variable in its quality.
Even as late as the 1970s, when Jatene pioneered the now routine arterial switch procedure for congenital transposition of the great arteries, the operative mortality was 60 per cent (1). Other surgeons around the world tried to emulate the procedure with similarly poor results.
The climate today is very different, but there are still some similarities with previous eras. Surgery today is safer than it has ever been in the past. However, there still remains considerable variation in performance between surgeons and institutions: recent work by Burns and colleagues has shown a sixfold difference in reoperation rate after elective hemicolectomy (2).
The combination of increasing patient involvement in healthcare and recent high-profile scandals of poor medical and surgical care have focused public attention on individual and institutional performance and resulted in a public demand that doctors be certified as competent. Poor performance, or even the appearance of poor performance, is less tolerated now than it ever has been in the past.
In the United Kingdom, this has resulted in two major changes. First, the General Medical Council, the body that regulates doctors, has led the introduction of mandatory revalidation every five years for all doctors that wish to retain practising privileges. An essential part of this is the demonstration and documentation of individual competency, which to date has been difficult in many specialties. Secondly, surgeons have been obliged to provide publicly accessible data on the outcomes of patients under their care, a measure which has been hugely controversial as it assumes a correlation between individual technical ability and post-operative outcome. One of the particular difficulties with this approach is that during the time it takes to obtain reliable, robust, comprehensive data incompetence may lead to morbidity and mortality. Most patients would surely prefer a practitioner deemed competent to perform their surgery.
Generations of surgeons have had no assessment of this ability and have relied on the assumption that time-based experiential learning was sufficient to imbue each trainee with the skills required to perform a range of complicated procedures. The technical complexity of surgery is also increasing with technological advances, such as laparoscopic and robotic surgery. In addition, the demise of the general surgeon and the rise of the specialist has meant that surgeons perform a significantly smaller range of procedures but at higher volumes and of a more specialist nature compared to their predecessors.
Given this, there is a growing transition from reliance on experiential learning to competency based assessments. Such assessments are now common in medicine with all UK trainees required to complete structured Directly Observed Procedural Skills (DOPS) assessments to gauge their ability to perform myriad tasks, from venepuncture to intercostal drain insertion.
The DOPS process has been specifically applied for some years in the certification of competency of endoscopists in the UK. As part of an effort to improve standards in colonoscopy nationwide, a cross-specialty national framework was devised whereby endoscopists would be assessed formatively whilst training and then undergo a structured summative assessment using the same DOPS tool. Recently an additional tool, the Direct Observation of Polypectomy Skills (DOPyS), has become a mandatory part of competency based assessment for endoscopists in training.
Since the introduction of certification, national technical proficiency in colonoscopy has improved inexorably with commensurate improvements in patient outcomes (3). Moreover these assessments are also used to determine competency at the higher level required to achieve Bowel Cancer Screening status. Although resisted at the outset, this accreditation process is now validated, well respected and even sought after as recognition of specialist skills in already independently practising endoscopists.
Miskovic and colleagues should be applauded for the creation of a robust methodology for the deconstruction of a complex multistep surgical procedure into component parts and the meaningful assessment process derived from these consensus criteria. The development of a validated objective scoring system in the setting of a national laparoscopic training program has significant implications both nationally and internationally in the way in which the acquisition and assessment of surgical skills are delivered.
It is of particular note that this study assessed those already undertaking independent practice as consultants as well as trainees at an earlier stage in their careers. Given the pace of technological change in medicine, it is unlikely that skills learnt during the latter stages of training today will be sufficient to sustain a specialist throughout an entire career. New competencies will therefore be necessary to keep pace with technological advances.
The benefits of minimal access surgery have been widely accepted for over a decade yet it still only comprises a minority of surgical procedures where it could be utilised (2). Those practising older techniques exclusively have never been credentialled in the manner described in the Miskovic paper and yet comprise the majority of surgeons. Although the model that has been used in this case is laparoscopic surgery, similar techniques could easily be applied across the breadth of surgical practice or other complex tasks. However, there are some inevitable caveats with the utilisation of similar tools.
As the authors admit, if standards are to be improved in surgery this requires a healthy combination of both technical operative skill and decision-making. The introduction of a tool focusing on one aspect of performance alone may detrimentally affect perceptions of the importance of non-technical skills (NTS), an area of considerable research interest at present. Ultimately, the aim should be to have a robust assessment method encompassing the full gamut of abilities required to optimise patient outcomes, including patient assessment, technical operative skills, NTS and post-operative care.
And so, in response to the question posed: is competency assessment at the specialist level achievable?, the answer is a guarded yes. Any complex assessment process is onerous and demanding both in terms of time and financial resources and in our enthusiasm for competency assessments per se, we should be mindful not to treat them as a panacea. It is vital that these processes correlate with positive clinical outcomes as optimisation of patient care is ultimately paramount, something that wise surgeons have realised for centuries. For as Sir Astley Cooper commented 200 years ago, "The best surgeon, like the best general, is he who makes the fewest mistakes."
Acknowledgements
Disclosure: The authors declare no conflict of interest.
References
- de Leval MR. Lessons from the arterial-switch operation. Lancet 2001;357:1814. [PubMed]
- Burns EM, Bottle A, Aylin P, et al. Variation in reoperation after colorectal surgery in England as an indicator of surgical performance: retrospective analysis of Hospital Episode Statistics. BMJ 2011;343:d4836. [PubMed]
- Gavin DR, Valori RM, Anderson JT, et al. The national colonoscopy audit: a nationwide assessment of the quality and safety of colonoscopy in the UK. Gut 2013;62:242-9. [PubMed]
