Global Evaluative Assessment of Robotic Skills: Validation of a Clinical Assessment Tool to Measure Robotic Surgical Skills
Abstract
Purpose:
We developed and validated a standardized assessment tool for robotic surgical skills, and report its usefulness, reliability and construct validity in a clinical setting.
Materials and Methods:
The Global Evaluative Assessment of Robotic Skills is a tool developed by deconstructing the fundamental elements of robotic surgical procedures in consultation with expert robotic surgeons. Surgical performance was assessed during robot-assisted laparoscopic prostatectomy on a 5-point anchored Likert scale across 6 domains. An overall performance score was derived by summing the ratings in each domain. Expert surgeons and postgraduate year 4 to 6 urology residents were evaluated to determine construct validity. Assessments were completed by the attending surgeon, a trained observer and the operator.
Results:
A total of 29 evaluations of 25 trainees and 4 attending surgeons were completed. Experts scored significantly higher on the assessment than novice operators (p = 0.004). Postgraduate year 4 and 5 residents scored significantly lower than the expert group (p <0.05) while no difference was observed between mean performance scores of postgraduate year 6 trainees and attending surgeons (p >0.05). The internal consistency of the assessment tool was excellent (Cronbach's α = 0.90 to 0.93). The overall assessment score ICC among raters was 0.80 (95% CI 0.65–0.90).
Conclusions:
The Global Evaluative Assessment of Robotic Skills is simple to administer and able to differentiate levels of robotic surgical expertise. This standardized assessment tool shows excellent consistency, reliability and validity. Further study is warranted to evaluate its usefulness for surgical education and the establishment of competency in robotic surgery.
References
- 1 : Validation of a novel virtual reality robotic simulator. J Endourol2009; 23: 503. Google Scholar
- 2 : Evolution of robot-assisted radical prostatectomy. Scand J Surg2009; 98: 76. Google Scholar
- 3 : Acquiring skills in new procedures and technology: the challenge and the opportunity. Arch Surg2005; 140: 387. Google Scholar
- 4 : The new paradigm of continuing education in surgery. Arch Surg2005; 140: 264. Google Scholar
- 5 : The financial impact of teaching surgical residents in the operating room. Am J Surg1999; 177: 28. Google Scholar
- 6 : Lessons learned from the evolution of the laparoscopic revolution. Surg Clin North Am2008; 88: 927. Google Scholar
- 7 : A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg2005; 190: 107. Google Scholar
- 8 : Validation of laparoscopic surgical skills training outside the operating room: a long road. Surg Endosc2009; 23: 1476. Google Scholar
- 9 : Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg1997; 84: 273. Google Scholar
- 10 : Evaluation of resident laparoscopic performance using global operative assessment of laparoscopic skills. J Am Coll Surg2007; 204: 308. Google Scholar
- 11 : Best practices for robotic surgery training and credentialing. J Urol2011; 185: 1191. Link, Google Scholar
- 12 : Training, credentialing, proctoring and medicolegal risks of robotic urological surgery: recommendations of the Society of Urologic Robotic Surgeons. J Urol2009; 182: 1126. Link, Google Scholar
- 13 : Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med1998; 73: 993. Google Scholar
- 14 : Robot-assisted radical prostatectomy: learning rate analysis as an objective measure of the acquisition of surgical skill. BJU Int2010; 106: 855. Google Scholar
- 15 : From proficiency to expert, when does the learning curve for robotic-assisted prostatectomies plateau?: The Columbia University experience. World J Urol2007; 25: 105. Google Scholar
- 16 :
Kendall's coefficient of concordance . In: Encyclopedia of Statistics in Behavioral Science. Edited by . Hoboken: John Wiley&Sons2005. Google Scholar - 17 : Training for minimally invasive surgery: Need for surgical skills. Surg Endosc1993; 7: 93. Google Scholar
- 18 : The effect of practice on performance in a laparoscopic simulator. Surg Endosc1998; 12: 1117. Google Scholar
- 19 : The impact of intense laparoscopic skills training on the operative performance of urology residents. J Urol2001; 166: 1658. Link, Google Scholar
- 20 : Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc2010; 24: 1834. Google Scholar
- 21 : Face validation of a novel robotic surgical simulator. Urology2010; 76: 357. Google Scholar
- 22 : Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology2009; 73: 1288. Google Scholar
- 23 : Does training on a virtual reality robotic simulator improve performance on the da Vinci surgical system?. J Endourol2010; 24: 467. Google Scholar
- 24 : Should experienced open prostatic surgeons convert to robotic surgery?: The real learning curve for one surgeon over 3 years. BJU Int2010; 106: 378. Google Scholar
- 25 : Robotic surgical education: a systematic approach to training urology residents to perform robotic-assisted laparoscopic radical prostatectomy. Urology2006; 68: 75. Google Scholar
- 26 : Reliability: on the reproducibility of assessment data. Med Educ2004; 38: 1006. Google Scholar
- 27 : Training, credentialling, and granting of clinical privileges for laparoscopic general surgery. Am J Surg1991; 161: 399. Google Scholar
- 28 : The endoscopic surgical skill qualification system in urological laparoscopy: a novel system in Japan. J Urol2006; 176: 2168. Link, Google Scholar
