University of Alberta
Assessing the assessment: Inter-user variability and proficiency of ureteral wall thickness measurements in stone patients
Patrick Albers1, Nick Dean2, Alexandra Bain2, Matthew Mancuso1, Ryan McLarty2, Sentil Senthilselvan3, Tim Wollin2, Gillian Shiau4, Shubhadip (Shubha) K. De2.
1Faculty of Medicine, University of Alberta, Edmonton, AB, Canada; 2Department of Surgery, University of Alberta, Edmonton, AB, Canada; 3Division of Mathematics and Statistics, University of Alberta, Edmonton, AB, Canada; 4Department of Radiology, University of Alberta, Edmonton, AB, Canada
Objective: Ureteral wall thickness (UWT) has been shown to predict shockwave lithotripsy outcomes, spontaneous stone passage and stent insertion success rates. The primary objective was to evaluate inter-user variability in the measurement of UWT. Secondary objectives included a proficiency analysis of time spent.
Methods: Index cases were selected from our retrospective nephrostomy tube (NT) database (2013-19), having CT imaging for an obstructing ureteric stone prior to NT decompression. A six-minute standardized teaching video was created to teach measurement techniques. Four evaluators (2 students and 2 residents) independently measured UWT (mm) and ureteric density (HU). Intraclass correlation (ICC) was calculated using MedCalc with single and average measurement, absolute agreement two-way mixed model. Proficiency was assessed from serial time-per-study assessments (TPS).
Results: 89 index subjects were identified, comprising the study series. Average stone size was 10.9 mm, UWT 2.9 mm, and ureteric density 33.7 HU. The individual ICC was calculated at 0.314 (95% CI 0.191-0.443), and average-ICC was 0.647 (95% CI 0.485-0.761). The first 30 measures had an individual-ICC of 0.232 (95% CI 0.070-0.438) and average-ICC of 0.547 (95% CI 0.232-0.757); whereas the last 30 measures had an individual-ICC of 0.442 (95% CI 0.258-0.634) and average-ICC of 0.760 (95% CI 0.582-0.874). The Initial time per study (TPS) was 142±95s for the first 10 cases, improving to 69±27s for the final 10 cases. Inexperienced readers (medical students) took significantly longer to make measurements than resident physicians (152s vs 61s, p<0.001), though all groups showed 1.5-3.1x decrease in TPS.
Conclusions: Our analysis suggests that even though the reliability of individual UWT measures by novice users is initially poor, reliability can be improved with grouped outcomes (average intraclass correlation). Repetition also seems to play a role in improving reliability and time per study.
 Yamashita S, Kohimoto Y, Iguchi T, et al. Ureteral wall volume at ureteral stone site is a critical predictor for shock wave lithotripsy outcomes: comparison with ureteral wall thickness and area. Urolithias 2019;
 Sarica K, Eryildirim B, Akdere H, et al. Could ureteral wall thickness have an impact on the operative and post-operative parameters in ureteroscopic management of proximal ureteral stones? Actas Urol Esp. 2019;43:474-479
||Unmoderated Posters||ASSESSING THE ASSESSMENT: INTER-USER VARIABILITY AND PROFICENCY OF URETERAL WALL THICKNESS MEASUREMENTS IN STONE PATIENTS||TBD|