Introduction:

Cystoscopy is the current gold standard for the evaluation of bladder cancer and is integral in the work-up of hematuria. As the elderly population of the United States grows, demand for urologists and cystoscopies is outpacing supply. One potential remedy is having advanced practice providers (APPs) perform cystoscopies. Reservations about APPs preforming procedures center on the lack of standardized training and liability concerns over interpretation of cystoscopy. Before urologists can measure APPs competence, the inter-rater reliability of urologist-performed cystoscopy is needed. In other words, do urologists agree with one another on completeness, interpretation and resultant action of cystoscopy? To establish the baseline agreement of physician-performed cystoscopies, we examined the degree of consensus between urologists in the analysis of digitally-recorded, flexible cystoscopies.    

Methods:

As part of a more extensive IRB-approved protocol, flexible cystoscopies were performed on six patients by a board-certified urologist who trained in a Society of Urologic Oncology approved fellowship. The de-identified images were recorded from a Storz digital, high-definition flexible cystoscope. The cystoscopy was indicated for either work-up of hematuria or bladder cancer surveillance and was performed in standard fashion assessing ureteral orifices, side walls, dome, and retroflexion. The urethra was not always included in the video due to IRB constraints of the larger protocol. Two external, unique board-certified urologists reviewed each video and completed an online Qualtrics survey. Agreement rates between paired-reviewers were compared for 3 aspects of cystoscopy: completeness, diagnosis, and action taken.  

Results:

The results from the twelve expert reviews of six cystoscopies are summarized in Figure 1. For completeness of exam, 83% of reviewer pairs agree the cystoscopy was sufficiently thorough (90% confidence interval: 49.8%, 96.2%). In terms of diagnosis, 67% of reviewer pairs agreed that a diagnosis could be made using the video (90% confidence interval: 34.7%, 88.3%). One hundred percent of reviewer pairs were in agreement in the action to be taken after video review, despite the unique pathologies encountered in the six videos (90% confidence interval: 60.7%, 100%).

Conclusion:

This pilot study found that board-certified urologists do not have high rates of inter-rater reliability when assessing diagnostic capability and completeness of cystoscopic examination. However, they were uniform in their actions based on the interpretation of the cystoscopy.  This work has implications for both residency training programs and incorporation of APPs into office-based procedures. While the sample size is small, these findings suggest there is room for standardization in the way urologists perform, interpret, and teach cystoscopies

Funding: N/A