Objective To construct a real-time monitoring system based on computer vision for monitoring withdrawal speed of colonoscopy and to validate its feasibility and performance. Methods A total of 35 938 images and 63 videos of colonoscopy were collected in endoscopic database of Renmin Hospital of Wuhan University from May to October 2018. The images were divided into two datasets, one dataset included in vitro, in vivo and unqualified colonoscopy images, and another dataset included ileocecal and non-cecal area images. And then 3 594 and 2 000 images were selected respectively from the two datasets for testing the deep learning model, and the remaining images were used to train the model. Three colonoscopy videos were selected to evaluate the feasibility of real-time monitoring system, and 60 colonoscopy videos were used to evaluate its performance. Results The accuracy rate of the deep learning model for classification for in vitro, in vivo, and unqualified colonoscopy images was 90.79% (897/988), 99.92% (1 300/1 301), and 99.08% (1 293/1 305), respectively, and the overall accuracy rate was 97.11% (3 490/3 594). The accuracy rate of identifying ileocecal and non-cecal area was 96.70% (967/1 000) and 94.90% (949/1 000), respectively, and the overall accuracy rate was 95.80% (1 916/2 000). In terms of feasibility evaluation, 3 colonoscopy videos data showed a linear relationship between the retraction speed and the image processing interval, which indicated that the real-time monitoring system automatically monitored the retraction speed during the colonoscopy withdrawal process. In terms of performance evaluation, the real-time monitoring system correctly predicted entry time and withdrawal time of all 60 examinations, and the results showed that the withdrawal speed and withdrawal time was significantly negative-related (R=-0.661, P<0.001). The 95% confidence interval of withdrawal speed for the colonoscopy with withdrawal time of less than 5 min, 5-6 min, and more than 6 min was 43.90-49.74, 40.19-45.43, and 34.89-39.11 respectively. Therefore, 39.11 was set as the safe withdrawal speed and 45.43 as the alarm withdrawal speed. Conclusion The real-time monitoring system we constructed can be used to monitor real-time withdrawal speed of colonoscopy and improve the quality of endoscopy.