Objective To establish an artificial intelligence deep learning model using clinical colonoscopy images and video to assist the diagnosis by colonoscopy. Methods More than 600 000 colonoscopy images were collected in endoscopic center of the Second Affiliated Hospital of Zhejiang University School of Medicine from 2014 to 2018, and endoscopic experts recorded a large number of high-quality operation video of colonoscopy as analysis data. After repeated discussion by six experts, the classified intestinal sites and pathological features were determined, and fuzzy and confusable images were deleted. The final selection result was approximately 1 out of 4. And then the features of images were marked using an independently developed software. The deep learning algorithm was developed using TensorFlow platform of Google. Results After repeated comparison and analysis of the results of machine training and judgment results combined with pathology from endoscopic experts, the sensitivity of the model for some diseases (such as colon polyps) was 99% under laboratory conditions. In the clinical colonoscopy test, the sensitivity, specificity, and overall accuracy of this model for diagnosis of colon polyps were 9830% (4 187/4 259), 8810% (17 620/20 000), and 9292% [2×9830%×8810%/(9830%+8810%)], respectively. The sensitivity and specificity for ulcerative colitis were 7832% (2 671/3 410), and 6706% (13 412/20 000), respectively. The diagnosis time spent on a single image was 05±003 s, and it was the real time for application, including system recognition, text prompt in video image, background record and storage. Conclusion The artificial intelligence assisted diagnosis model developed by our team can identify colonic polyps, colorectal cancer, colorectal eminence, colonic diverticulum, ulcerative colitis, etc. The auxiliary diagnosis model of colon disease can guide beginners to carry out colonoscopy, and can improve lesion detection rate, reduce misdiagnosis rate, and improve the overall operating efficiency of endoscopic center, which is conducive to the quality control of colonoscopy.