A Google research team developed a deep learning artificial intelligence (AI) system to detect referrable diabetic retinopathy from fundus images. A retrospective evaluation based on data from over 25,000 images of diabetic patients in Thailand suggested that the AI system had significantly higher sensitivity and slightly lower specificity compared with a panel of 13 human graders.16 A subsequent qualitative study by Google researchers evaluated the real-world impact of the algorithm as deployed in 11 clinics across Thailand.17 The findings of this study highlight many of the human factors and ergonomics aspects that emerge only once the AI is embedded in the real world, including:
a high degree of variation in the process of eye screening across different clinics variability regarding the physical environment including light conditions affecting the quality of fundus photos a high rejection rate of images by the AI system due to inadequate image quality even though human readers were able to screen the images additional workload of staff as they were trying to retake photos several times to ensure suitable image quality (for the AI), also resulting in increased patient waiting times increased numbers of non-essential referrals due to the inability of the AI to process lower-quality images causing frustration among patients who were required to travel large distances as a result significant backlogs during periods where the internet speed dropped, causing delays, patient frustration and increased stress levels among staff.
|