The increasing integration of Artificial Intelligence (AI) into healthcare is reshaping medical diagnostics, particularly in dermatology. With skin cancer, especially melanoma, early and accurate detection is critical to patient outcomes. This is where Explainable AI (XAI) comes into play—offering not just predictions but also explanations that make AI decisions more transparent to clinicians. In a study by Chanda et al. (2024), researchers explored the impact of XAI on dermatologists' accuracy in diagnosing melanoma using eye-tracking technology. To capture how dermatologists engaged with AI systems, the study employed RealEye, a webcam-based eye-tracking software.
The study aimed to objectively assess how dermatologists engage with AI and XAI systems when diagnosing skin cancer lesions. A total of 76 dermatologists participated, and their visual attention patterns were tracked while diagnosing dermoscopic images of melanomas and nevi. To understand where and how their attention was allocated during diagnosis, RealEye was employed for the majority of participants.
RealEye is a webcam-based tool that records participants' eye movements as they complete visual tasks. In this study, it was crucial for determining the number of times and the duration for which dermatologists fixated on specific areas of interest, such as AI predictions and XAI-provided explanations. Participants completed two phases: one where they diagnosed with AI support (AI Phase) and another with XAI support (XAI Phase). RealEye enabled the research team to map out precise ocular fixations during these phases, correlating visual engagement with diagnostic accuracy.
In the AI phase, dermatologists used an AI tool to provide predictions ("melanoma" or "nevus") for 16 dermoscopic images. They had no access to explanatory features during this phase. Eye-tracking data was collected by RealEye as they assessed each image and compared their diagnoses with the AI's predictions. This phase captured visual data on how often and how long participants looked at key areas on the screen, such as the lesion itself or the prediction text.
In the XAI phase, dermatologists re-examined the same set of images but with additional support from the XAI tool, which not only gave a prediction but also offered region-based explanations (e.g., highlighting areas of the image most indicative of melanoma). RealEye tracked whether and how long participants focused on these explanations and whether their interaction with the image changed after being provided more contextual information.
Importantly, the software ensured that even though the task was conducted remotely, the data on where participants directed their attention remained reliable. For each participant, a calibration step was performed to ensure accurate tracking before they started diagnosing images.
The eye-tracking data collected via RealEye provided several key insights into how dermatologists interacted with both AI and XAI. One of the most important findings was that disagreements between the AI's prediction and the dermatologist's diagnosis significantly increased the number of fixations. In other words, when dermatologists did not initially agree with the AI, they spent more time reviewing the image, indicating a deeper cognitive engagement.
In contrast, data from RealEye revealed that XAI explanations, when provided, tended to increase the cognitive load, as evidenced by the additional time spent reviewing regions of interest highlighted by the XAI system. This was captured through increased fixation counts during the XAI phase, demonstrating that dermatologists took additional time to assess and reconsider their decisions based on the explanations provided.
Some of the major findings include:
To further validate the reliability of the RealEye system, a subset of dermatologists participated in an on-site phase using a remote eye-tracking device. This comparison between webcam-based tracking and remote screen-based hardware enabled researchers to assess the accuracy of the RealEye system. Notably, the trends identified through RealEye—such as increased fixations during instances of diagnostic disagreement—were supported by the remote eye-tracking system. This confirmation underscores the effectiveness of RealEye in providing accurate insights, even in a more accessible and less controlled environment. This validation step reinforced the robustness of the data collected via RealEye, ensuring that the study's conclusions are reliable and applicable to real-world diagnostic settings.
To gain deeper insights into how the RealEye system compares with remote eye tracking technologies, we invite you to read the article Comparison of Webcam and Remote Eye Tracking, which offers a detailed analysis of their respective performance and application contexts.
By recording how visual attention was distributed across diagnostic tasks, RealEye provided crucial evidence on how dermatologists interpret AI predictions and the impact of XAI explanations on their decision-making process.
This research highlights not only the value of eye-tracking in understanding human-AI interaction but also the role of RealEye as a powerful, accessible tool for such studies. As AI becomes more integrated into clinical practice, tools like RealEye will be essential in refining how clinicians engage with these technologies and in ensuring that they enhance rather than hinder the diagnostic process.
