Protocol for a Multimethod Evaluation of AI-Based Colorectal Cancer Risk Prediction in an Emulated Primary Care Electronic Health Record Environment: Trust, Usability, and Diagnostic Decision-Making
This multimethod study evaluates how general practitioners interpret and utilize AI-based colorectal cancer risk predictions within an emulated electronic health record environment by combining quantitative assessments of diagnostic decision-making with qualitative interviews on trust and usability, aiming to inform future prospective trials in routine primary care.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the quiet, often chaotic world of a family doctor's office, one of the most difficult tasks is spotting a specific type of illness before it becomes advanced. Colorectal cancer, a disease affecting the large intestine, is frequently missed in its early stages because its early warning signs can be vague or easily mistaken for less serious digestive troubles. While doctors rely on their training and the information available in patient records, the sheer volume of data and the subtle nature of symptoms make early detection a persistent challenge. To help, researchers have begun developing computer programs that use artificial intelligence to sift through medical records, looking for patterns that might signal danger long before a diagnosis is officially made. These tools promise to act as a second pair of eyes, highlighting risks that a human might overlook. However, a computer program is only as good as how well a doctor trusts it and understands it. If a tool is confusing or if the doctor does not know how to interpret its suggestions, it will not improve patient care, no matter how smart the algorithm is.
This study set out to test exactly how general practitioners interact with such a tool in a safe, controlled setting. The researchers created a simulated version of a primary care electronic health record, a digital system where doctors normally store patient notes and test results. They gathered historical records from real patients who had been diagnosed with colorectal cancer, removing all names and personal details to protect their privacy. Ten doctors, a mix of experienced specialists and those still in training, were invited to review between forty and sixty of these de-identified cases. The doctors worked through the records as if they were seeing the patients for the first time, but they were not told that every single person in the study had already been diagnosed with cancer. This setup allowed the researchers to see what the doctors would suspect and decide to do based solely on the information presented to them.
The experiment was designed to see if showing the doctors a computer-generated risk score changed their minds. For some of the cases, the doctors saw only the standard medical history, including symptoms, lab results, and past visits. For other cases, the same records were displayed, but with an added feature: a percentage indicating the computer's calculated risk of colorectal cancer, along with a list of the specific factors that led to that number. The doctors were asked to state whether they suspected cancer, how strong that suspicion was, and what they would do next, such as ordering a test or referring the patient to a specialist. They also had to explain which symptoms or findings in the record led them to their conclusion. After reviewing the cases, each doctor sat down for a private interview to discuss their experience, sharing their thoughts on whether the computer predictions felt useful, trustworthy, or confusing.
The study did not measure whether patients were cured or if the tool saved lives, because no real patients were involved in this phase. Instead, the focus was entirely on the human element of the process. The researchers wanted to know if the artificial intelligence tool influenced the doctors' reasoning, if it helped them spot the disease more accurately, or if it caused them to second-guess their own judgment. By comparing the doctors' decisions when they had the computer's help versus when they did not, the team could identify how the tool fits into the flow of clinical thinking. The interviews provided a deeper look into the doctors' feelings, revealing whether they found the risk scores easy to understand and whether they believed the tool was reliable enough to rely on in a real clinic.
At the time the study was reported, all ten participating doctors had finished their reviews and interviews, and the analysis of their answers was underway. The researchers noted that the simulated environment worked well, allowing them to test the tool without risking patient safety. They found that the process of having doctors review cases and then immediately discuss their thoughts was practical and manageable. The study is not a final verdict on whether these computer tools should be used in every clinic, but rather a crucial step in understanding how they work before they are tested in the real world. The insights gained here will help the researchers refine the design of the tool and plan a larger, future trial where the system is tested in actual primary care settings with real patients. The ultimate goal is to ensure that when such technology is introduced, it supports doctors effectively, helping them catch colorectal cancer earlier and more accurately, while fitting seamlessly into the busy reality of a doctor's day.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.