HCI4H

Osler – Computer-Assisted Anamnesis

Osler is an innovative research project focused on improving the medical anamnesis process through the use of digital technologies. Our goal is to streamline and improve the patient-doctor interaction by leveraging artificial intelligence and human-computer interaction (HCI) principles. Osler aims to create a more efficient and comprehensive system for gathering and analysing patient information, ultimately leading to faster diagnoses and more accurate treatments.

At the heart of the Osler project are two key components:

  1. Osler Patient Application: A user-friendly platform where patients can complete their anamnesis, providing vital information about their medical history, symptoms, and overall health status. This interaction is guided by a computer-assisted system designed to adapt to the patient’s responses, making the process more dynamic and personalised.
  2. Osler Physician Application: A companion tool for doctors, which provides them with detailed, structured reports based on the patient’s responses. This system enables doctors to quickly review key information and gain insights into the patient’s condition, while also allowing them to offer feedback to improve future patient interactions.

By integrating these components, Osler seeks to enhance the quality of medical interviews, reduce the time burden on both patients and doctors, and ensure that critical health information is accurately captured and utilised.

The Osler project is a collaborative initiative involving INESC-TEC, the Human-Computer Interaction Institute at Carnegie Mellon University, and São João University Hospital. This partnership provides interdisciplinary expertise in artificial intelligence, human-computer interaction, and medical practice to ensure that the Osler system is not only technologically advanced but also practical, scalable, and aligned with real-world clinical needs.

The project is led by Nuno Rodrigues from INESC-TEC; John Stamper from Carnegie Mellon’s Human-Computer Interaction Institute; and João Coimbra from São João University Hospital.The project is funded by Fundação para a Ciência e a Tecnologia (FCT) under grant number 2022.07391.PTDC. We are committed to advancing the future of healthcare by providing a platform that empowers both patients and healthcare professionals, making the anamnesis process more effective and accessible for all.

The Problem

Misdiagnosis is still a major health care problem, with nearly half of all severe diagnostic errors resulting from failures of physicians to gather and analyse important data about the patient throughout medical history-taking (~10%), bedside examination (~10%), and bedside decision logic (~30%). The problem is of particular concern in the ED, where misdiagnosis is reported to account for the majority of errors, many resulting in serious injury or death.

Current anamnesis practice is impairing the quality of modern health care systems, reducing its capacity to reach a wider population, and preventing the development and adoption of personalised health care technologies that need quality data.

Physicians have limited time and are increasingly under pressure not just to see and treat more patients, but also to conduct other tasks, such as inputting information into electronic health systems. This makes seldom the case for a traditional face-to-face clinical encounter to produce a complete or, in some cases, even relevant medical history. Evidence of the impact caused by non-productive tasks assigned to physicians is reported in [1], where ambulatory practice clinicians were found to spend 20% of their day writing; in [2], dictation and charting outside examination rooms occupied 56 minutes of an eight-hour working day of an Ohio family practice; and in [3], the authors estimated that two thirds of the working day was spent recording information in an antenatal clinic.

While physicians are spending their time in non-medical tasks, the knowledge required to practice modern medical care is moving beyond human cognitive reach, through a continuous expansion driven by more than 75 trials and eleven systematic reviews being published every day [4]. A set of symptoms can have extensive differential diagnoses, requiring physicians to remember many questions relating to the exploration of each condition. Moreover, medical science changes established facts at an unprecedent rate, putting an extra burden on clinicians to keep up with the latest medical knowledge and practices. The only way for human capacity to deal with such a complex working environment is to elevate medical practice towards exercising the many tasks out of reach of current and future technology, with a particular emphasis on tasks that involve human reasoning and sensibility, and let the other for computers to resolve.

Electronic health record (EHR) was an important innovation in medical practice, but it does not incorporate any structural organisation for history taking, cannot improve adherence to particular anamnesis procedures and hence lack the benefits that come from automating such processes. Intelligent automation of anamnesis is a most needed component of second generation EHR systems and an inevitable development [5].

It doesn’t come as a surprise then that physicians incur in so many errors in the anamnesis process. There is a vast literature reporting on the dimension and consequences of this problem, like in [6], where 50% of psychosocial and psychiatric problems were reported being missed, and in [7], where 54% of patient problems and 45% of patient concerns were neither elicited by the physician nor disclosed by the patient.The problem is that current anamnesis practice is already falling short on fulfilling its role in traditional medical diagnosis and patient follow-up. The rapid pace of new medical knowledge production and lack of physicians (time) are just two aspects of modern health care systems that contribute to the current situation where physicians interrupt patients every 24 seconds in medical interviews [8], and primary care clinicians ask only 59% of essential history items that were relevant for deciding diagnostic and treatment plans [9]. How then will we meet the expectations of reaching the next level of health systems to improve diagnosis and treatment outcome with such a low-quality of input data? Given that these limiting aspects of the healthcare systems are unlikely to be resolved any time soon, the solution can only come from new methodologies and tools to aid the physician in the anamnesis process and produce the data necessary to fuel the next generation of AI healthcare systems.

Research Hypothesis

We have 2 main hypothesis we explore in Osler, addressing the 2 major reasons for adoption resistance of CAA mentioned in the literature. The first idea is to collect as much information as possible about the patient condition at his first interaction with system. This involves not just information the patient inputs to the system, but most importantly, the collection and runtime analysis of physiological data that can be obtained with the integration of a recently available range of certified biosensors (e.g. BP, SpO2, HR, BBT, ECG, GSR), including more complex sensors capable of performing blood analysis. The access to this data right from the beginning of the process allows to improve previous approaches in several ways: (1) use the physiological data to inform the digital anamnesis process – e.g., if a patient has high BP, the system will ask another set of specific questions; (2) dismiss the intervention of health professionals to collect basic physiological data; (3) reduce the number of interruptions in the diagnosis process for collecting physiological data, which adds cognitive load to the physician each time he as to re-evaluate a patient condition; and (4) facilitate physician decision-making.

The second idea is to go beyond the approach taken by previous CAA interventions of presenting mountains of patient data to the physician without proper pre-processing and visual treatment. Instead of delivering (more) data, we give tools to reduce the cognitive load of physicians and help them carry their work, through visual analytics and interactive mining functions over each patient data, comparison with similar recorded cases and medical algorithm recommendations and pre-filling.

Funding

2023-03-01 to 2024-08-31 | Contract
Fundação para a Ciência e a Tecnologia (Lisboa, PT)
GRANT_NUMBER: 2022.07391.PTDC
http://dx.doi.org/10.13039/501100001871

Publications

[P1] G. Ferreira, E. Oliveira, J. Stamper, A. Coelho, H. Paredes and N. F. Rodrigues, “A Human-Computer Interaction Perspective on Clinical Decision Support Systems: A Systematic Review of Usability, Barriers, and Recommendations for Improvement“, 2023 IEEE 11th International Conference on Serious Games and Applications for Health (SeGAH).

[P2] Silva E, Pereira MF, Vieira JT, Ferreira-Coimbra J, Henriques M, Rodrigues NF. “Predicting hospital emergency department visits accurately: a systematic review“. Int J Health Plann Mgmt. 2023.

[P3] A. Torneiro, E. Oliveira and N. F. Rodrigues, “Current Devices and Future Perspectives on Neuromuscular Blockade Monitoring: A Systematic Review“, 2023 IEEE 11th International Conference on Serious Games and Applications for Health (SeGAH).

[P4] N. Fernandes, E. Oliveira and N. F. Rodrigues, “Future Perspectives of Deep Learning in Laparoscopic Tool Detection, Classification, and Segmentation: A Systematic Review“, 2023 IEEE 11th International Conference on Serious Games and Applications for Health (SeGAH).

[P5] E. Oliveira, J. Ferreira, J. Alves, M. Henriques and N. F. Rodrigues, “Review Apps to Evaluate Stroke Risk in Prehospital Setting“, 2023 IEEE 11th International Conference on Serious Games and Applications for Health (SeGAH).

[P6] G. Pinto, B. Barroso, N. Rodrigues, M. Guimarães and E. Oliveira, “The Effects And Viability Of Video Games On The Rehabilitation Of Schizophrenic Patients: A Systematic Review“, 2023 IEEE 11th International Conference on Serious Games and Applications for Health (SeGAH).

[P7] E. Oliveira et al., “The Role of Kiosks on Health Services: A Systematic Review“, 2023 IEEE 11th International Conference on Serious Games and Applications for Health (SeGAH).

[P8] Silva, Eduardo and Ferreira-Coimbra, João and Oliveira, Eva and Henriques, Mariana and Rodrigues, Nuno F., “COVID-19 Impact on Forecasting Emergency Department Visits Performance“, 2022.

[P9] P. Pacheco, F. Santos, J. Coimbra, E. Oliveira and N. F. Rodrigues, “Designing Effective User Interface Experiences for a Self-Service Kiosk to Reduce Emergency Department Crowding“, 2020 IEEE 8th International Conference on Serious Games and Applications for Health (SeGAH).

Team Members