5 Salient Points

• Methodology: The study utilized Markov models and reinforcement learning to develop and validate the AI Clinician using data from two nonoverlapping intensive care databases: MIMIC-III and eRI. The databases contained high-resolution patient data, including demographics, vital signs, laboratory tests, medications, and diagnostic coding, for adult patients fulfilling sepsis-3 criteria.

• Model Development: The AI Clinician was based on a partially observable Markov decision process (MDP), modeling the patient's physiological state and decision-making process. The MDP was defined by a set of states, actions, transition probabilities, and rewards. The AI policy, representing optimal treatment decisions for maximizing long-term survival, was learned using policy iteration and evaluation techniques.

• Model Evaluation: The AI policy was evaluated using off-policy evaluation (OPE) with the Weighted Importance Sampling (WIS) method. Bootstrap confidence intervals were used to estimate the true distribution of the policy value in the test sets. The AI policy consistently outperformed the clinicians' actual actions, demonstrating its potential clinical value.

• Clinical Implications: The AI Clinician's treatment suggestions aligned with sensible clinical and biological parameters, making the decisions clinically interpretable. In an independent cohort, patients receiving treatments suggested by the AI Clinician had significantly lower mortality rates. The model's reinforcement learning approach allowed for personalized treatment strategies based on individual patient data.

• Prospective Evaluation and Future Directions: Although the study demonstrated promising results, prospective evaluation using real-time data and clinical trials is required for validation in real-world scenarios. The AI Clinician's application could be extended to different healthcare settings and interventions, potentially improving outcomes for various medical conditions beyond sepsis.

📝Research Appendix