publications

2024

1. arXiv

   Personalized language modeling from personalized human feedback

   Xinyu Li, Ruiyang Zhou, Zachary C Lipton, and Liu Leqi

   arXiv preprint arXiv:2402.05133, 2024

   Abs HTML Code

   Personalized large language models (LLMs) are designed to tailor responses to individual user preferences. While Reinforcement Learning from Human Feedback (RLHF) is a commonly used framework for aligning LLMs with human preferences, vanilla RLHF assumes that all human preferences share the same distribution, preventing fine-tuned LLMs from generating personalized content when user preferences are diverse. In this work, we propose Personalized-RLHF (P-RLHF), an efficient framework that utilizes a lightweight user model to capture individual user preferences and jointly learns the user model and the personalized LLM from human feedback. P-RLHF exhibits the following three characteristics: (1) It enables an LLM to generate personalized content and scale efficiently with growing number of users. (2) It handles both explicit user preferences described as textual input and implicit user preferences encoded in the feedback data. (3) It eliminates the need for users to fully articulate their preferences, which are normally needed for prompting LLMs to generate personalized content yet are often impractical to obtain in real-world scenarios. Our experimental results show that personalized LLMs trained using P-RLHF generate responses that are more closely aligned with individual user preferences, outperforming vanilla, non-personalized RLHF and prompting-based personalization approaches across different tasks. We opensource our code at https://github.com/HumainLab/Personalized_RLHF.

2. AAAI

   Adapting Animal Models to Assess Sufficiency of Fluid Resuscitation in Humans (Student Abstract)

   Ryan Schuerkamp, Xinyu Li, Brian Kunzer, Leonard S Weiss, and 4 more authors

   In Proceedings of the AAAI Conference on Artificial Intelligence, 2024

   Abs HTML

   Fluid resuscitation is an initial treatment frequently employed to treat shock, restore lost blood, protect tissues from injury, and prevent organ dysfunction in critically ill patients. However, it is not without risk (e.g., overly aggressive resuscitation may cause organ damage and even death). We leverage machine learning models trained to assess sufficiency of resuscitation in laboratory animals subjected to induced hemorrhage and transfer them to use with human trauma patients. Our key takeaway is that animal experiments and models can inform human healthcare, especially when human data is limited or when collecting relevant human data via potentially harmful protocols is unfeasible.

2022

1. Sensors

   Automated Assessment of Cardiovascular Sufficiency Using Non-Invasive Physiological Data

   Xinyu Li, Michael R Pinsky, and Artur Dubrawski

   Sensors, 2022

   Abs HTML

   For fluid resuscitation of critically ill individuals to be effective, it must be well calibrated in terms of timing and dosages of treatments. In current practice, the cardiovascular sufficiency of patients during fluid resuscitation is determined using primarily invasively measured vital signs, including Arterial Pressure and Mixed Venous Oxygen Saturation (SvO2), which may not be available in outside-of-hospital settings, particularly in the field when treating subjects injured in traffic accidents or wounded in combat where only non-invasive monitoring is available to drive care. In this paper, we propose (1) a Machine Learning (ML) approach to estimate the sufficiency utilizing features extracted from non-invasive vital signs and (2) a novel framework to address the detrimental impact of inter-patient diversity on the ability of ML models to generalize well to unseen subjects. Through comprehensive evaluation on the physiological data collected in laboratory animal experiments, we demonstrate that the proposed approaches can achieve competitive performance on new patients using only non-invasive measurements. These characteristics enable effective monitoring of fluid resuscitation in real-world acute settings with limited monitoring resources and can help facilitate broader adoption of ML in this important subfield of healthcare.

2021

1. IEEE JBHI

   Deep survival machines: Fully parametric survival regression and representation learning for censored data with competing risks

   Chirag Nagpal, Xinyu Li, and Artur Dubrawski

   IEEE Journal of Biomedical and Health Informatics, 2021

   Abs HTML Code

   We describe a new approach to estimating relative risks in time-to-event prediction problems with censored data in a fully parametric manner. Our approach does not require making strong assumptions of constant proportional hazards of the underlying survival distribution, as required by the Cox-proportional hazard model. By jointly learning deep nonlinear representations of the input covariates, we demonstrate the benefits of our approach when used to estimate survival risks through extensive experimentation on multiple real world datasets with different levels of censoring. We further demonstrate advantages of our model in the competing risks scenario. To the best of our knowledge, this is the first work involving fully parametric estimation of survival times with competing risks in the presence of censoring.

2019

1. MLHC

   Dynamically personalized detection of hemorrhage

   Chirag Nagpal, Xinyu Li, Michael R Pinsky, and Artur Dubrawski

   In Machine Learning for Healthcare Conference, 2019

   Abs HTML

   Rapid detection of hemorrhage is of major interest to the critical care community, enabling clinicians to take swift actions to mitigate adverse outcomes. In this paper, we describe a model that allows rapid detection of the onset of hemorrhage by monitoring the Central Venous Pressure (CVP). As opposed to prior work in the domain, our model does not rely on prior availability of a stable physiology of a patient as a baseline of reference, and it makes generative assumptions on the monitored vital sign. This allows for rapid on-the-fly personalization to a previously unseen patient’s physiology. This property makes the proposed approach particularly relevant to e.g. trauma care and other scenarios where reference hemodynamic data may not be readily available for any new patient. We compare our model against strong discriminative alternatives and demonstrate its potential utility through empirical evaluation.

2018

1. ML4H

   Leveraging Routine Pre-Operative Blood Draws to Predict Hemorrhagic Shock During Surgery

   Xinyu Li, Michael R Pinsky, Gilles Clermont, and Artur Dubrawski

   NeurIPS Machine Learning for Health (ML4H) Workshop, 2018

   Abs PDF

   We investigate the utility of machine learning to estimate the likelihood of surgical patients to develop a hemorrhagic shock during surgery. We use hemodynamic responses to routine blood draws as virtual indicators of future shock. Our results suggest that by capturing sequential patterns in vital signs waveform data observed in pre-surgical settings, it is possible to predict which patients will likely get into shock in the course of surgery. These findings, after further validation, can enable building clinically useful screening tools to preempt complications in surgical patients and to inform medical resource allocation and planning, with the use of only already performed diagnostic procedures: routine lab test blood draws.