LSU recently joined Meta’s open science efforts with five artificial intelligence-powered health projects across the state, with $300,000 in financial support from Meta.
Each project was presented at the Meta Open Source AI for Science Summit in October, which hosted nearly 100 scientists across the country, according to an LSU press release. Three out of the five projects are taking place on LSU’s Baton Rouge campus, each trying to solve a different healthcare issue with AI.
“No single campus can do this alone, which is the point of leveraging a university system tied to a unified research enterprise with two health sciences centers,” said LSU President Wade Rousse in the press release. “This partnership with Meta recognizes what LSU can do when we leverage all the components of our brand. It’s why we’re proud to be one LSU.”
Felipe Fronchetti, LSU assistant professor of computer science and engineering, is working on a project to develop a medical care virtual reality training software called COPES. The software stands for communication, optimal pathway, procedural training, environment and specific coaching. Fronchetti said that the project is in collaboration with Dr. Peter DeBlieux, the LSU Health New Orleans assistant dean for advanced learning and simulation.
The software uses the AI models Muse Glimmer and Muse Spark, according to Fronchetti. He said trainees will be able to practice operations like chest tube insertions and interact with virtual reality patients that can react to speech using AI.
“I approach the patient, the patient can be anxious, the patient can be stressed, and the interactions are not, they are not scripted,” Fronchetti said. “Meaning that if I ask the wrong question, the patient is going to behave in one way; if I ask the right question, it’s going to be more calm.”
He added that AI is also used to react to the trainee to give coaching in real time. The team at the Baton Rouge campus received $80,000 of the funding to develop the program.
Fronchetti said that the program may be useful to rural areas of Louisiana, as they have less access to training for specific procedures. He also said that virtual reality technology could be more affordable than any previous training method.
Supratik Mukhopadhyay, LSU environmental science professor, is working on the other two projects taking place in Baton Rouge. One of the projects is a tool that predicts the future development risks of head and neck cancer patients. He is working with LSU Health New Orleans Assistant professor Dr. Sagar Kansara on the project.
“Currently, we are trying to use this data for discovering drugs for head and neck cancer that can prevent recurrence,” Mukhopadhyay said.
The project will use data from various institutions like the LSU Health Sciences Center and Perelman School of Medicine at the University of Pennsylvania, according to Mukhopadhyay. He explained that the program will run on an open weight AI model, which means no data is ever uploaded to the cloud.
“A variable can be highly predictive without being what drives the outcome, and separating those two things takes more than data.” Mukhopadhyay said in the press release. “Meta’s open-weight AI models encode what the literature knows as constraints on the causal structure, so the same reasoning can run over records that arrive as scans.”
Mukhopadhyay also works with Dr. Michael Dunham for the other project, who is a professor of medicine and director of the Medical Innovation Design Studio at LSU Health New Orleans.
The project is called “Eyes-On” and uses cribs with cameras to look after babies that need technological support for breathing, according to Mukhopadhyay. Babies may disrupt medical interventions helping them stay healthy with their movements, but doctors need to know when to step in.
“Now of course it’s an infant, right?” Mukhopadhyay said. “They’re moving around. You know, throwing their arms, throwing their legs and so on.”
He said the cameras can detect whether the baby is distressed or if their breathing tube is displaced. Mukhopadhyay added that if it detects that something is wrong, then it will send alerts to the hospital staff for them to take action.
This project uses Muse Glimmer, Distillation with No Labels v3 and Segment Anything Model 3, according to the press release. All three three programs are also open weight.
“This is patient data, right?” Mukhopadhyay said. “We cannot send it out to Florida, open ChatGPT or anything that might be sitting in a data center thousands of miles away.”
The use of AI with these cameras, specifically Meta’s open science efforts, can help doctors act faster when an infant is in need.
“Most medical AI needs thousands of annotated patient images before it can do anything useful—we wanted to skip that step,” Mukhopadhyay said in the press release. “A clinician traces the tube once, on one baby, and the models take it from there. This is only possible because of Meta’s computer vision and open-weight models.”