Have you ever talked to Siri or Alexa, only to find the virtual assistant didn’t interpret what you said exactly right? It’s a frustration for many people, especially non-native English speakers.
Now imagine a low-income Latina breast cancer survivor who did not finish high school. She’s afraid to ask her doctor all the questions she has about her health. That’s where Francisco Iacobelli’s intelligent tutoring system comes in.
Iacobelli is a visiting researcher at the Northwestern Center for Advancing Safety of Machine Intelligence (CASMI) and an associate professor of computer science at Northeastern Illinois University. His research team is designing a digital avatar that uses artificial intelligence (AI) technologies to interact with low-literacy minority individuals and to help them understand their medical conditions.
On May 10, Iacobelli gave a presentation entitled, “An Intelligent Tutoring System for the Minority: An Intentional Choice for Equitable AI.” The talk focused on his CASMI-funded research project and emphasized a tailored approach to address the needs of a particular community.
“A lot of the technology that we see now in the emerging field of fair, equitable AI aims to remove biases in AI products,” Iacobelli said. “Not a lot of it is aimed at designing solutions.”
U.S. government data shows 30 million Americans have below basic-level literacy skills. Of that group, roughly 12 million are Hispanics and another 6 million are Black. For reference, someone with basic skills can read cooking instructions or street signs.
Iacobelli demonstrated how an avatar named Dr. Maria Gonzalez can answer questions with minimal interruptions to the human, a key objective. The language model is being trained to recognize the pauses in natural conversations.
“This woman and this population that I was working with, they pause to finish their ideas and to formulate their ideas,” Iacobelli said. “Our speech recognition system needs to be able to understand that some pauses are yielding the floor, and some pauses are not.”
An Inclusive Approach to Research
Supporting research projects like these is critical to CASMI’s mission of building intelligent systems that are safe and effective, said Kristian Hammond, Bill and Cathy Osborn professor of computer science and director of CASMI.
“We build systems for everybody, and our notion of everybody is ‘everybody like me,’” Hammond said. “That means there are people who are left behind. We don’t hear their voices. There are ways to find those communities, bring them in, and serve them well.”
The tutoring system is specifically designed for low-literacy Latina breast cancer survivors, a group Iacobelli is interviewing. His diverse research team includes Hispanic students and non-traditional students, such as single parents and older adults. He believes a team like that is fundamental to bridge the gap between researchers and these marginalized communities to identify unique challenges in them. Iacobelli is also consulting with community leaders and physicians for this project.
“They’re going to give me a lot of good criticism,” Iacobelli said.
One community leader told Iacobelli that the avatar shouldn’t have a robotic voice because people would not engage with that. Human voices are now being recorded for the avatar to use.
Iacobelli’s research team includes high schoolers, undergraduate students, graduate students, and post-doctoral scholars. Northwestern third-year computer science major Kevin Rueda joined the research project in January. He built a configuration tool that helps people communicate with the intelligent tutoring system.
“The main reason I was interested in joining this project was that the people this tutoring system aims to help are lower literacy Hispanic individuals,” Rueda said. “These are people in my community. They represent some of my family members, my neighbors, and people I run into at the bus stop every day. These are people in my day-to-day life. It’s incredibly important and meaningful to work on this project, especially since navigating healthcare is hard enough on its own.”
Northwestern third-year computer science major Raquel Saldivar began working as a research assistant in October 2022. She was involved in implementing an interface that controls the avatar. She said she is passionate about helping minority populations.
“As a Latina woman myself, that’s something that’s close to me and something I’m very happy to help with,” Saldivar said. “I have family members who aren’t literate. I know a system like this would definitely be helpful. It’d be cool to see how it can expand.”
Iacobelli stressed the importance of being involved in the communities where you work. He is a member of the Medical Organization for Latino Advancement and the Chicago Cancer Health Equity Collaborative.
“To break structures of power and to build trust, you have to be involved in the community,” Iacobelli said.
How the Tutoring System Works
The tutoring system is designed so that people with proficient literacy skills (like doctors, community leaders, and researchers) can control the avatar. Computer scientists or web programmers can make other modifications so that the tutoring system can better interpret what the human is saying and use vocabulary that is easier for people to understand.
To train the model, Iacobelli accounted for instances in which Latina breast cancer survivors used the wrong word. For example, all the women Iacobelli worked with incorrectly believed that after breast surgery, they could develop lymphoma – cancer of the lymph nodes – when in fact the increased risk was for lymphedema, swelling caused by a blockage in the lymphatic system.
“I don’t want the system not to give them the information because it’s so picky about it being called the right word,” Iacobelli said. “It should understand the alternative.”
Breast cancer surgeons who have seen the digital avatar love it, Iacobelli said. But for him, it’s equally important for everyone who is impacted by these systems to feel the same way.
