Play Live Radio
Next Up:
0:00 0:00
Available On Air Stations

Artificial intelligence could soon diagnose illness based on the sound of your voice

Yael Bensoussan, MD, is part of the USF Health's department of Otolaryngology - Head & Neck Surgery. She's leading an effort to collect voice data that can be used to diagnose illnesses.
Allison Long
Allison Long
Yael Bensoussan, MD, is part of the USF Health's department of Otolaryngology - Head & Neck Surgery. She's leading an effort to collect voice data that can be used to diagnose illnesses.

Voices offer lots of information. Turns out, they can even help diagnose an illness — and researchers are working on an app for that.

The National Institutes of Health is funding a massive research project to collect voice data and develop an AI that could diagnose people based on their speech.

Everything from your vocal cord vibrations to breathing patterns when you speak offers potential information about your health, says laryngologist Dr. Yael Bensoussan, the director of the University of South Florida's Health Voice Center and a leader on the study.

Dr. Olivier Elemento of Weill Cornell Medicine is the other co-principal investigator on the project.
Travis Curry / Olivier Element
Olivier Element
Dr. Olivier Elemento of Weill Cornell Medicine is the other co-principal investigator on the project.

"We asked experts: Well, if you close your eyes when a patient comes in, just by listening to their voice, can you have an idea of the diagnosis they have?" Bensoussan says. "And that's where we got all our information."

Someone who speaks low and slowly might have Parkinson's disease. Slurring is a sign of a stroke. Scientists could even diagnose depression or cancer. The team will start by collecting the voices of people with conditions in five areas: neurological disorders, voice disorders, mood disorders, respiratory disorders and pediatric disorders like autism and speech delays.

The project is part of the NIH's Bridge to AI program, which launched over a year ago with more than $100 million in funding from the federal government, with the goal of creating large-scale health care databases for precision medicine.

"We were really lacking large what we call open source databases," Bensoussan says. "Every institution kind of has their own database of data. But to create these networks and these infrastructures was really important to then allow researchers from other generations to use this data."

This isn't the first time researchers have used AI to study human voices, but it's the first time data will be collected on this level — the project is a collaboration between USF, Cornell and 10 other institutions.

"We saw that everybody was kind of doing very similar work but always at a smaller level," Bensoussan says. "We needed to do something as a team and build a network."

The ultimate goal is an app that could help bridge access to rural or underserved communities, by helping general practitioners refer patients to specialists. Long term, iPhones or Alexa could detect changes in your voice, such as a cough, and advise you to seek medical attention.

To get there, researchers have to start by amassing data, since the AI can only get as good as the database it's learning from. By the end of the four years, they hope to collect about 30,000 voices, with data on other biomarkers — like clinical data and genetic information — to match.

"We really want to build something scalable," Bensoussan says, "because if we can only collect data in our acoustic laboratories and people have to come to an academic institution to do that, then it kind of defeats the purpose."

There are a few roadblocks. HIPAA — the law that regulates medical privacy — isn't really clear on whether researchers can share voices.

"Let's say you donate your voice to our project," says Yael Bensoussan. "Who does the voice belong to? What are we allowed to do with it? What are researchers allowed to do with it? Can it be commercialized?"

While other health data can be separated from a patient's identity and used for research, voices are often identifiable. Every institution has different rules on what can be shared, and that opens all sorts of ethical and legal questions a team of bioethicists will explore.

In the meantime, here are three voice samples that can be shared:

Credit to SpeechVive, via YouTube.

The latter two clips come from the Perceptual Voice Qualities Database(PVQD), whose license can be found here. No changes were made to the audio.

Copyright 2023 NPR. To see more, visit

Carmen Molina Acosta
Carmen Molina Acosta (she/her) is a producer at Morning Edition, where she pitches and produces pieces and two-ways for the air and for the web. In February 2023, she helped produce the network's first bilingual State of the Union special coverage. In a past life, she worked in investigative journalism, where she dug into the use of solitary confinement against ICE detainees and the lack of protections for migrant workers during the pandemic. Her work has been published in The Associated Press and The Washington Post, among other outlets. Molina Acosta is trilingual and spent a year abroad living in central Italy and the south of France. She studied journalism and international development as a Banneker/Key scholar at the University of Maryland.
Lisa Weiner is a line producer on Morning Edition. For NPR, she's covered the humanitarian crisis in Afghanistan and traveled to Ukraine to cover the Russian invasion in 2022. Prior to joining NPR, she held positions as an editor at WTOP-FM, as an engineer at Radio Free Asia and recorded audio books for the Library of Congress. Weiner has a master's degree in audio technology from American University. She got her start in radio working the late-night shift as a student DJ in the basement of WRUR-FM at the University of Rochester. Weiner has lived in Tel Aviv, Israel, and Budapest, Hungary.