AI innovation helps create authentic, pitch perfect vocals

February 13, 2024

"Its precision and control can not only help musical artists and producers but also open new possibilities in areas such as voice rehabilitation and assistive technologies." In music, the ability of pitch-correcting software to adjust the sound of a voice or instrument to the desired pitch is crucial. Unlike older software that trained on pairs of corrected and original vocals, Diff-Pitcher analyzes the visual representation (spectrogram) of original vocals requiring correction. According to Hai, Diff-Pitcher is notable not only for its ability to create authentic and controlled voices but also for the method used to achieve it. They believe Dif-Pitcher has a wide range of potential applications, including aiding those undergoing voice rehabilitation, particularly those with vocal impediments.

Once a solution for amateur, off-key singers on karaoke nights, Auto-Tune and other voice tuning software have progressed from the club to the recording studio but never beyond sounding artificial.

But researchers at Johns Hopkins have advanced this technology, making significant improvements to traditional voice tuning software's previous capabilities. More than just Auto-Tune on steroids, this innovation—called Diff-Pitcher—seamlessly corrects out-of-tune singing while maintaining the original vocal timbre and naturalness, expanding its possible applications beyond entertainment and the music industry and into the health care setting.

"Diff-Pitcher is a generative deep neural network that takes pitch-correction technology to a new level," said team member Jiarui Hai, a PhD student in the Whiting School of Engineering's Department of Electrical and Computer Engineering. "Its precision and control can not only help musical artists and producers but also open new possibilities in areas such as voice rehabilitation and assistive technologies."

Hai and lead researcher Mounya Elhilali, a professor in electrical and computer engineering, presented their new technology at the 2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics last fall.

In music, the ability of pitch-correcting software to adjust the sound of a voice or instrument to the desired pitch is crucial. Diff-Pitcher employs advanced algorithms to enhance the naturalness and quality of pitch correction, surpassing previous tools. Unlike older software that trained on pairs of corrected and original vocals, Diff-Pitcher analyzes the visual representation (spectrogram) of original vocals requiring correction. It identifies target notes, predicts needed adjustments, and transforms the corrected spectrogram into audio, achieving pitch correction in two easy steps.

According to Hai, Diff-Pitcher is notable not only for its ability to create authentic and controlled voices but also for the method used to achieve it.

"[The results sound] really natural, and unlike in older ways of fixing pitch, we can still regulate how high or low the voice goes," Hai said.

The researchers plan to continue to refine Diff-Pitcher to make voices sound even more natural and to extend the tool's capabilities beyond enhancing music production. They believe Dif-Pitcher has a wide range of potential applications, including aiding those undergoing voice rehabilitation, particularly those with vocal impediments.

"The technology could revolutionize a spectrum of speech-related disabilities," Hai said, "offering valuable support for post-laryngectomy patients and contributing to the voice rehabilitation of stroke victims."

The source of this news is from Johns Hopkins University

Popular in Research

1

Apr 6, 2024

Conspiracy theory runs wild linking New York City’s 4.8-magnitude earthquake to date of solar eclipse

2

Apr 4, 2024

Five Sydney researchers honoured by Australian Academy of Science

3

Apr 9, 2024

The rise of Dawn

4

Apr 1, 2024

VIU researcher exploring tourism resilience in rural BC communities

5

Apr 4, 2024

Conservative activist Charlie Kirk helped oust Ronna McDaniel at the RNC. Now the knives are out for him.

Fight or flight: Fearful Trump critics weigh the risk of retribution if he's re-elected

1 day ago

MSN

1 day ago

Silence broken on gender pay gaps but we must hold organisations to account

1 day ago

Nasdaq Futures Up 2% as Nvidia Powers Global Rally: Markets Wrap

Apr 8, 2024

Investigating and preserving Quechua

19 hours ago

Biden visits his Pennsylvania hometown to call for more taxes on the rich and cast Trump as elitist

19 hours ago