The Acoustics of Emotion: Creation and Characterization of an Emotional Speech Database
Open Access
- Author:
- Moriarty, Peter Mcphillps
- Graduate Program:
- Acoustics
- Degree:
- Master of Science
- Document Type:
- Master Thesis
- Date of Defense:
- November 06, 2017
- Committee Members:
- Michelle Celine Vigeant, Thesis Advisor/Co-Advisor
Pamela Marie Cole, Committee Member
Karl Martin Reichard, Committee Member - Keywords:
- affective prosody
emotion
speech
acoustics
voice
vocal tract
emotion recognition
voice production
communication
affect
prosody - Abstract:
- Paralinguistic features of speech communicate emotion in the human voice. In addition to semantic content, speakers imbue their messages with prosodic features comprised of acoustic variations that listeners decode to extract meaning. Psychological science refers to these acoustic variations as affective prosody. This process of encoding and decoding emotion remains a phenomenon that has yet to be acoustically operationalized. Studies aimed at sifting and searching for the salience in emotional speech are often limited to conducting new analyses on material generated by other researchers. This project presented an opportunity for analyzing the communication of emotion on a corpus of naturalistic emotional speech generated in collaboration with Penn State’s Psychology Department. To this end, fifty-five participants were recorded speaking the same semantic content in angry, happy, and sad expressive voicings in addition to a neutral tone. Classic parameters were extracted including pitch, loudness, timing, as well as other low-level descriptors (LLDs). The LLDs were compared with published evidence and theory. In general, results were congruent with previous studies for portrayals of more highly aroused emotions like anger and happiness, but less so for sadness. It was determined that a significant portion of the deviations from the scientific consensus could be explained by baseline definitions alone, i.e. whether deviations referenced neutral or emotional LLD values. A listening study was subsequently conducted in an effort to qualify and contrast the objectively determined effects with perceptual input. Only three of the fifty-five speakers were sampled due to practical concerns for testing time. The study tested whether the sampled recordings reflected naturally recognizable emotion, and the perceived intensity of these emotions. Listeners were able to discriminate the intended emotion of the speaker with success rates in excess of 87%. Perceptual intensity ratings revealed that some of the prototypical acoustical cues did not significantly correlate with the perception of emotional intensity. Results from both rounds of analysis indicate that a wealth of emotionally salient acoustical information has yet to be fully characterized.