In one of my second-year coding classes, I experimented with training a machine-learning model to recognize emotions in my tone of voice. The data that I used for the neutral state mainly consisted of recordings of news broadcasting, my reading letters and words, and other neutral passages like excerpts from non-fiction literature. For the emotional state, I used data from my reading dramatic scripts, personal voice journal entries about emotional experiences, and in-the-moment recordings of me speaking emotionally. I also included a background state where a wide range of sounds in the intended usage environment was used, including sounds of air vents, fans, traffic noise from outside, sounds of people walking by outside, and sounds of silence. I mapped the model's prediction to a point cloud shape that grows when emotions are detected, shrinks when neutral, and is static when only background noise is present; the shape also changes colour (pink/purple=emotional; green=neutral) depending on what is detected. Another iteration of this project allows the user to paint with a computer mouse with the recorded states, a concept that is directly related to painting with emotions. This is a basic model that only considers two states and the results are limited by my training data and are not generally applicable to other people's voices. As I test this model later, the model is not as reliable. It was a fun project to work on. 
Back to Top