“Sounding out” and “Seams”, are two installations that are part of the one artistic project and share their content using different perspectives. On the one hand “Sounding out” has as its core the “energy band analyser”; it is a live interactive installation in which the audio signal from the participant is captured by a microphone and analysed in real time. On the other hand, “Seams” is not interactive in the sense that no live information is produced or detected (except for the movement of the head of the participant wearing the goggles) but it is an immersive art experience that shows the contents of my research; this installation is a way for people to visualise the patterns of emotions in sound, a sort of phenomenological approach to my research content and findings. This second installation is meant to address the topic from a different perspective in order to give people a figurative as well as practical overview of this study.
In this artwork, I created an inclusive framework for real-time recognition of emotions from acoustic properties of speech not using content information. I then linked these properties to a visual output to arrange an interface of the emotions.
ABSTRACT & INTRODUCTION to the art project
sounding out PICTURES
FLORENCE JUNE 2018
sounding out VIDEO MATERIALS
FLORENCE JUNE 2018
RECORDINGS OF LIVE PARTICIPATION
“Seams” is an immersive audio-video representation of the processes of analysis and detection of emotions used in “sounding out” and in the research carried out in my studio using the “energy band analyser”.
THE BEGINNING - PRIVATE TECHNICAL TESTS
FIRST ATTEMPT - PREMIER - LIBRARY CONSERVATORY OF CESENA APRIL 2018
seams VIDEO MATERIALS
2D VERSION OF WHAT PARTICIPANTS SEE IN THE HEAD MOUNTED DISPLAY WITH FIXED STEREO VERSION OF SOUNDTRACK -
(NO SONIFICATION AND NO SPATIALIZATION OF SOUND WHICH ARE AVAILABLE ONLY IN LIVE EXHIBITION)
The videos below are those shown inside the televisions with the different sonification of the audio.
During the 360 degree immersion (using 3D goggles) the participant, turning the head on the left, hears sad voices and sonification of sadness, facing the centre, s/he hears happy voices and sonification of happiness and turning the head to her/his right, s/he hears angry voices and sonification of anger.
The interaction is only possible during the exhibition through the use of a max/MSP patch designed for the intent.
Below you can press play and stop on each video alternately to try to create the feeling of when you are turning your head in a direction (left/center/right) to understand the change in the sound environment.
left - sadness
centre - happiness
right - anger
seams Audio soundtrack
Below you can listen to each track alternately (the audio is the same as the above 3 videos) to understand the differences between the audio sonification of the 3 emotions and to focus only on sound.
Sonification of Happiness
Sonification of Anger
Sonification of Sadness