2022
Performance for Emotion Artificial Intelligence and generative poetry
Walter Bruno Brix | Masks
Claudia Robles-Angel | Composition, artistic direction, and Live Electronics
World premiere in 2022 at the International Conference „Improvisation, Ecology and Digital
Technology “. NRW Forum Düsseldorf.
Inspired by the German word Einfühlung (empathy), Claudia Robles-Angel explores in this new
work the idea of empathic machines with the question: can a computer react empathetically with the user?
The performance explores therefore the combination of biomedical signals, particularly in this occasion, GSR (Galvanic Skin Resistance) and machine learning techniques used in Emotion Artificial Intelligence to create an empathic sound environment.
In this work, the performer’s emotions are constantly monitored by a Facial Expression Recognizer system (FER), capturing pictures of the performer in real-time and evaluating them through a Deep Neural Network. After recognizing the emotion, the system generates random keywords related to the analysed emotion, which is used thereafter as an input to generate a poem with a Markov Chain Model trained with ca. 3 million lines of poetry. The generated poem is then read aloud by the computer using a text-to-speech algorithm and is thereafter used as sound material by the MAX software, where specially programmed algorithms in MAX produce sound effects, that are transformed in real-time with the values from the humidity of the performer’s skin measured by the GSR interface. Hence, the immersive sound environment is continuously defined by new combinations of sounds and words as an empathic response of the machine to its user/visitor/performer.