Quasimondo 2020
"Die Jugend spaltet das Atom", 2020, Generative Music Video
StyleGAN2, Custom Python Code
Song: "Das Atom" by Kraftamt, 2013
I love dancing—not professionally—but whenever I do it I try to translate my interpretation of the music’s momentum and gestures into the multidimensional space of my body. Working with the latent spaces of neural networks, particularly GANs, I found a parallel process: translating sound into visuals. Dancing makes music visible through movement; my art tries to do the same through AI, letting the music shape images.
"Das Atom" by Kraftamt is part of my ongoing practice of creating generative music videos. In 2020 I was exploring the possibilities of the StyleGAN2 model to create artificial human-looking "puppets" that allowed me to control their appearances and facial expressions via latent direction vectors that I experimentally discovered. These direction vectors can be seen as "strings" that I can pull or push and which in turn change a selected aspect of the figure: from simple actions like rotating or tilting the head, smiling or frowning, opening or closing the mouth to complex ones like altering the age or apparent ethnicity or gender.
The video was generated in real-time without any manual edits by a custom program I wrote. Via code I detect features of the song, like beats or frequencies and connect them with the latent direction vectors that control the face. My work in this case is more that of a choreographer than a videographer when I iteratively adjust the connections and their parameters until I have a system that I feel captures the energy and mood of the song.
In "Die Jugend spaltet das Atom" in addition to the sound controlled motion of the faces I let the age vector cycle back and forth between "young" and "old".
Thank you very much to Kraftamt for sharing their songs under the creative commons licence and for their special permission to mint this work as an NFT! You can find more songs by the artist here: https://ccmixter.org/people/karstenholymoly