Video Premiere: Lorem-‘The Sky would clear what the Man had wrapped to the Link’

Lorem is a music-driven multidisciplinary project by the Italian musician and visual artist Francesco D’Abbraccio. Lorem is an inquiry about human-computer interaction in the age of Artificial Intelligence. During the last two years, D’Abbraccio has built a collaborative platform, involving AI and video artists, IT researchers and music instruments designers such as Mario Klingemann with the aim to produce original audio, visuals, and lyrics. Deep neural networks, hacked hardware, and programming errors are used to drive machines toward aesthetical, autonomous behaviors.

We are presenting a new video produced by AI art pioneer Mario Klingemann for Lorem’s upcoming album Adversarial Feelings.
For Kilingemann who was awarded a Lumen Gold Prize 2018 and is a Google Arts and Culture resident his preferred tools are neural networks, code and algorithms.

Different kinds of Neural Networks (NN) were involved in the generation process of lyrics, MIDI and video fro Adversarial feelings, that consist of 9 AI-generated A/V pieces.
The video was generated by Mario Klingemann with a couple of Generative Adversarial Networks (GANs), while audio and texts by Lorem with a Recurrent Neural Network and Long Short Memory Term (LSMT) Neural Network respectively. The whole process was generated by artificial intelligence and the project, aims is to teach AI to express human and non-human feelings. Francesco told us that the original drums by Lorem were rearranged by the Neural Network that was converted into an audio stream, then Klingemann used them to interfere in the process of two GANs (Generative Adversarial Networks) so the music could interact with the neural image generation process: Klingemann analysed the song and extracted the frequency spectrum and the onsets to use that data to directly influence the output of two GANs that he had trained for portrait generation.

Klingemann, whose artistic interests involve not only artificial intelligence and deep learning, but also glitch art, hooked the frequency data directly up to the learned weights inside the GANs in order to glitch them. When weights are manipulated even small changes can create “wrong” results, in the video is the music itself “the small change” that directly glitches what the model has learned and a similar combination of frequencies will result in similar errors.

 The Sky would clear what the Man had wrapped to the Link is a collaboration by Lorem and Mario Klingemann ahead of Lorem’s debut album out on April 19.


Text by CLOT Magazine (Twitter @clotmagazine)

28 Mar 2019