KI-Musik

Just do it

According to Co-Founder Felipe Sanchez Luna, kling klang klong doesn’t “use technologies like AI as an end in itself, but conjures up a certain emotional experience. This takes a lot of experimentation to see what’s actually possible with this tech.” The Berlin-based sound designer, artist and programmer is a big fan of so-called rapid prototyping.

With this technique, developers don’t spend a lot of time on planning and theory, but dive right into building, testing, and trying new approaches. The first models of Kling Klang Klong’s clever instruments were actually crafted from cardboard. “We are makers at heart, so we started out trying a lot of different approaches, which really sped up the whole development process. And occasionally even delivered results we never would have expected.”

Let the music take control

A simple rule of thumb: the easier it seems, the more there is to it. Intuitive use was the key for the Futurium exhibits to make sure visitors weren’t held back by any fear of technology. You don’t need any computer expertise or tech know-how to get started. Nor is there the need for years of violin practice or perfect pitch because programmed, data-trained AI takes the place of practice and execution.

Conduct, compose, create

At the Futurium, visitors can ‘conduct’ and shape the sound of a whole band with a grand gesture or a single flick of the wrist. Their arm movements influence the music’s richness, intensity, and effects. As a positive side effect, every interaction trains the AI to get even better. If you prefer to get creative, feel free to compose your own song at the next station. All you need to do is reassemble musical ‘building blocks’ and then use the controls to tweak the track’s length and style. Once you push the button, the AI will turn your composition into a real song.

Finally, the improvisation station really underscores how creative human-machine interactions could work. Musicologist Fernando Knof explains that kling klang klong “has first trained the AI with a huge amount of data of professional pianists to help it understand how people actually improvise.” Once you touch the station’s eight keys, you’re basically starting a duet with the AI. The installation’s neural network analyses your sequence of notes – and answers with a rich-sounding variation using all 88 piano keys. What’s even better, it will give you a download code for your spontaneous masterpiece.

Complexity behind the scenes

“We wanted to make sure that absolutely anyone could direct a band like a real conductor, or play like a pro pianist or compose chart-friendly music. So, we put a lot of thought into creating especially easy, intuitive user interfaces and controls,” adds Sanchez Luna. “But the actual development behind it was really quite complex. While I served as the creative director and developer on this project, we also involved experts from a lot of very different disciplines: design, user experience, programming, visualisation, composition, and so forth. Together, we worked out the best possible solution.”

Does technology touch a chord?

So will we all become lazy button pushers who get their culture on tap from the AI lab? Fernando and Felipe certainly don’t think so. “I’m definitely not afraid of artificial intelligence taking my job away,” Fernando grins. “To be honest, a lot of music creation still boils down to basic craft and some fairly tedious tasks. Any idea you have still needs to be refined and realised. Sometimes, you might really not want to play every instrument yourself. People will stay in charge of the creative part and the AI will just execute their ideas.”

His partner adds that he’s really intrigued by “the way this process helps me discover brand new ways to understand music and see new connections. AI can make music more interesting, diverse, and multi-faceted. We’re witnessing the creation of an entirely new instrument.”

Emerging tech in its infancy

So, what happens when you give Artificial Intelligence free rein in the arts? Spawn, the official ‘offspring’ of the avant-garde musicians Holly Herndon and Mat Dryhurst, might offer us a glimpse of the future. The couple’s permanent AI partner continues to evolve and find its very own, unconventional voice. “At some point, she actually started beatboxing like Timbaland,” Holly enthuses. “She definitely didn’t get that from me. Kids do the darnedest things …”