The project explores the possibilities of Artificial Intelligence in a bio-semiotics context. The AI is looking for some patterns in the structure of birds’ sounds to build a mathematical model of the universal grammar of Bird Language. In the first stage of the project, we trained a neural network on sounds of nightingales and created a situation of communication between non-human agents: birds and AI. This is a metaphor for communication between nature and technology where a human being is not necessary. The second stage of the project entails the creation of an AI translator from bird language to human language. The Deep Learning is identifying patterns in Big Data of birds’ signs and grouping similar signs into clusters. We are interpreting these clusters to build the AI translator for interspecies communication.
What did you create?
We created a universal tool to understand the structure of bird language. We can translate from human language to bird language and vice versa, in other words we have translator and text-to-speech for bird language.
Why did you make it?
The motivation was the paradigm of non-anthropocentric world, where we can understand non-human others, like animals and birds. The inspiration came from Jakob Johann von Uexküll's concept of Umwelt - the experiential world that a being inhabits, its perceptual surrounds. I'm inspired by the idea, that Artificial Intelligence as a un-human agent, can help us to understand birds, also un-human agents. In this case AI is not only a mediator or interface between human beings and birds but rather an organ or full partner, semiotically active. It helps to understand bird’s subjectivity through the language.
How did you make it?
We started to work with the Great Tit, one of the most widespread species from Europe to Asia. The first machine learning approach we used, was XGBoost - boosted decision tree algorithms. While it classified bird signs in two groups: calls and songs with an accuracy of 83%, it was not a universal tool to understand the structure of the language. Now we are developing the second machine learning approach based on auto encoders. First, we take the waveforms from the audio of the bird language and process it with an auto-encoder, in order to extract the shape of the bird phonemes. Once we have these shapes, we then take a second auto-encoder, and feed it with the shapes of the phonemes, which allows us to see clusters. These clusters reveal the language structure and let us deconstruct the bird language into a series of phonemes, which we are using to build an AI-translator for interspecies communication.
Your entry’s specification
1) Video-documentation (duration 2:40 sec) 2) Print(s) (visual materials - size is site specific) 3) Application for generative sound + visualization (self-developed software, self-designed and trained neural network) Detailes: 1) TV-screen + player + headphones\speakers (depends on exhibition space) 2) Print on wallpaper or foam board 3) Mac Mini with self-developed software + TV-screen (for visualization) + headphones\speakers (depends on exhibition space)