Through this interactive work, my earliest artwork with AI (made in 2015)- i began pondering the question of coming together of human and machine creative expression. Tandem is an interactive drawing tool, with an interface like any simple drawing application, however it's one with an AI capable of 'imagination'- which can turn your doodles into something unexpected and invite you to have a visual creative dialogue with itself. The audience begins the interaction by doodling or drawing something. The audience can then select a 'personality' type for the AI between options of 'Happy, Dreamy, Dark, Angry' and a non-personality option of 'Pencil'. These give different visual stylistic outputs. The other choice to make is between 'Abstract' or 'Figurative' outputs- this results in the AI output to vary from abstract pattern outputs to more figurative identifiable elements based on the categories of AI's training. The system works using the Deepdream AI technique primarily, which relies on AI's training on the Imagenet dataset comprising ~14 million images in 1000 categories. The distinction between Abstract and Figurative happens by determining the layer the Deepdream algorithm uses to be the final layer in each step of its creation. Abstract patterns are achieved by chosing earlier layers in the Deepdream algorithm and figurative ones achieved by letting the Deepdream algorithm run till the final layers. The 'personality' is applied with the Neural Style Transfer Algorithm run on the Deepdream output using pre-curated images for each category of the personality. For the pencil stylistic category- i use a custom written Open CV script to transform the Deepdream output to a pencil sketch style artwork. The Deepdream algorithm comprising of Convolutional Neural Networks was a major breakthrough in deep learning, exhibiting the machine's capability to 'imagine' visual material. Through Tandem, the input for machine imagination is human doodles and Tandem explores this interaction from the capacity of forming a human-machine creativity continuum.
When the human starts to doodle something, the AI ‘recognizes’ it as objects it knows and the biases inherent in it. It then imagines on top of the human drawing with each iteration of its creation. The human could choose to continue this cycle of imaginations by editing the machine's output or re-feeding it its own output. Someone interacting with this work experiences the eerie continuities and discontinuities between the human and machine imagination where our own human visual categories (which the machine has been trained on) are filtered and estranged through the (sometimes alien) associations the machine makes. Mindless doodles often turn into definitive objects which we might not have imagined ourselves, however once we start to see those emerge from the machine's imagination- we start to associate with those objects in the context of our drawings too. It becomes a playful mechanism to peep into the inner functioning of the AI's algorithm or 'imagination' too. This was a critical part of the thinking behind the work, especially given how new the idea of AI generating completely new visuals was back when Deepdream released- making an interactive system to explore its workings would invite people to form their own insights and relations with the machine rather than being prescribed one. Such active engagement with AI systems is crucial for taking ownership of the narrative and influence of these technologies, in debunking their hype and confusions- instead creating more tangible ways to critique it.
The output raises some pertinent points to ponder. We are left to wonder whose was that final drawing? The human interlocutor’s? The AI’s? Or does it really belong to the deep formal patterns in the human subconscious that the machine learns through looking at the way we form associations between the verbal and the visual?
Tandem has been exhibited internationally on various occasions, including at the early AI Art exhibition (alt-AI at the School for Poetic Computation in 2016, about which you can learn more in this video of interviews https://vimeo.com/189974949). This video piece records the functioning cycles of the system- from human doodle or sketch to selecting parameters of input and the AI made output in steps of emergence.
