When AI art generators like DALL-E-2 and Stable Diffusion were released, they created lots of discussion and discourse about a new era of art. But, machine learning and AI have been used to help people be more creative for years. According to the World Economic Forum, AI is at its best when it supports human creativity and researchers at the Gottlieb Duttweiler Institute found that AI can catalyze human creativity in three ways:
In the art world, artists are discovering ways that AI can boost their creativity. So, we thought it would be worth exploring some of the uses for AI models in art, from artists who sculpt with data to robots painting alongside humans and archaeologists using AI to piece together history.
While there is a lot of concern about the role of AI in creating art, a few artists have been incorporating AI learning and tools into their process for years. Artist Refik Anadol uses AI, algorithms, and data to drive his work as a new media creative and painter. If you were watching the 2023 Grammy Awards and noticed the stunning art in the background of the ceremony, you were watching Anadol’s work in action. The paintings were generated using data collected by AI. The pieces were part of his “Machine Hallucinations - Space and Nature” collection, which uses a data set of 300 million nature images that Anadol and his studio processed through software they created to generate art. Another famous installation by Anadol is at the MoMa. He trained AI with all the pieces and installations at the famous art museum and asked it to reflect on what it had learned. The resulting installation is a free-moving, ever-changing installation that creates and imagines art. Every little piece of his work represents a piece of data put through a neural network by Andadol and his team - forming what he calls “data sculptures.”
Anadol is part of a community of AI artists who are incorporating machine learning and AI as collaborative tools to create art. But, researchers at Carnegie Mellon are taking the idea of collaborating with AI a step further by working on a robotic-art making assistant called FRIDA. While image generators like DALL-E-2 create an image on a screen, FRIDA is a robot that uses generative AI to actually paint on canvas. Carnegie Mellon University’s newest artist-in-residence is designed to collaborate with humans to create art, responding to prompts and painting pieces. FRIDA uses AI to learn to use the paintbrush, then to understand the input, and then to develop a plan to achieve the user’s request. The whole process takes hours, and ultimately, FRIDA is designed to be a creative assistant for artists.
The team at Carnegie Mellon has explored a variety of inputs, like text descriptions, photographs, and other works of art. The team even played “Dancing Queen” by ABBA and asked FRIDA to paint the song. One of the interesting things about FRIDA is that it evaluates its work as it paints and is able to riff off of its mistakes. According to one of the researchers, Jim McCann, “FRIDA is a project exploring the intersection of human and robotic creativity.” Researchers on the project also addressed some of the limitations in current large vision-language models. In order to keep FRIDA from developing bias and stay up to date on current events, students working on the project fed the models various headlines from news articles. They also trained the AI with images and text from diverse cultures. The data contributions came from countries like China, Japan, Korea, Mexico, Nigeria, Norway, and Vietnam. While many AI art generators take seconds, FRIDA’s masterpieces take hours to create.
While some artists are using AI to create new works of art, some are using AI to piece ancient art back together. Archaeologists are using AI to piece together frescos and artifacts from Pompeii. The most time-consuming task for archaeologists is trying to piece together artifacts and frescos that have been shattered. In Pompeii, archaeologists have adopted a new approach to puzzling together these valuable pieces of history. They are using an AI-powered robot that determines how frescos fit together, then uses its arms to reassemble them. The project, dubbed RePair, which stands for Reconstructing the Past: Artificial Intelligence and Robotics meet Cultural Heritage, uses AI software that can analyze the patterns of fragments found at Pompeii and help archaeologists speed up the process of reconstructing artifacts. The project’s leaders are currently training the AI software to tackle a pair of frescoes from Pompeii that are about 2,000 years old. The frescoes were shattered into tens of thousands of fragments, and since it would be essentially impossible for humans to piece them back together, the pieces have been sitting untouched in a warehouse outside of Pompeii. The project leaders are training the AI with images of the fragments and hope that it will piece together the puzzle and assist archaeologists in reconstructing the famous frescos.
Artists like Anadol, researchers, and archaeologists alike are using AI as a personal assistant to supercharge their creativity. According to experts like the World Economic Forum, the future of responsible AI is to build tools that support humans and make them more effective, creative, and efficient. They also caution that AI is not a tool to use complacently and that the goal of AI solutions should be to support human creativity, not replace it with machine-learning-driven creativity. There are clearly many exciting creative use-cases for AI and we'll be watching closely as this field evolves.