Meet Frida, the robot who paints AI-driven art in real life

Remember the old days? When AI tools like Dall-E and Stable Diffusion turned your short text prompts into digital art? It’s so 2022…

Meet Frida, an AI-driven robot from Carnegie Mellon University who turns your prompts into physical paintings, complete with bold brushstrokes in a variety of techniques. Perhaps most strikingly, the bot can change course while painting to mimic the iterative nature of art creation.

“It will work with his failures and it will alter his goals,” said Ph.D. Peter Schaldenbrand. student at CMU’s school of computer science and one of the creators of the robot, said in a video describing the project.

Frida aims to explore the intersection of robots and creativity, says the team, which presents its research paper in May at the IEEE International Conference on Robotics and Automation in London. Robots have produced art before and even exhibited it. But Frida is designed expressly to collaborate with humans using the same kind of generative intelligence that drives experimental tools like the AI ​​chatbot ChatGPT.

Frida stands for Framework and Robotics Initiative for Developing Arts, but it also shares a name with famous painter Frida Kahlo. It looks nothing like Kahlo or any other human though. For now, it’s just a robotic arm with a paintbrush attached, a setup that underscores the team’s insistence that Frida is a “robotic painting system, not an artist.”

“Frida doesn’t generate the ideas to communicate,” Schaldenbrand said. This is where the humans come in, communicating Frida’s goals with text entries. They can also show the bot images in a style they like or even flash photos they want to see represented as a painting. Frida suggests the appropriate paint colors on the screen, then the humans mix them in the robot’s palette.

Frida will probably never achieve the fame of her namesake, but there are some impressive skills that set her apart from other art robots, whose input images usually match their end goal. Art creation is a dynamic, ever-changing process, and after planning its trajectory in a simulated environment, Frida uses machine learning to assess and progress in real time. The robot does its planning, as CMU robotics professor James McCann notes in the video, “in a space of senses instead of a space of outputs.”

Frida doesn’t appreciate precision like most robots and could, for example, incorporate an “error” like an errant blobs of paint into her final product. Each painting takes hours to complete and the results are often whimsical and very colorful.

“There’s this painting of a frog ballerina which I think turned out really well,” Schaldenbrand said in a statement. “It’s really silly and fun, and I think the surprise of what Frida generated based on my input was really fun to see.”

To create their AI datasets, the team fed their models with news headlines and further trained them on images and text representative of various cultures to avoid an American or Western bias.

The role of AI in creating visual art, composing songs, and even writing poems and movie scripts is generating excitement, but also raising ethical and human rights concerns. author among artists and even lawyers. AI art is not created in a vacuum. It works by absorbing and reconstructing existing art created by humans. As machine-made art improves, will these humans – real graphic designers, illustrators, composers and photographers – find themselves unemployed?

Some artists I’ve spoken with describe feeling nervous about the complex questions raised by AI art. Others – like Steve Coulson, a comic book enthusiast who wrote a series of comics drawn entirely by Midjourney – embrace what they see as inevitable change. The comic artists Coulson has long loved “have an eye for dramatic composition and dynamic storytelling that I highly doubt machine learning can match,” Coulson says. “But as a visualization tool for non-artists like me, it’s a lot of fun.”

Frida’s inventors share a similar perspective.

The weaponized bot, the research paper states, “is a robotic initiative aimed at promoting human creativity, rather than replacing it, by providing humans with intuitive ways to express their ideas using natural language or text examples. ‘pictures’.

Editors’ note: CNET uses an AI engine to create personal finance explanations that are edited and verified by our editors. To learn more, see this post.