- We’ve built a first-of-its-kind method for using AI to automatically animate children’s drawings of people.
- Parents can upload their children’s drawings to our prototype system and watch them become moving characters that dance, skip and jump.
Update on April 13, 2023 at 9:00 AM PT
Today, Mark Zuckerberg announced that we’re open-sourcing the code for our artificial intelligence research demo that lets you animate your hand-drawn artwork. In addition to open-sourcing the code, we’re also releasing a dataset of nearly 180,000 drawings to help researchers and creators continue innovating in this space. This new dataset will help researchers build tools to more accurately analyze drawings and unlock new forms of storytelling.
Learn more about the demo and new dataset.
Originally published on December 16, 2021 at 8:00 AM PT
Children draw unique and imaginative characters that require us to think a little differently to recognize the people and things in their pictures. While it can be fairly simple for a parent or teacher to see what a child’s drawing is meant to show, AI struggles with this task because kids’ drawings are often constructed in abstract ways.
Our researchers have developed the first-of-its-kind method for using AI to automatically animate children’s hand-drawn figures of people and humanlike characters (i.e., a character with two arms, two legs, a head, etc.). By uploading drawings to our prototype system, parents and children can watch their drawings become moving characters that dance, skip and jump. And, they can even download a video of their animated drawings to share with friends and family. Parents can also opt in to have their child’s drawing used to continue to teach the AI model.
While many AI tools can handle realistic images of humans, children’s drawings add a level of variety and unpredictability that makes identifying what’s being portrayed much more complex. Our goal was to build an AI system that can identify and automatically animate the humanlike figures in children’s drawings with a high success rate and without any human guidance.
We accomplished this by developing a system for teaching AI to identify the humanlike figure in a child’s drawing, which can be tricky when other objects or figures on the page have similar features, colors or dimensions. After that, we isolate the humanlike figure without including anything else on the page so that the figure that’s animated is as close to what the child intended as possible. Then, we pinpoint the joints of the figure so that the animation can move about in an appealing way. Once all those steps have been completed, it’s ready to be animated. Here’s what our process looks like:
By teaching AI to work effectively with children’s drawings, we hope this project will move us closer to building AI that can understand the world from a human point of view. We invite you to test out the systems’ animation capabilities by uploading your children’s drawings to our prototype. In the coming year, we also hope to release the data set and share more details on our research.
Learn more about this research on our AI blog.