A new Facebook AI research project can make outfits more fashionable by suggesting subtle style changes that will have a big impact. The system, called Fashion++, might recommend tucking in a shirt, ditching an accessory, or swapping a skirt for jeans — all easy adjustments that wouldn’t force you to drop big bucks on a new purchase.
Why would anyone listen to fashion advice from computer code, which can’t even put on pants? Expressing ourselves through what we wear is part of what makes us human, after all. Nevertheless, AI scientists are increasingly finding that technology can amplify our creativity in unexpected ways. Fashion++ is purely a research project, with no immediate plans for related products or features, but it’s pointing the way forward.
“A tool like Fashion++ could one day allow a user to tweak his or her outfit based on personalized style advice before stepping out the door, or allow a designer to envision novel ways to enhance familiar looks,” says Kristen Grauman, the research scientist who helped create the tool.
“While we think such potential applications are exciting, we were also drawn to take on this project because it presents interesting technical problems for AI — such as how to quantify visual compatibility or how to optimize for one property (style) without altering others.”
Like many AI systems, Fashion++ learns from the examples it is given. So its recommendations will vary, depending on whether it has learned from images of 1920s flapper dresses or 1970s punk rock street styles. The system analyzes those images and applies its learnings to decide whether your jeans-and-polo-shirt ensemble is perfect as is or would benefit from a statement necklace.
The Fashion++ team is particularly proud that their system is designed to help with everyday style dilemmas. (The “++” in the project’s name is a coding reference, meaning “plus one,” or an incremental improvement.) Most previous AI research work on fashion has focused on making photographed garments searchable or recommending entirely new outfits. By focusing on more subtle edits, Fashion++ could lead to tools with more real-world utility.
Building a fashion-forward AI system
Computer systems have gotten very good at recognizing whether a picture shows a dog or a cat, but it is much more challenging to teach them to spot when a sweater clashes with a pair of pants. For Fashion++ to work, it must be able to take a photo and then differentiate the individual items of clothing and accessories, as well as how they’re being worn. (For example, are the sleeves rolled up or down?) It also has to pick up on the essential aesthetic shown in the sample images it is trained on. Do they show a preference for minimalist, understated, office-ready outfits? Or something a little more funky?
“Automatically suggesting minimal edits is challenging because the difference between outfits is subtle — often just a few pixels, which is difficult to capture with traditional computer vision models,” says Grauman. “Furthermore, ideal training data would consist of curated pairs of better and worse versions of the same outfit, but that would be much too time-intensive to gather manually. Our approach takes steps to address both issues.”
To round out Fashion++’s sense of style, the system also creates its own examples of “unfashionable” outfits. Fashion++ takes the on-trend images it has been given as samples and alters them by swapping out garments, changing patterns, and adjusting the fit. It then uses these artificially generated looks to help teach itself what qualifies as a fashion “don’t.”
After analyzing the examples, Fashion++ is ready to help dress to impress. Give it an image of what someone is wearing and it can respond with a modified version that more closely resembles the examples it was trained on.
To gauge whether these changes were successful, the research team asked human evaluators to rate Fashion++’s advice. Overall, participants not only preferred the AI’s suggestions, but also judged those upgraded looks as being similar to their examples used to train it.
“A desirable feature of Fashion++ is how it incrementally improves an outfit’s fashionability, thereby providing a spectrum of edits,” says Kimberly Hsiao, an AI researcher who worked on the project. “Users can choose a preferred endpoint, starting from the least changed and moving towards the most fashionable.”
Because Fashion++ learns from whatever examples its given, it might be possible to have it account for local or regional style trends. If you trained it only with photos of, say, Harajuku styles, then Fashion++ would try to help you adjust your look accordingly. And if you decided that Fashion++ was falling behind the times, you could feed it new images of the latest spring runway looks.
Computer-enhanced creativity, from food to filmmaking to music
Creativity and artificial intelligence research are connecting in other ways as well. Facebook AI researchers have built an AI-powered system that can create a recipe just by analyzing an image of a particular dish. This “inverse cooking” research project shows how AI might be helpful even with something as profoundly human and subjective as baking the perfect chocolate chip cookie.
In films like Jérôme Blanquet’s VR drama Alteration, AI is not just a topic to explore but also an active contributor. The movie uses an algorithm developed by Facebook AI researchers to perform “style transfer” on particular scenes — taking one visual look and using AI to apply it to something else altogether.
And with music, researchers at Facebook have created a way for AI to augment a simple melody and turn it into a richly orchestrated composition. This “universal music translation network” can transform an acoustic guitar solo into a string quartet piece or make it fit the compositional style of Bach or Beethoven. Just as Fashion++ learns from photo examples, the music translation tool uses recordings to figure out the essential characteristics of different composers or musical genres. In the future, a professional composer might use this sort of tool to try out new arrangements for a song, or someone could just use it to craft a uniquely silly version of “Happy Birthday” for a family member.
In each of these research projects, the central creative work — deciding what’s stylish or picking a meal to cook or crafting a new tune — is done by a living, breathing person, not a machine. The AI systems can provide suggestions for how to implement an idea or help brainstorm some ways to adjust it. But they all build on human creative expertise rather than trying to replace it. An AI-powered map might tell you the best way to get somewhere, after all, but it needs a person to decide where to go.
Working toward assistive technologies that offer creative help
AI researchers hope projects like these will one day help make digital assistants vastly more useful and enjoyable. Whereas today you can ask your phone to give the weather forecast or add a calendar event, in the not-too-distant future an AI-powered assistant might help you pick your weekend outfit, round up the ingredients needed for a pancake breakfast, or suggest the perfect visual effect to liven up a video chat.
Fashion++ is just one step toward building these futuristic tools, but it shows how AI systems actually learn about creative tasks. Since Fashion++’s definition of good style is refined by the fashionable images the system it learns, there’s no need to ruminate over AI taste levels, the subjective nature of fashion, or other philosophical conundrums. As it offers style advice, Fashion++ wears its influences on its figurative sleeve — even as it suggests you roll up yours.