Picture, if you will, a dog: brown fur, floppy ears, mouth full of drool. Easy, right? Now picture an AI system doing the same thing you just did. Since computer language is essentially ones and zeroes, translating human words into picture data isn’t as easy as you might think. And yet, this is what Microsoft’s newest unnamed project is capable of.
Based on their work with CaptionBotauto-captioning software, and Seeing AI, an app capable of answering details about images, researchers are building a new AI platform capable of taking certain words and converting them into images based on their descriptions.
It’s worth mentioning that it isn’t an automated version of Google Images, as every image made by the AI is completely original. According to Xiaodong He, a principal researcher, and research manager on the project, the AI goes full circle from what they started with CaptionBot and Seeing AI. Whereas the former two programs generated text and details from image data, this drawing AI creates an image from words given to it.
The AI works on a General Adversarial Network (GAN), which prioritizes two learning models: the first being an image generator and the second being a discriminator. The image generator takes text descriptions, fills in the imaginative blanks between them, and converts everything into one final image. The discriminator, on the other hand, uses the same text descriptions to make sure the image generator didn’t mess up when creating the original image.
Considering possible applications, designers and engineers will likely get a kick out of creating early-stage concept sketches using only their words—but what about animators? Xiaodong He hopes that with enough refinement, the technology could be used to create outlandish animated films based on their screenplays alone.
As to be expected, the AI still has some bugs though. Close examination of finished images has revealed inaccurate colored beaks of birds and mutated bananas on fruit stands. That said, seeing as it is still early in development, the progress Microsoft has made to a word-based drawing method looks promising.
Read the entire research paper behind the project and check out the Microsoft blog for more details.
