The future is now! DALL·E by OpenAI, is a neural network that creates images from text. That’s right, you simply input a sentence, a sentence & a reference image, or a partial image as a prompt & DALL·E will render your input into art.
Launched a few months ago, DALL·E is work in progress. Its a 12-billion parameter version of GPT-3 (an autoregressive language model with 175 billion parameters), which is trained to generate images from text descriptions, using a dataset of text–image pairs.
The name DALL·E is a mix moniker, after the artist Salvador Dalí & Pixar’s WALL·E, but I don’t know how happy Dali would be with this tool that potentially retains the power to eventually put human artists out of work, especially graphic artists. This AI model is so much better than anything previously seen; it simply blows you away.
See the example below: Given the ‘text input’: an armchair in the shape of an avocado, DALL·E rendered these images:
Overall, DALL·E is pretty impressive. You may try it out for yourself by changing the text input here. For instance, if you change the text to, “an armchair imitating a peacock” the output below, is very interesting considering that it’s computer generated.
The transformer language model receives text & an image as a single stream of data containing up to 1280 tokens, & is trained using maximum likelihood to generate all of the tokens. DALL·E can generate an original image from text inputs only, but it can also regenerate any rectangular region of an image input that extends to the bottom-right corner, in a way that is consistent with the text & image prompt.
As for a limited set of objects DALL·E can position & construct them in an image in the correct orientation as long as the text input is phrased appropriately.
However, when it comes to multiple object orientation & positioning within the image, the model gets confused when the number of items increases & the input text sentence gets longer. No doubt that these issues will be overcome with time, but for now, DALL·E can compete with any human artist, given the same text input.
Image credit: OpenAI
