Google has taken the wraps off its “human-like” chatbot, “Meena”. Talking of it on its Google AI Blog, Google said Meena was a 2.6 billion parameter end-to-end trained neural conversational model.
This open-source bot can conduct conversations that “are more sensible & specific than existing state-of-the-art chatbots”, said Google. The improvements are because of a new human evaluation metric that Google has proposed for open-domain chatbots called “Sensibleness and Specificity Average” (SSA), which captures basic, but important attributes for human conversation.
Who or what exactly is Google Meena?
Meena is an end-to-end, neural conversational model that learns to respond sensibly to a given conversational context, says Google. The training objective is to minimize perplexity, the uncertainty of predicting the next token (in this case, the next word in a conversation). At its heart lies the Evolved Transformerseq2seq architecture, a Transformer architecture discovered by evolutionary neural architecture search to improve perplexity.
The Meena model has 2.6 billion parameters & is trained on 341 GB of text, filtered from public domain social media conversations. Compared to an existing state-of-the-art generative model, OpenAI GPT-2, Meena has 1.7x greater model capacity and was trained on 8.5x more data.
For each chatbot, Google collected between 1600 and 2400 individual conversation turns through about 100 conversations. Each model response was labeled by crowdworkers to indicate if it was sensible & specific. The sensibleness of a chatbot was the fraction of responses labeled “sensible”, & specificity was the fraction of responses that were marked “specific”. The average of these two was the SSA score.
To read up on Meena, click here.
