Artificial Intelligence (AI) machines created by IT giant Microsoft & Chinese tech giant Alibaba have passed humans in a human language reading test known as the ‘Stanford Question Answering Dataset’ (SQuAD).
This major AI milestone took place earlier this month. AI by these 2 IT cos beat the human score for Exact Match (providing exact answers to questions).
Microsoft made the announcement on its official AI blog. It said:
A team at Microsoft Research Asia reached the human parity milestone using the Stanford Question Answering Dataset, known among researchers as SQuAD. It’s a machine reading comprehension dataset that is made up of questions about a set of Wikipedia articles.
According to the SQuAD leaderboard, on Jan. 3, Microsoft submitted a model that reached the score of 82.650 on the exact match portion. The human performance on the same set of questions and answers is 82.304. On Jan. 5, researchers with the Chinese e-commerce company Alibaba submitted a score of 82.440, also about the same as a human.
The two companies are currently tied for first place on the SQuAD “leaderboard,” which lists the results of research organizations’ efforts.
The development is a “major milestone” in the push to have search engines such as Bing & intelligent assistants such as Cortana interact with people & provide information in more natural ways, much like people communicate with each other.
The post said Microsoft was already applying earlier versions of the models that were submitted for the SQuAD dataset leaderboard in its Bing search engine, & the company was working on applying it to more complex problems.
For example, Microsoft is working on ways that a computer can answer not just an original question but also a follow-up. For example, let’s say you asked a system, “What year was the prime minister of Germany born?” You might want it to also understand you were still talking about the same thing when you asked the follow-up question, “What city was she born in?”
It’s also looking at ways that computers can generate natural answers when that requires information from several sentences. For example, if the computer is asked, “Is John Smith a US citizen?,” that information may be based on a paragraph such as, “John Smith was born in Hawaii. That state is in the U.S.”
Ming Zhou, Assistant Managing Director of Microsoft Research Asia, said the SQuAD dataset results were an important milestone, but he also added that, overall, people are still much better than machines at comprehending the complexity & nuance of language.
