Digital marketing firm Stone Temple’s latest “test” of digital voice assistants has shown that Google Assistant on phone continues to be the smartest. The company tested Alexa, Cortana, Google Assistant, & Apple Siri via 4,942 queries.
Microsoft Cortana surprisingly outperforms even Google on Google Home, where surprisingly the Google Assistant is not as smart as on mobile. Cortana surpasses Alexa by a significant margin, & Apple’s Siri is not even close.
Google Assistant on mobile attempted to answer almost 80% of the questions. For the ones it attempted, it scored well over 90% in accuracy & completeness. Cortana attempted fewer questions — about 65% — but also achieved over 90% success in answering.
Between last year & 2018, the study found:
- Google Assistant still answers the most questions & has the highest percentage answered fully & correctly.
- Cortana has the second highest percentage of questions answered fully & correctly.
- Alexa has made the biggest year-over-year gains, answering 2.7 times more questions than in last year’s study.
- Every competing personal assistant made significant progress in closing the gap with Google.
In the chart, dark green is qs attempted while the light green bar shows fully correct answers.
A digital personal assistant is a software-based service that resides in the Cloud, designed to help end users complete tasks Online.
Each of the 5 contestants were asked identical set of questions, & noted many different possible categories of answers, including:
- If the assistant answered verbally
- Whether an answer was received from a database (like the Knowledge Graph)
- If an answer was sourced from a 3rd-party source (“According to Wikipedia …”)
- How often the assistant did not understand the query
- When the device tried to respond to the query, but simply got it wrong
Click here to read the survey.
