What you need to know
- The Brain Team at Google Research outlined a new type of chatbot in a paper published earlier this week.
- Named Meena, this virtual talker was explicitly designed to address chatbots' seeming inability to understand the broader context of conversations.
- It can also be funny, sometimes.
Meena, what Google Brain's research team so callously described as a "multi-turn open-domain chatbot" — she's got feelings, people! — could just be the next generation of chatbot. And perhaps your next best friend.
The paper describes the flaws with the current generation of chatbots as follows:
they often respond to open-ended input in ways that do not make sense, or with replies that are vague and generic.
Trained on 40 billion words mined from the valleys of Twitter and Facebook, Meena was explicitly designed to be contextually aware of not just some specific customer support tasks but rather the ongoing conversation as a whole and the world at large. To ensure Meena was smarter, and funnier, than those 'other chatbots,' Google's team coined two new metrics for testing itsperformance.
These being perplexity, which automatically generates data on the ability of a chatbot to understand a conversation, and Sensibleness and Specificity Average (SSA), which relies on humans to give their input on how well the AI performs in a conversation.
If anything seems off — confusing, illogical, out of context, or factually wrong — then it should be labeled as, "does not make sense".
However, being sensible is not enough. A generic response (e.g., I don't know) can be sensible, but it is also boring and unspecific.
For example, if A says, "I love tennis," and B responds, "That's nice," then the utterance should be marked, "not specific". That reply could be used in dozens of different contexts. However, if B responds, "Me too, I can't get enough of Roger Federer!" then it is marked as "specific", since it relates closely to what is being discussed.
In other news, I'm more boring than a chatbot, because all my conversations end with a "That's nice."
Getting back to Meena, what's the end result of all that geeky end-to-end neural network training? Meena is not only far more human-like in its conversations than its counterparts, but she's also funny.
And a budding philosopher, apparently.
That joke, by the way, was all Meena and officially makes her funnier than me. Whoever said chatbots are dumb and robotic?
Overall, Google was able to hone Meena to an SSA rating of 79%. That's particularly impressive given that it's not only 23% better than its closest competitor but, more importantly, it's also within spitting distance of the 86% score that was associated with humans.
If you'd like to get into all the nitty-gritty of how Google made Meena, you can read the 38-pages-long paper here.
We may earn a commission for purchases using our links. Learn more.
The Snapdragon 888 is here to turbocharge 2021 Android flagships
The Snapdragon 888 is now official, and the chipset boasts incredible gains across the board from the last-gen Snapdragon 888. The flagship chipset will make its way to phones from 14 manufacturers next year, but Samsung is a curious omission.
Here's your most detailed look yet at the Samsung Galaxy S21+
More renders of Samsung’s Galaxy S21 Plus have appeared online, giving us a better look at the design of the upcoming flagship phone. Like the Galaxy S21 and S21 Ultra, the S21+ will feature an Infinity-O display and a redesigned rear camera array.
What do you prefer — light mode or dark mode?
Light mode or dark mode? It's a timeless debate in the Android space, and we want to know which one you prefer.
Here are the best webcam lighting to improve your Zoom calls in 2020
If you find that the lighting is always off when you're on Zoom video conference calls, whether it's too harsh or too dark, webcam lighting can help. And there are some of the best webcam lighting products you can find.