Saturday, August 20, 2022
HomeArtificial IntelligenceContextual Rephrasing in Google Assistant

Contextual Rephrasing in Google Assistant

When individuals converse with each other, context and references play a vital function in driving their dialog extra effectively. As an illustration, if one asks the query “Who wrote Romeo and Juliet?” and, after receiving a solution, asks “The place was he born?”, it’s clear that ‘he’ is referring to William Shakespeare with out the necessity to explicitly point out him. Or if somebody mentions “python” in a sentence, one can use the context from the dialog to find out whether or not they’re referring to a kind of snake or a pc language. If a digital assistant can not robustly deal with context and references, customers could be required to adapt to the limitation of the know-how by repeating beforehand shared contextual data of their follow-up queries to make sure that the assistant understands their requests and may present related solutions.

On this publish, we current a know-how at present deployed on Google Assistant that enables customers to talk in a pure method when referencing context that was outlined in earlier queries and solutions. The know-how, based mostly on the most recent machine studying (ML) advances, rephrases a person’s follow-up question to explicitly point out the lacking contextual data, thus enabling it to be answered as a stand-alone question. Whereas Assistant considers many kinds of context for decoding the person enter, on this publish we’re specializing in short-term dialog historical past.

Context Dealing with by Rephrasing

One of many approaches taken by Assistant to know contextual queries is to detect if an enter utterance is referring to earlier context after which rephrase it internally to explicitly embrace the lacking data. Following on from the earlier instance during which the person requested who wrote Romeo and Juliet, one could ask follow-up questions like “When?”. Assistant acknowledges that this query is referring to each the topic (Romeo and Juliet) and reply from the earlier question (William Shakespeare) and may rephrase “When?” to “When did William Shakespeare write Romeo and Juliet?”

Whereas there are different methods to deal with context, as an example, by making use of guidelines on to symbolic representations of the which means of queries, like intents and arguments, the benefit of the rephrasing strategy is that it operates horizontally on the string degree throughout any question answering, parsing, or motion achievement module.

Dialog on a sensible show machine, the place Assistant understands a number of contextual follow-up queries, permitting the person to have a extra pure dialog. The phrases showing on the backside of the show are solutions for follow-up questions that the person can choose. Nevertheless, the person can nonetheless ask totally different questions.

A Extensive Number of Contextual Queries

The pure language processing subject, historically, has not put a lot emphasis on a basic strategy to context, specializing in the understanding of stand-alone queries which are absolutely specified. Precisely incorporating context is a difficult drawback, particularly when contemplating the massive number of contextual question varieties. The desk under accommodates instance conversations that illustrate question variability and a number of the many contextual challenges that Assistant’s rephrasing technique can resolve (e.g., differentiating between referential and non-referential circumstances or figuring out what context a question is referencing). We display how Assistant is now in a position to rephrase follow-up queries, including contextual data earlier than offering a solution.

System Structure

At a excessive degree, the rephrasing system generates rephrasing candidates by utilizing several types of candidate turbines. Every rephrasing candidate is then scored based mostly on quite a few alerts, and the one with the best rating is chosen.

Excessive degree structure of Google Assistant contextual rephraser.

Candidate Technology

To generate rephrasing candidates we use a hybrid strategy that applies totally different strategies, which we classify into three classes:

  1. Turbines based mostly on the evaluation of the linguistic construction of the queries use grammatical and morphological guidelines to carry out particular operations — as an example, the alternative of pronouns or different kinds of referential phrases with antecedents from the context.
  2. Turbines based mostly on question statistics mix key phrases from the present question and its context to create candidates that match widespread queries from historic information or widespread question patterns.
  3. Turbines based mostly on Transformer applied sciences, corresponding to MUM, study to generate sequences of phrases based on quite a few coaching samples. LaserTagger and FELIX are applied sciences appropriate for duties with excessive overlap between the enter and output texts, are very quick at inference time, and should not susceptible to hallucination (i.e., producing textual content that isn’t associated to the enter texts). As soon as offered with a question and its context, they’ll generate a sequence of textual content edits to remodel the enter queries right into a rephrasing candidate by indicating which parts of the context must be preserved and which phrases must be modified.

Candidate Scoring

We extract quite a few alerts for every rephrasing candidate and use an ML mannequin to pick probably the most promising candidate. A number of the alerts rely solely on the present question and its context. For instance, is the subject of the present question just like the subject of the earlier question? Or, is the present question a great stand-alone question or does it look incomplete? Different alerts depend upon the candidate itself: How a lot of the data of the context does the candidate protect? Is the candidate well-formed from a linguistic standpoint? And many others.

Not too long ago, new alerts generated by BERT and MUM fashions have considerably improved the efficiency of the ranker, fixing about one-third of the recall headroom whereas minimizing false positives on question sequences that aren’t contextual (and due to this fact don’t require a rephrasing).

Instance dialog on a telephone the place Assistant understands a sequence of contextual queries.


The answer described right here makes an attempt to resolve contextual queries by rephrasing them with a view to make them absolutely answerable in a stand-alone method, i.e., with out having to narrate to different data in the course of the achievement part. The advantage of this strategy is that it’s agnostic to the mechanisms that will fulfill the question, thus making it usable as a horizontal layer to be deployed earlier than any additional processing.

Given the number of contexts naturally utilized in human languages, we adopted a hybrid strategy that mixes linguistic guidelines, giant quantities of historic information via logs, and ML fashions based mostly on state-of-the-art Transformer approaches. By producing quite a few rephrasing candidates for every question and its context, after which scoring and rating them utilizing a wide range of alerts, Assistant can rephrase and thus accurately interpret most contextual queries. As Assistant can deal with most kinds of linguistic references, we’re empowering customers to have extra pure conversations. To make such multi-turn conversations even much less cumbersome, Assistant customers can activate Continued Dialog mode to allow asking follow-up queries with out the necessity to repeat “Hey Google” between every question. We’re additionally utilizing this know-how in different digital assistant settings, as an example, decoding context from one thing proven on a display screen or enjoying on a speaker.


This publish displays the mixed work of Aliaksei Severyn, André Farias, Cheng-Chun Lee, Florian Thöle, Gabriel Carvajal, Gyorgy Gyepesi, Julien Cretin, Liana Marinescu, Martin Bölle, Patrick Siegler, Sebastian Krause, Victor Ähdel, Victoria Fossum, Vincent Zhao. We additionally thank Amar Subramanya, Dave Orr, Yury Pinsky for useful discussions and assist.



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments