Google is revealing new details about the creation of its DeepRank algorithm, which makes search results more relevant by understanding language the way humans do.
DeepRank is explained in detail in a new Google video about how search works.
Among other aspects of search, the Google video reviews the development, testing and approval process that each algorithm update goes through.
Google DeepRank was launched in 2019 as BERT, and is named for the deep learning methods that BERT uses and the ranking aspect of search.
Think of DeepRank as the integration of BERT into Google search. Although it’s not as simple as it sounds.
Here is a summary of what is shown in the new Google video.
DeepRank/BERT
DeepRank is based on Google’s machine learning and natural language processing capabilities.
Natural language processing allows search engines to enter real questions in the same way they would ask a friend in a text message, for example.
Google has been working on natural language processing for most of the 20 years, starting first with spell checking and understanding synonyms.
Google’s machine learning capabilities have been in development for 10 years.
BERT further improves the understanding of natural language query search algorithms.
Search ranking has advanced greatly over the past two decades, but before BERT, the algorithms could not understand the subtleties of language as humans do.
DeepRank is the first time that Google Search has a signal that understands the relationship between human terms.
Ultimately, it is designed to make search more intuitive and make it seem like Google understands its users.
Development and testing of Google DeepRank
DeepRank captures the subtleties of language that come naturally to humans but are difficult to program.
Google’s search algorithms are used to ignore stop words and leave them out of the query.
Over time, Google learned that these words play an important role in communicating what people are really trying to say.
With Google DeepRank, people can phrase search queries naturally and not be bothered by machines not grasping the subtleties.
The Google video shows an example of the query: “What temperature should you preheat the oven to when you cook fish?
Without DeepRank, Google’s algorithms surface some useful information, but they also get confused.
When you try DeepRank in the query, it is understood that the page is about cookies, not fish, and the result does not appear.
Here’s an example that the DeepRank team is especially proud of that has to do with the question “can you get medicine for a pharmacy?
DeepRank may seem like a simple solution, but getting BERT to play nice with the search isn’t easy, according to Google.
DeepRank went through months of testing, and you can see some of it firsthand in the Google video.
The video of the process
There’s a rare behind-the-scenes video of Google search engineers meticulously analyzing individual queries to determine if DeepRank helps or hurts search results.
Any change in the search is subject to great scrutiny, no matter how large or small.
However, Google engineers have a finite amount of time to work on this project.
DeepRank, even in testing, requires an enormous amount of computing power. If the team working on DeepRank had not made sufficient progress in testing, computer resources would have been allocated to another project.
The video has more behind-the-scenes footage of a meeting where DeepRank’s progress is presented to the Google launch committee.
If you’ve ever wanted to see the process Google Search engineers go through to get updates and changes approved, this might be the best look we’ve had so far.
The Google committee approved the DeepRank for an eventual launch, and the sense of euphoria from Google’s engineers was palpable.
See the full video below. The part about natural language processing and DeepRank starts around the 42-minute mark.
This post is also available in:
Español (Spanish)
