the latest Nokia TVs arrive in Europe

This is how Google knows what song you're humming


Google can know the name of the songs you hum. But how? This is how this spectacular feature works.

Finding out the name or creator of a song through Google is as easy as hum the song and wait for the seeker to do his magic. In a few seconds - depending on your humming technique - you will know All the information you need About that music that has been playing in your head for days.

But, What is that magic really? Through its artificial intelligence blog, Google wanted to explain us in more detail how their systems work in charge of identifying the songs based on the sounds we produce.

Google Music Search

Google can tell what song you are humming.

A database of millions of songs and lots of AI: this is how “Hum to search” works

Google explains that when developing this feature, called “Hum to Search,” they had to face several challenges.

One of the most important is the fact that each person hums his own interpretation of the song, so the rhythm or pitch may not be suitable, making it difficult to search for the original song, especially when using a database which only includes the original versions of each song.

That is why Google decided model a system capable of producing a melody from a spectrogram of a song, without generating an intermediate representation. Or, in Spanish: the system is capable of match the hummed melody with the original polyphonic recording of the song.

Spectrogram

To represent how difficult it can be to identify a song based on a hummed melody, Google shows us a spectrogram display of the popular song Bella Ciao in the image above these lines. On the left, you can see the hummed version, while on the right you can see the original melody.

From the visualization of the hummed melody, the developed model must look for features that allow associating it to the original song, by searching a database of 50 million songs with similar spectrograms.

In that sense, Google has trained its system to focus only on the predominant melody, forgetting letters, instruments or the timbre of the voice. It is also capable of filter other noises such as ambient sound or that which can be produced by the reverberation of a room.

The company also explains that this function has been developed taking advantage of some advances in this field achieved in the past, such as the system Now Playing that Google first introduced together with the Google Pixel 2, and that allows identify any song that is playing around us, without the need for an Internet connection or a server.

Now Playing on Pixel 2 XL

Now Playing, one of the best features of the Pixel.

Through modifications carried out in the models used to bring to life both Now Playing and the Sound Search function, the neural network was trained with audio pairs, formed by a hummed version and another recorded from the original melody. Thus, a embedding each of them that would later be processed in search of traits that make clear the similarities between one and the other.

This, carried out millions of times with millions of different songs, was one of the great challenges when it came to train the neural network. Not because of the machine learning process itself, but because of the added difficulty of having to find hummed versions of the millions of songs used during this process.

And it is that for this, Google was forced to use SPICE, a tone extraction model that was used to generate hummed melodies that could be used to train the model. In that sense, Google shows us how the extracted melodies sounded, and how the original ones did.

From there, when the system was sufficiently trained, the technique was refined through other machine learning systems with which the search accuracy.

Furthermore, the company specifies that, while user hums are not shared automatically, it is possible to activate an option that allows “Hum to Search” to continue learning and improving its capabilities. Day by day, the database already has more than half a million songs.

How to use "Hum to Search"

Any user of an iOS or Android mobile can ** use “Hum to Search” to find songs simply by humming them –or whistling them–. To do this, you just have to follow these steps:

  1. Open the Google search engine, through the mobile app or from a web browser.
  2. Tap on the microphone icon and then on "What's this song?"
  3. Start humming or whistling the song you want to find, and wait for the search engine to identify it.

Functions like this, or like the Google Maps tool that knows how to tell us how busy a place is, are great examples of how artificial intelligence can make our lives easier.

Comments