MIT artificial intelligence creates incredibily realistic sound effects.
MIT scientists have developed a system that can produce remarkably realistic sounds on its own. These sounds are so realistic that even human listeners cannot tell the difference.
Don't Miss: The Best HDR TVs
The effort is a small part of a larger plan, which is intended to help robots understand how different objects interact with the world. When robots will be able to make reasonable assumptions about their surrounding and consequences of different events, it will help them navigate the world more intelligently and realistically.
MIT researchers demonstrate how an algorithm has effectively learned to fit sound effects in silent videos where a drumstick can be seen hitting different objects such as grass, metal surface, cups, sidewalks and a small pool of water.
“When you run your finger across a wine glass, the sound it makes reflects how much liquid is in it,” said lead author Andrew Owens from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL). “An algorithm that simulates such sounds can reveal key information about objects’ shapes and material types, as well as the force and motion of their interactions with the world.”
The technique is similar to how people learn things through sounds. For instance, kids poke different objects not only for fun but for developing an intuition about physics or to understand an untrained perception all humans have about objects in the physical world.
In robotics, it is an example of “deep learning” where a system learns to recognize patterns in huge pile of data and use them efficiently in many ways.
For adding sounds to silent videos, researchers had to train a sound-producing algorithm. They fed several videos of sounds to the algorithm. Those were roughly 1,000 videos of an estimated 46,000 sounds being produced when different objects were hit with a drumstick. Algorithm analyzed the various features of the sounds such as pitch and loudness and successfully able to reconstruct them.
“To then predict the sound of a new video, the algorithm looks at the sound properties of each frame of that video, and matches them to the most similar sounds in the database,” said Owens. “Once the system has those bits of audio, it stitches them together to create one coherent sound.”
To test how realistic those fake sounds were, the team conducted an online study in which subjects saw two videos - one with the actual recorded sound, and one with the algorithm’s - and were asked which one was real. Subjects picked the fake sounds over the real ones twice as often, indicating how remarkably realistic sounds were produced by the algorithm.
Don't Miss: Incredible Pokemon Gifts
Still, researchers believe there is a room for improvement. They were aiming to enhance the capacity of algorithm. So, it can catch even minute details and produce sound effects for movies and TV shows automatically.