Skip to content
AI Adventures in the Arts and Letters – TechCrunch


There are more AI news out there that anyone can follow. But you can stay up to date on the most interesting developments with this column, which collects advances in AI and machine learning from around the world and explains why they might be important to tech, startups, or civilization. .

To start on a light note: The ways researchers find to apply machine learning to the arts are always interesting – but not always practical. A team from the University of Washington wanted to see if a computer vision system could learn to say what is being played on a piano simply from an aerial view of the keys and the player’s hands.

Audeo, the system formed by Eli Shlizerman, Kun Su, and Xiulong Liu, watches a piano playing video and first extracts a simple sequence of key presses in the shape of a piano roll. Then it adds an expression in the form of the length and force of the presses, and finally tweaks it for input into a MIDI synthesizer for output. The results are a bit loose but definitely recognizable.

Image credits: Shlizerman, et. Al

“Creating music that feels like it could be played in a musical performance was previously considered impossible,” said Shlizerman. “An algorithm has to determine the cues, or ‘features’, in the video frames that are related to the generation of music, and it has to ‘imagine’ the sound that occurs between the video frames. It requires a system that is both precise and imaginative. It was a surprise that we made music that sounded pretty good.

Another in the field of arts and letters is this extremely fascinating research on the computer unfolding of old letters too delicate to handle. The MIT team were examining 17th-century “locked” letters that are so tightly folded and sealed that removing the letter and flattening it could permanently damage them. Their approach was to x-ray the letters and define a new advanced algorithm to decipher the resulting images.

AI Adventures in the Arts and Letters – TechCrunch

Diagram showing x-ray views of a letter and how it is analyzed to virtually unfold it. Image credits: MIT

“The algorithm ends up doing an impressive job of separating the layers of paper, despite their extreme thinness and tiny spaces between them, sometimes below the resolution of the scan,” said Erik Demaine of MIT. “We weren’t sure it would be possible.” The work can be applicable to many types of documents which are difficult to disentangle for simple x-ray techniques. It’s a bit of a stretch to categorize this as “machine learning”, but it was too interesting not to include it. Read the full article on Nature Communications.

AI Adventures in the Arts and Letters – TechCrunch

Image credits: Asensio, and. Al

You arrive at a charging station for your electric car and find that it is out of order. You might even leave a bad review online. In fact, thousands of such reviews exist and are a potentially very useful map for municipalities looking to expand electric vehicle infrastructure.

Georgia Tech’s Omar Asensio trained a natural language processing model on such criticisms and he quickly became an expert at analyzing them by the thousands and extracting information such as where outages were common, comparative costs and costs. other factors.



Source link