When we listen to an mp3—or a CD track, a mix tape, a record, even a wax cylinder—we’re conjuring up the sound of a past performance. Now there’s a new computer program that does the reverse, sort of: It takes an audio file and creates a piano-playing cartoon, using sound (or related bits of information) to animate a performance anew.
The fingers of an expert pianist look relaxed as they tickle the ivories, never seeming overexerted or out of place. The computer program is designed to work on the same principle: It “listens to” a midi file and decides how the cartoon should finger each chord in order to put out the least possible effort. In addition to telling the active fingers which keys to press down, it also decides what to do with the idle fingers so that they are optimally relaxed and poised to play their next notes. Looking this at ease is a lot of work.
The virtual world is getting more realistic. New animation advancements in true-to-reality rumpling of clothes and face reddening are pushing us closer to the event horizon of the Uncanny Valley.
The first advancement is an algorithm designed to give animated clothes life-like wrinkling and crumpling while you are besting that orc. While more realistically rendered clothing won’t increase your manna, it may make digital effects in the next Matrix movie even better, New Scientist reports:
“This is exactly what people like me want,” says Andy Lomas, a software developer who produced digital effects for the film The Matrix and is based at computer graphics firm The Foundry in London. “I want to be able to capture the fundamental nature of an actor’s clothing, but also have the freedom to change the way he or she moves.”
The algorithm was created from footage of people IRL. The researchers, lead by Carsten Stoll at the Max Plank Institute, mapped the actor’s movements (and how their clothes moved in reaction) onto a skeleton, which they could animate. Animations of new movements of the skeleton were able to recreate how clothing would move in real life, Stoll told New Scientist:
“If the double is wearing a chiffon skirt in the original sequence, it will swish realistically in all of the new sequences too,” says Stoll.