Iowa Alumni Magazine | June 2009 | Features

Science in Motion

By Tina Owen
Made possible by breakthrough technology at the UI College of Engineering, a virtual human is redefining the digital world—and the real one.

The armored Humvee sprawls upside-down, wheels spinning and its vulnerable underside exposed to enemy attack. Inside, a camouflage-clad soldier struggles to escape the vehicle that now represents a danger instead of a safe haven. As he attempts to squirm out of a hatch, his armored vest and weapon holster snag. He's trapped.

He breathes heavily and his heart beats wildly before he manages to shrug free of his gear and scramble to safety. He'll live to fight another day and another battle. Virtually, anyway.

This is Santos. He isn't a real person, and he isn't soldiering away in Iraq or Afghanistan. He's a digital human—a software package created by University of Iowa engineers and researchers at the Virtual Soldier Research (VSR) lab—who carries out mission after mission inside simulated environments on a computer. What exactly is his mission? To save dollars and time and to improve human comfort and safety by testing prototypes of products that range from military armor to factory production lines to office furniture.

When he clambers into a digital version of a new tank to check its ergonomic design and safety features, Santos is saving the U.S. Army multiple years and millions of dollars that it takes to construct a real prototype. As well as offering information about the tank, Santos provides crucial feedback about how humans react in this environment.

When he steps into virtual prototypes of military clothing designed for chemical, biological, and nuclear warfare, Santos helps work out the product's kinks. Before a manufacturer even pieces together the garment or a real soldier wears the garb in a real emergency, Santos can report on whether the gear is hot or uncomfortable, whether it impedes his movements in a potentially hazardous way.

Simulated product testing is now the cost-effective norm in many industries. Before investing big money in a new car, an auto company will create a virtual model to test how well the design works. Difficulties arise when researchers need to assess how humans interact with the new product. Virtual humans are the perfect solution, says Karim Abdel-Malek, UI professor of biomedical and mechanical engineering, VSR director, and the "father of Santos": "Now there's a digital human to do that last step in production."

Santos isn't just a fancy product tester, though. Probably the world's most advanced digital being, he represents enormous technical breakthroughs in the science of simulated human movement. He combines advanced computer animation and motion capture technology with the UI College of Engineering's renowned expertise in robotics, biomechanics, and modeling and simulation.

Born almost six years ago on the University of Iowa campus, Santos still spends most of his time at the College of Engineering's Virtual Soldier Research lab, where large colorful posters bearing his image proudly proclaim the facility to be "Home of Santos." Abdel-Malek, an expert in robotics, devised Santos and later recruited a 36-strong team of mechanical and biomedical engineers, software designers, and computer animation experts to help this digital human push the limits of technology.

Since then, Santos has earned a lot of money, publicity, and kudos. He's showcased his talents on the Discovery Channel and in Wired and Forbes magazines. He's garnered the Virtual Soldier Research team $17 million in grants, as well as three "best paper" awards for academic research and a 2007 Prometheus Award from the Technology Association of Iowa.

On the surface, Santos is a carefully drawn, shaded, and textured representation of a three-dimensional figure. Unlike the sword-wielding, zombie-killing avatars made so familiar in video games and Hollywood movies, though, he's much more than a sophisticated cartoon.

Somewhere in the real world, Santos has a human doppelgänger—the bodybuilder whose detailed, digitized 3-D body scans laid the physiologically accurate foundations for the virtual human's musculoskeletal system. Santos can realistically lift an arm or a rifle, walk up stairs, or swing a leg across a motorcycle because complex mathematical algorithms and computer code translate the actual movements and functions of human bones, muscles, tendons, and skin.

Santos has penetrating blue eyes, a shaven head, dark hair sprinkled across strong hands and arms, and a colorful tattoo that stretches across his contoured shoulder. Overall, he's fit, muscular, and looks capable of meeting any challenge. Even his name is quirkily human rather than a technical acronym. In the early days of the VSR project, a member of the UI research team took one glance at the image of the stocky bodybuilder and said, "He looks like a Santos."

Santos is incredibly versatile. One day, he might stand six feet tall and weigh 250 pounds; at other times, depending on the requirements of his particular assignment, he can alter his height, weight, and other physical attributes. In this way, he provides feedback from a variety of human perspectives. He can even change his tattoo, depending on whether he's working for the Army or the Marine Corps.

Although he started life as a soldier—thanks to a $3 million grant from the U.S. Army that provided early funding for the VSR project—Santos has since acquired other skills. He's tackled assignments for companies and organizations including John Deere, Caterpillar, Rockwell Collins, and the United States Council for Automotive Research. Blue collar worker Santos assembles cars on a factory production line, while corporate executive Santos sits at a desk and works on a computer. He's equally at home in the cockpit of an airplane or behind the wheel of a John Deere tractor.

In all these scenarios, Santos is the strong, silent type. He communicates with his human overseers not through speech but via computer data that provide information such as his breathing rate, temperature, exertion level, and even whether he's tired or suffering from an aching back. Eventually, UI researchers hope, Santos will be able to talk—and even think independently. If an ambitious project between the VSR team and UI Hospitals and Clinics works out, Santos will further blur the line between digital and human. A collaborative effort drawing on the expertise of engineers, neuroscientists, psychologists, and computer scientists aims to give Santos a brain—and a heart.

The project's goal is to improve Santos's human-like decision-making so that he's completely independent in his virtual environments. Right now, Santos is like a two-year-old child: still learning, still figuring out his world, still relying greatly on others' judgment and instructions. "He's the ideal soldier," laughs Abdel-Malek. "He doesn't think for himself; he just obeys orders."

In the crashed Humvee scenario, Santos couldn't have figured out that he needed to ditch his gear to escape from the vehicle. A programmer had to tell him to do it. But, like any child, Santos does learn from repetition of tasks and can apply that knowledge autonomously as he encounters new situations and interacts with new environments.

His new, improved brain will be based on a neural network, a type of artificial intelligence that imitates the way a human brain works. With this approach, Santos will break down missions into hierarchical structures with a number of subtasks, selecting the most optimal solution for each task.

As for his "heart," Santos will one day benefit from a computer-based cognitive model that will enable him to experience emotions such as sadness, happiness, and anger. Like any human, he'll incorporate these reactions into his decision-making process.

Those advances are still a ways off, but, just a few weeks ago, Santos took his next big step. The VSR research team launched a public company, SantosHuman, to make the technology more widely available.

Promoted as "the Science of Human Motion," Santos is going to change the world. Virtually.