Skip to content

UC-Merced, UC-Davis Collaborate on New Virtual Physical Therapy Software

MERCED — In the not-so-distant future, physical therapy patients could toss their hand outs with stick-figure drawings to the wind and instead do their prescribed exercises in the company of virtual physical therapists.

A new system using avatars and patient-specific motions offers new promise for physical therapy.

Computer scientist Marcelo Kallmann once used his expertise in computer animation, virtual reality and motion planning to develop new applications for video games. 

Then he shifted his focus. He knew gaming systems like Nintendo’s Wii and Microsoft’s Kinect had the potential to go beyond entertainment. He asked himself: “Could we use those same technologies to create therapeutic medical tools and to improve and deliver health care?”

Now the associate professor at UC-Merced is working on an answer to that question, one that could offer a long-term solution to an anticipated shortage of physical therapists — virtual physical therapy software.

Boomers’ Spurring New Delivery Solutions

Kallmann has collaborated with Jay Han, an expert in physical medicine and rehabilitation at UC-Davis. Han has long been an advocate for melding medicine and technology. As baby boomers age, he said it’s vital to come up with new ways of delivering health care.

“We’ve got to rely on innovativeness and technology to make the medical field more efficient, to see patients in a more targeted manner,” Han said.

With the help of engineers and physical therapists, they have developed a low-cost prototype that uses Kinect. The prototype is being evaluated at UC-Davis, and as physical therapists provide input, refinements are made.

The prototype’s genesis began at UC-Merced in a room with a “powerwall,” a 3-D, floor-to-ceiling projection screen. 

“We started here with the powerwall, and then we started to work on gestures and how gestures can be used” by autonomous characters in training applications, said graduate student Carlo Camporesi. “We realized one possible application was physical therapy.”

So how does it work? “You have a virtual human in front of you, like a character in a video game,” Kallmann said. The patient or user watches a computer-generated character demonstrate a set of exercises and then repeats them. The user’s avatar appears on screen, too, so the patient can see him or herself performing the exercises.

The software has a menu of exercises the therapist can provide to the patient. But the therapist can also customize the program by adding exercises specific to a particular patient’s needs. If a patient can’t perform an exercise in the typical manner, the therapist can record a new applicable movement.

The system can detect how well the patient is following the demonstration and can adjust the exercises in response. For instance, if a patient isn’t doing the repetitions as quickly as the virtual therapist, it will slow down.

Kallmann is now determining what parameters a therapist might want to control, while still keeping the system as simple as possible. These could range from speed and pause functions to how the exercises are displayed for different age groups.

Software Saves Time, Money, Motivates Patients

“I think it’s got great potential,” said Linda Johnson, a clinical evaluator for the project and a physical therapist at UC-Davis Medical Center. “Movement is a 3-D thing. When we’re reliant on paper, we end up having to cross things out, scratch and draw stick figures.”

She said the printed handouts that physical therapists currently use can be frustrating because the clients might not do the exercises enough or at all, or they might do them incorrectly.  

“They’re more likely to sit in front of a TV and do a physical therapy “game” than pick up a piece of paper and follow it,” she said.

With the virtual system, patients are more likely to do the exercises correctly, she said. A therapist can log in and see how often and how well the patient is doing the exercises, another potential motivator. And because it’s a video capture system, the therapist will also know when there is improvement in the patient’s motion and can adjust the exercises accordingly.

One of Johnson’s concerns is that elderly patients might be less inclined to use the software because they aren’t as tech savvy. But they are also typically the most intent on doing their exercises to get better. If the program started and senior patients can open up, they’ll be more likely to comply than other age groups, she said.

There’s another benefit: saving money. Insurance companies will like the program because patients will likely have fewer office visits, Johnson predicted.

Physical Therapists Remain Hands On

Feedback from physical therapists has been positive, Kallmann said. But there are some situations where a patient still needs real life interaction.

For instance, the system can’t be certain if the patient is compensating for one motion by moving another part of the body. A therapist might need to hold the patient in place, for instance, if the exercise is to move the arm but not the shoulder.

But the system can still be beneficial in such circumstances because the therapist could give attention to two patients at once if only half of the exercises require some sort of hands-on therapy, Kallmann said. 

“Physical therapy is unique because what we do is hands on, but this is an interesting way to meld technology with therapy in general, and it may allow us to extend the time between office visits rather than just seeing them automatically,” Johnson said. “And if clients have to [pay out of pocket], it also benefits them.”

She said physical therapists already use systems like the Wii for movement purposes, but there are no programs specifically made for physical therapy purposes. “I can have them do a bowling program or a tennis program, but that doesn’t have them do the actual exercise I would have them do as a PT.”

Many teleconference capabilities are in place, and Kallmann and his colleagues are still developing such functions. For instance, the physical therapist can also have an avatar that interacts and works with the patient’s avatar.  

There’s no fixed deadline on when the program will be sold commercially. “We are after longer-term funding to continue to develop ideas,” Kallmann said.

Related Topics

Health Industry Insight