Hope for an artificial hand with new simulation

By Dr. Liji Thomas, MDSep 27 2019

The human hand is among the most amazing and complicated parts of the body in its ability to exert both brute strength and delicate manipulation, depending on the need. Despite decades of study, scientists know only a little about its underlying structure, how the muscles and the tendons operate to move the many bones of the hand in relation to one another. Without knowing how a real hand is built, a model that replicates its anatomy and movements is almost impossible to build. This lack of inside information is why making a computer simulation of a human hand’s working is one of the most difficult problems in the world of computer graphics, and especially animation.

belushi | Shutterstock

Now, a new study called Hand Modelling and Simulation using Stabilized Magnetic Resonance Imaging, reported at ACM SIGGRAPH shows a simulation that incorporates not only the skin but the muscles, the bones, the tendons and the joints.

The hand is very complicated, but before this work, nobody had built a precise computational model for how anatomical structures inside the hand actually move as it is articulated.”

Researcher Jernej Barbic

The skillfully detailed model could drive the development of an artificial hand, besides being crucial in teaching a new generation of medical and paramedical students, building robotic hands and simulations for virtual reality training models and games.

How they did it

The first step was to form a team comprising experts in computer animation and those who knew how to build simulations based on physical reality, as well as radiologists and other anatomical specialists.

The next challenge was to find the right imaging method that could pick up details of the hand’s anatomy at each step of its movement, in a systematic manner. MRI scans provide a wealth of detailed information as to the anatomy of the hand but require that the hand be kept completely motionless in each pose for about 10 minutes – which is not achievable in realistic terms.

Barbic says, “Holding the hand still in a fixed pose for 10 minutes is practically impossible. A fist is easier to hold steady, but try semi-closing your hand and you'll find you start to shake after about a minute or two. You can't hold it still for 10 minutes.”

Making a support mold

To achieve this, therefore, they set up a production process to keep the hand stable in each pose, using materials borrowed from the special effects field. In lifecasting, the human form is first molded and then rebuilt in plastic, silicone, or other materials. Barbic found a cheap and easily available tool to clone a human hand in a visual effects store. Says Barbic, of his find, “That was the eureka moment.”

The third step was building a plastic lifecast of the hand they wanted to image, which showed each minute detail, including the pores and tiny lines on the skin’s surface. They built a lifecast in an elastic rubbery material, producing a 3D negative mold which could ergonomically support the real hand in the position required for as long as needed to complete the MRI scan. Now, 10-minute scans were taken of the hand, each time in a different position, using one male and one female model. There were 120 scans in total.

Understanding bone movements

The scientists cut up the whole-hand into equal segments called bone meshes, corresponding to the animator’s mesh of connected vertices and triangles, for each pose. These help show how individual bones changed positions in each pose. In the end, the scientists could delineate the exact musculoskeletal apparatus in action for each hand pose. This was fundamental to producing an accurate bone rigging driven by interpolative and extrapolative MRI-based data for all the bone meshes.

Building the moving animation

This led to the final step: constructing a moving simulation which allows every possible hand pose to be modeled by using the underlying data on skeletal movement, including complex rotations and displacements of the individual bones with different types of hand movement.

The soft tissue simulation was then built using a method called FEM (finite element method) to incorporate the computed movement of the muscles, the tendons and the linked fatty tissue of the hand as expected from the skeletal movement. They introduced modifications which allow a stable and faithful representation of skin folds and creases with joint movement. Finally, they added in the surface details, which culminated in an evenly moving animated hand that can take up any position, even one which is not part of the original set.

Value of this simulation

Of course, the work will be extremely valuable to those who design and manufacture computer games and movies based on computer-generated images (CGI).

This is currently the most accurate hand animation model available and the first to combine laser scanning of the hand's surface features and to incorporate an underlying bone rigging model based on MRI.” Barbic adds, “Understanding the motion of internal hand anatomy opens the door for biologically-inspired robotic hands that look and behave like real hands.”

Co-researcher George Matcuk

As the next step, the researchers want to take their MRI data to the public domain, and to add on many other poses, taken on ten models in all, over three years. This will help to simulate the human hand and eventually recreate it. It could also be used to reach medical students who need to understand how the hand moves and how it is built. According to Matcuk, “As we refine this work, I think this could be an excellent teaching tool for my students and other doctors who need an understanding of the complex anatomy and biomechanics of the hand.”

The team also wants to improve the model’s sensitivity to muscle and tendon movement, allowing it to react to actual movement in real-time as opposed to the current hour-long computational process for a minute-long simulation. They aim to increase the speed of data retrieval and computation without compromising on the quality of the simulation.