Fingers as Legs, ETH Zurich Turns a Store Bought WUJI Hand Into a Walking Addams Family Extra

Amirhossein Kazemipour, Hehui Zheng, and Robert Katzschmann of ETH Zurich’s Soft Robotics Lab began with a commercial WUJI right hand with five fingers designed for grabbing. They put a Raspberry Pi, an IMU, and a battery to its back, resulting in a package weighing 818 grams and operating without a cable. Twenty powered joints remained factory standard, four on each finger, with different lengths and a thumb positioned to one side. Lift one finger to step, and a support point disappears.

In NVIDIA Isaac Lab, researchers used reinforcement learning to teach the hand. They conducted thousands of virtual trials at the same time, using a simulator that was fine-tuned to mimic the actual thing. A neural network examined joint angles, tilt, spin rate, and the most recent command it had received before determining the next twenty target positions for the joints. What it utilized to determine how well it was doing was derived from rewards that simply moved the fingertips toward reference points in a body frame, which proved to be far faster than using prizes from four-legged robots. The policy determined which finger to lift, how far to reach, and how the rest of the hand should remain off the ground.

In the real world, the hand crawled on its own across fourteen different surfaces, ranging from rubber mats, carpets, and hardwoods to gravel, grass, diamond plate, and stone along Zurich’s Limmat River. The crew had a controller connected to give steering inputs, and you know what? The geometry would do the smart thing and pull it back on course if it discovered itself drifting off course. The crew thought of it as a quirk, but it was still quite lovable. In terms of falls, a separate policy showed that it righted itself 21 out of 25 times, with the majority of those occasions taking less than 20 seconds.

It didn’t end there since the same fingers that could crawl about were also holding the rest of the hand up. Without any fancy cameras, the hand was able to tap out keyboard keys to play Sokoban and correctly executed 29 out of 32 commands with a quarter-second delay. With a camera above it, it was able to pick up a small cube and push it toward targets, performing reasonably well on average, but 17 millimeters off.

Kazemipour and his team believe this could be useful in a larger robot that is unable to approach a specific object. They believe they can just unclip this hand, allow it to do its job, and then walk straight back. There are no extra wheels or legs; only the robot’s existing fingers.
[Source]