Amazon and University of Michigan give robots a sense of touch

Amazon and the University of Michigan have developed HydroShear, a physics-based simulator that gives robots the ability to accurately model tactile forces, according to Amazon Science. The work targets contact-intensive tasks such as warehouse automation and surgical assistance, where robots need to sense forces on their fingertips to manipulate objects effectively.

HydroShear builds on hydroelastic contact models and adds path-dependent force tracking. Instead of computing forces from instantaneous contact alone, the simulator tracks the motion history of an object as it moves across a soft sensor membrane, tracking individual contact points and converting soft elastomer deformations into force fields that account for friction, slipping, and material properties. It handles full 3-D motion, including tilting and rolling as well as in-plane sliding, and is GPU parallelizable for large-scale policy training. The team calibrated the simulator using controlled real-world data collected with a robot arm and vision-based GelSight Mini tactile sensors, isolating four parameters: force dissipation across the sensor surface, tangential and normal force buildup, and the friction coefficient between object and elastomer.

Existing tactile simulators face a trade-off: physics-based methods like finite-element methods model contact forces accurately but are too slow for training reinforcement learning policies at scale, while faster approximations oversimplify how forces change during contact and miss events such as the onset of sliding or sensor deformation over time.

The researchers evaluated HydroShear on four contact-rich tasks: peg insertion, bin packing, book shelving, and drawer pulling. In all tasks the robot perceived touch and proprioception, with no access to object poses. Policies trained entirely in simulation were deployed on a real Franka robot with GelSight Mini sensors without modification or fine tuning, achieving a 93 percent average success rate. Baselines TacSL and FOTS reached 34 percent and 58 to 61 percent, respectively.

The team says HydroShear's principles could extend to other tactile modalities, and that it is exploring higher-resolution sensor simulations and more complex object geometries.