Multi-pixel magnetic sensing gives robots a smooth touch

Industrial robots are more affordable than ever for high-speed pick/packing and precision assembly tasks. Thanks to improvements in abilities like vision, each new generation offers more human-like dexterity and flexibility. A reliable and cost-effective sense of touch now enables them to manipulate fragile objects to perform an even greater variety of tasks and interact more safely with people.

Force sensors for robots

Various techniques have been explored to introduce touch sensing into robots. Among these are liquid metal sensors, which measure the resistance of a liquid metal flowing in microfluidic channels and modulated by external forces. Although this type of sensor can be embedded around the robot's fingertips, it does not measure localized 3D force. Instead, the distributed forces are assigned to a resistance change. A group of these sensors can detect a pattern. Hence, this technique is used above all in specialized classification tasks after specific training.

Another approach uses a high-end optical camera to measure the deformation of an elastomeric material embedded in or covering gripping surfaces. This technology is commercially available and is already used in multimodal intelligent robotic grippers. However, the camera requires a considerable pixel array and transmitting the data at video rate for analysis requires considerable communication bandwidth and power.

A different solution, based on optics, overcomes some of these challenges by using a quadrant of photodiode detectors instead of a complete camera. A light source illuminates an elastic dome from the inside, and detectors detect deformation of the dome due to contact forces. However, the power consumption is several times that of typical 3D magnetometers, which can provide an even simpler and more efficient alternative.

In these magnetic sensors, a magnet is embedded in an elastomeric material similar to that used in the camera-based system. A rear-mounted magnetometer provides 3D force sensing by measuring the change in magnetic field caused by magnet displacement as the elastomer deforms. Several such sensors have been demonstrated using a single output magnetometer, which can be thought of as a touch pixel or 'taxel'. The researchers have built various configurations, from simple 2x2 single-pixel arrays to a continuous 15mm2 magnetic skin made up of magnetic microparticles. Single-pixel magnetic force sensors such as these have achieved sub-1mm resolution by combining the sensor with the sinusoidal magnetization of a flexible film and deep learning techniques.

The magnetic sensors referenced here have used the Melexis MLX90393 single-pixel magnetometer. While the advantages of magnetic detection include relatively low power and minimal computation and communication overhead, single-pixel detection is vulnerable to interference from external magnetic fields. The output of the magnetometer can be distorted by unrelated effects in the vicinity, such as the activation of an electric motor, the presence of other magnets, or variations in the earth's magnetic field. A magnetic force sensor with multiple nearby pixels within the same integrated circuit (IC) package (Figure 1) can provide immunity to stray fields by allowing differential measurement. This article describes how the Tactaxis multi-pixel gradiometric magnetic sensor was built and tested.

sensor prototype

The MLX90372 linear displacement sensor provides a convenient platform for demonstrating the gradiometric sensing principle. This sensor typically outputs the angular displacement along an arc. However, setting the device to test mode allows direct access to the raw magnetic readings of individual pixels from memory. The sensor is housed in a standard 5mm x 4,3mm x 0,9mm TSSOP package and contains two side-by-side CMOS chips, with two pixels per chip. Therefore, this single and compact component contains four magnetic pixels placed about 2 mm apart that allow the measurement of the magnetic field gradient. Each pixel detects the normal component of the Bz field and the Bx component in the plane.

On top of the IC package is placed a soft elastomer containing an embedded disk magnet with axial magnetization. The use of a cylindrical shaped elastomer sample minimizes magnet tilt and presents a flexible surface. The application of a contact force on the elastomer displaces the magnet, which modulates the pattern of the magnetic field. Each of the four magnetic pixels detects this displacement and therefore the effects of normal and lateral forces. For a normal displacement of the magnet, the gradiometric component ∂Bx/∂x is the most impacted. On the contrary, for a lateral displacement, the impact is mostly on the gradiometric component ∂Bz/∂x.

Note that the sensor only measures the displacement of the magnet. The connection between the displacement and the applied force depends on other factors and requires additional calibration and calculation. The size of the magnet, the hardness of the elastomer, and the diameter of the elastomer barrel all influence the magnitude of the sensor output when a force is applied. A larger, more powerful magnet increases signal-to-noise ratio (SNR), with little impact on full-scale force. A harder elastomer allows for greater full-scale force. However, for the same applied force, the displacement of the magnet will be less compared to a softer material. Therefore, increasing the hardness also reduces the change of the magnetic signal and therefore the SNR. Finally, the diameter of the elastomer acts as a scaling factor between the global force and the localized pressure just above the magnet. A larger diameter spreads the force over a larger area, thereby reducing the SNR, while accommodating a larger full-scale force.

Signal processing and inference

Figure 2 shows the functional block diagram of the signal chain.

Signal processing is done off-chip and begins by scaling the eight digital output signals from the chip to correct for the Hall effect sensitivity drop with increasing temperature (-0,5%/◦C).

Stray magnetic fields are then rejected using combinations of field components. The mean of the Bx field and the Bz field are removed first, leaving the remaining terms that are related to the magnetic field gradient. In effect, the force sensor algorithm processes the magnetic field differences within the two arrays.

The feature augmentation block computes the √(Bx2 + Bz2) norm at each detection pixel, producing a 12-dimensional vector signal {Bx, Bz, Bnorm} at each pixel.

The last step generates a new vector containing all second-order polynomial combinations of the 12-dimensional vector, including the interaction terms. This produces a vector signal of dimension 91.

Finally, the inference stage calculates the planar force and torque values ​​from the vector signal using a 91 by 5 weight matrix. The weights are obtained by a training procedure using a reference load cell mounted on a 3-axis moving platform to tension the elastomer by applying a known displacement. The load cell force and the corresponding magnetic signals from the sensor were measured and stored at 13.000 positions over a travel range of 1,5 mm depth and 1,1 mm radius.

Stray Field Immunity (SFI)

To demonstrate how the effects of external fields are removed, the sensor was placed between two Helmholtz coils generating ±2 mT (Figure 3a). A field of equivalent intensity can be experienced at a distance of about 3 cm from common electrical appliances.

The force measured by the sensor using the gradiometric concept is shown in figure xb (blue curve), showing that the stray field error is limited to 0,3% of full scale. The sensor prototype was then reconfigured to function as a simple magnetometer without stray field rejection, emulating the behavior of previous single-pixel sensors. The stray field is filtered directly into the signal path without rejection, producing errors of up to 20% at -2mT (red curve). This is almost two orders of magnitude more than the gradiometric sensor.

Integration in the robotic hand

The Tactaxis sensor prototype was mounted on a commercial robotic hand. A basic force control algorithm was implemented for the hand to gently grasp a balloon. Figure 4 shows the demo setup.

Using the sensor as a single pixel planar magnetometer, the force is initially well regulated in the absence of stray field disturbances. The introduction of a stray field via a magnet corrupted the force sensor, causing the hand to release or crush the balloon depending on polarity.

When the Tactaxis sensor was used in its proper multi-pixel mode, the force remained correctly regulated at all times, unaffected by the magnet approaching up to a distance of a few centimeters.

Summary

The table shown compares the properties of the multi-pixel Tactaxis sensor with commercially available single-pixel magnetic, optical, and piezoelectric force sensors.

Table 1. Gradiometric sensor compared to other magnetic, optical and piezoelectric sensors.

The single-pixel magnetic sensor is compact and achieves 3D force vector sensing with state-of-the-art resolution, especially in multi-sensor configurations. However, sensitivity to stray fields remains a key limitation.

The optical sensor is naturally immune to stray magnetic fields and offers similar 3D force sensing performance. While this is an excellent feature for robotic hand integration, the discrete optical components add to the cost.

Piezoresistive sensors have the advantage of a small form factor, about the size of an IC package, and achieve competitive force resolution, but can only detect normal force.

Instead, the Tactaxis multi-pixel sensor prototype offers the familiar advantages of magnetic sensors, ie 3D force sensing, smoothness, economy and compactness, with superior immunity to real-world stray fields. Therefore, this gradiometric sensing concept has advanced the robustness of force sensing for robotic applications.