Q&A: Teaching robots to touch and interact like humans

Robots are widely used in the automotive industry and have started entering new application domains such as logistics in the last few years. However, current robots still face many limitations. They typically perform a single action or a fixed sequence of actions, repeating them the same way each time.

To achieve greater efficiency and open new possibilities, robots must develop more human-like skills, such as fast physical interaction, spatial understanding, and fast adaptation to changes. Eindhoven University of Technology spoke with Alessandro Saccon, Associate Professor in nonlinear control and robotics at the department of Mechanical Engineering at the TU/e. He recently completed the I.AM project that explicitly focuses on the advancement of fast physical interactions.

Why are impact-aware robots so important for humanity?

Certain jobs are not particularly suited for humans from a safety or ergonomic perspective. For instance, when handling 20-kilogram luggage at airports, working in unsafe areas of a nuclear plant, or dealing with disaster scenarios, you might prefer a machine instead. There are also various plans to send them to space for planet exploration.

However, robots still statically interact with the environments when compared to us: the execution of certain key tasks is not yet possible or the execution is too slow. That’s why, in our project, we aimed to develop impact-aware robots. That means a robot has to learn to predict and react to what happens when it comes into fast contact with heavy objects in the environment.

What makes those robots different from the traditional robots we’ve known forever?

Typical robots are not designed to interact dynamically with their environment; making fast contact with surroundings is generally avoided at all costs. There is a very large number of scientific papers in the robotics literature whose focus is collision avoidance.

In the I.AM project, we targeted instead collision exploitation. We looked into how the robots can, for example, pick up heavy objects quickly while ensuring that the execution of this type of motion remains reliable, despite disturbances and perception inaccuracies. An object might be heavier than the robot anticipated, or it assumes that an object is at a certain location, but its slightly off—maybe even by a few centimeters. How do you make these movements robust despite such uncertainties? That’s one of the things that we have been researching in depth.

Practically speaking, what main activities were involved in your project?

The project employed first-principle physics calculations, using basic concepts such as mass and friction, along with software simulations to identify discrepancies between mathematical models and real-world events. Although simulations can never perfectly replicate robot behavior, we improved and most importantly understood how to still use these algorithms for controlling the robots.

We did this by taking real-time measurements of robots interacting with various objects in different scenarios. Its an iterative cycle where you develop and implement a theoretical robot control algorithm in a simulation, evaluate the results, and compare them with real-world outcomes.

Can you highlight some key findings from this project?

We discovered how we can make a robot reliably and swiftly grab a heavy object with two arms, by developing a new control algorithm that respects the natural impact dynamics. We also understood how to use software simulations to obtain predictions that can be used for this purpose or other impact tasks.

While working on this project, I also further appreciated how complex movements and spatial perception come so naturally to us humans. We academic researchers are now working very hard on hardware development, spatial perception, and planning—especially the ability to understand the environment in real time and quickly decide what to do next, also in case of failure. This is one of the grand challenges in modern robotics. These actions are natural and intuitive for us, yet we don’t fully understand how we do them nor how we should build a machine with similar abilities.

Were there also companies involved in this project?

Yes, for example, one of our partners was logistic process automation specialist VanderLande. This is a well-known and large company from the Netherlands that operates worldwide in various fields, such as airports, warehousing, and parcel handling. They provided a lot of real-life use cases and insights on what are the current bottlenecks in the field, the so-called market pains.

One of the great aspects of this collaboration was having a shared lab on TU/e campus, which facilitated hands-on testing and close cooperation. Students and researchers loved it. We have done various comparisons between real and software-simulated impact experiments as well as developed new models of suction grippers to be used for motion control and planning.

Would you say that the Netherlands is specialized in robotics?

The Netherlands as a whole is definitely making significant strides when it comes to robotics. The country has long been known for contributing to medical robotics, robot learning, and mobile robotics, to cite a few. I like to think that via this project and its international collaborations, we have also made significant strides in impact-aware robotics.

This area of research, which we could say originated here, has garnered global attention and recognition. Our project has played a key role in bringing the subject to the forefront, and its gratifying to see that our work has been well-received and acknowledged, including in recent publications.

The project is done. What does the future look like for you?

I will continue investigating and exploring new opportunities, including national and European funding. I’m considering follow-up projects focusing on areas that we couldn’t address during this one, like fast planning and perception. I’m also still in close contact with local and international companies to explore further collaboration opportunities. Some of the many students involved in the project have been hired in companies that were partners in the project: things are growing in the right direction. The visibility this project has generated is both challenging and exciting at the same time. While it means juggling more tasks, I’m also looking forward to seeing what the future holds.