Teaching Robots Interactively (TERI)

Project type

ERC Starting Grant; 2019-2023

Abstract

Programming and re-programming robots is extremely time-consuming and expensive, which presents a major bottleneck for new industrial, agricultural, care, and household robot applications. My goal is to realize a scientific breakthrough in enabling robots to learn how to perform manipulation tasks from few human demonstrations, based on novel interactive machine learning techniques.

Current robot learning approaches focus either on imitation learning (mimicking the teacher’s movement) or on reinforcement learning (self-improvement by trial and error). Learning even moderately complex tasks in this way still requires infeasibly many iterations or task-specific prior knowledge that needs to be programmed in the robot. To render robot learning fast, effective, and efficient, I propose to incorporate intermittent robot-teacher interaction, which so far has been largely ignored in robot learning although it is a prominent feature in human learning. This project will deliver a completely new and better approach: robot learning will no longer rely on initial demonstrations only, but it will effectively use additional user feedback to continuously optimize the task performance. It will enable the user to directly perceive and correct undesirable behavior and to quickly guide the robot toward the target behavior. In my previous research I have made ground-breaking contributions to the existing learning paradigms and I am therefore ideally prepared to tackle the three-fold challenge of this project: developing theoretically sound techniques which are at the same time intuitive for the user and efficient for real-world applications.

The novel framework will be validated with generic real-world robotic force-interaction tasks related to handling and (dis)assembly. The potential of the newly developed teaching framework will be demonstrated with challenging bi-manual tasks and a final study evaluating how well novice human operators can teach novel tasks to a robot.

Project members

dr. Zlatan Ajanović, dr. Carlos E. Celemin Paez, dr. Marta Ferraz, ing. Giovanni Franzese, Dr.-Ing. Jens Kober, dr. Ravi Prakash, dr. Leandro de Souza Rosa,

Publications with videos


Giovanni Franzese, Leandro de Souza Rosa, Tim Verburg, Luka Peternel, and Jens Kober. Interactive Imitation Learning of Bimanual Movement Primitives. IEEE/ASME Transactions on Mechatronics, ():1–13, 2023. [bibtex] [doi] [code] [video] gold open access


Carlos E. Celemin and Jens Kober. Knowledge- and Ambiguity-Aware Robot Learning from Corrective and Evaluative Feedback. Neural Computing and Applications, 35(23):16821–16839, 2023. [bibtex] [pdf] [doi] [code] [video] gold open access


Jelle Luijkx, Zlatan Ajanović, Laura Ferranti, and Jens Kober. PARTNR: Pick and place Ambiguity Resolving by Trustworthy iNteractive leaRning. In NeurIPS 2022 - 5th Robot Learning Workshop: Trustworthy Robotics, 2022. [bibtex] [pdf] [url] [webpage] [video] bronze open access


Mariano Ramirez Montero, Giovanni Franzese, Jeroen Zwanepol, and Jens Kober. Solving Robot Assembly Tasks by Combining Interactive Teaching and Self-Exploration. arXiv:2209.11530 [cs.RO], 2022. [bibtex] [pdf] [doi] [code] [video] bronze open access


Linda van der Spaa, Giovanni Franzese, Jens Kober, and Michael Gienger. Disagreement-Aware Variable Impedance Control for Online Learning of Physical Human-Robot Cooperation Tasks. In ICRA 2022 full day workshop - Shared Autonomy in Physical Human-Robot Interaction: Adaptability and Trust, 2022. [bibtex] [pdf] [url] [code] [video] bronze open access


Anna Mészáros, Giovanni Franzese, and Jens Kober. Learning to Pick at Non-Zero-Velocity From Interactive Demonstrations. IEEE Robotics and Automation Letters, 7(3):6052–6059, 2022. [bibtex] [pdf] [url] [doi] [video] gold open access


Giovanni Franzese, Anna Mészáros, Luka Peternel, and Jens Kober. ILoSA: Interactive Learning of Stiffness and Attractors. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7778–7785, 2021. [bibtex] [pdf] [url] [doi] [code] [video] green open access


Snehal Jauhri, Carlos E. Celemin, and Jens Kober. Interactive Imitation Learning in State-Space. In 2020 Conference on Robot Learning (CoRL) (Jens Kober, Fabio Ramos, Claire Tomlin, eds.), PMLR, vol. 155 of Proceedings of Machine Learning Research, pp. 682–692, 2021. [bibtex] [pdf] [html] [code] [video] gold open access


Giovanni Franzese, Carlos E. Celemin, and Jens Kober. Learning Interactively to Resolve Ambiguity in Reference Frame Selection. In 2020 Conference on Robot Learning (CoRL) (Jens Kober, Fabio Ramos, Claire Tomlin, eds.), PMLR, vol. 155 of Proceedings of Machine Learning Research, pp. 1298–1311, 2021. [bibtex] [pdf] [html] [code] [video] gold open access


Rodrigo Pérez-Dattari, Carlos E. Celemin, Giovanni Franzese, Javier Ruiz-del-Solar, and Jens Kober. Interactive Learning of Temporal Features for Control: Shaping Policies and State Representations From Human Feedback. IEEE Robotics & Automation Magazine, 27(2):46–54, 2020. [bibtex] [pdf] [doi] [code] [video] green open access

Publications without videos

Runyu Ma, Jelle Luijkx, Zlatan Ajanović, and Jens Kober. ExploRLLM: Guiding Exploration in Reinforcement Learning with Large Language Models. arXiv:2403.09583 [cs.RO], 2024. [bibtex] [pdf] [webpage] [doi] bronze open access

Zlatan Ajanović, Bakir Lačević, and Jens Kober. Value Function Learning via Prolonged Backward Heuristic Search. In ICAPS 2023 Workshop: PRL Workshop Series – Bridging the Gap Between AI Planning and Reinforcement Learning, 2023. [bibtex] [pdf] [url] bronze open access

Yulei Qiu, Jihong Zhu, Cosimo Della Santina, Michael Gienger, and Jens Kober. Robotic Fabric Flattening with Wrinkle Direction Detection. In 18th International Symposium on Experimental Robotics (ISER), 2023. [bibtex] [pdf] [webpage] [video] green open access

Tomás Coleman, Giovanni Franzese, and Pablo Borja. Damping Design for Robot Manipulators. In Human-Friendly Robotics 2022 (Pablo Borja, Cosimo Della Santina, Luka Peternel, Elena Torta, eds.), Springer International Publishing, Cham, pp. 74–89, 2023. [bibtex] [file] [doi] green open access

Ravi Prakash and Laxmidhar Behera. Neural Optimal Control for Constrained Visual Servoing via Learning From Demonstration. IEEE Transactions on Automation Science and Engineering, :1–14, 2023. [bibtex] [doi] [video]

Jianyong Sun, Jihong Zhu, Jens Kober, and Michael Gienger. Learning from Few Demonstrations with Frame-Weighted Motion Generation. In 18th International Symposium on Experimental Robotics (ISER), 2023. [bibtex] [pdf] [video] green open access

Eveline Drijver, Rodrigo Pérez-Dattari, Jens Kober, Cosimo Della Santina, and Zlatan Ajanović. Robotic Packaging Optimization with Reinforcement Learning. In IEEE 19th International Conference on Automation Science and Engineering (CASE), pp. 1-7, 2023. [bibtex] [pdf] [doi] green open access

Cristian Meo, Giovanni Franzese, Corrado Pezzato, Max Spahn, and Pablo Lanillos. Adaptation Through Prediction: Multisensory Active Inference Torque Control. IEEE Transactions on Cognitive and Developmental Systems, 15(1):32–41, 2023. [bibtex] [file] [doi] green open access

Armin Avaei, Linda van der Spaa, Luka Peternel, and Jens Kober. An Incremental Inverse Reinforcement Learning Approach for Motion Planning with Separated Path and Velocity Preferences. Robotics, 12(2):61, 2023. [bibtex] [pdf] [doi] [video] gold open access

Zlatan Ajanović, Emina Aličković, Aida Branković, Sead Delalić, Eldar Kurtić, Salem Malikić, Adnan Mehonić, Hamza Merzić, Kenan Šehić, and Bahrudin Trbalić. Vision for Bosnia and Herzegovina in Artificial Intelligence Age: Global Trends, Potential Opportunities, Selected Use-cases and Realistic Goals \\ BOS: Vizija Bosne i Hercegovine u Doba Umjetne Inteligencije: Svjetski Trendovi, Mogućnosti, Odabrani Primjeri i Realistični Ciljevi. In Scientific-Professional Conference "Artificial Intelligence in Bosnia and Herzegovina"- research, application and development perspectives, Federalno ministarstvo obrazovanja i nauke/znanosti : Fondacija za inovacijski i tehnološki razvoj, pp. 13–46, 2022. [bibtex] [pdf] [url] green open access

Carlos Celemin, Rodrigo Pérez-Dattari, Eugenio Chisari, Giovanni Franzese, Leandro de Souza Rosa, Ravi Prakash, Zlatan Ajanović, Marta Ferraz, Abhinav Valada, and Jens Kober. Interactive Imitation Learning in Robotics: A Survey. Foundations and Trends® in Robotics, 10(1–2):1–197, 2022. [bibtex] [pdf] [url] [doi] green open access

Carlos E. Celemin and Jens Kober. Uncertainties Based Queries for Interactive Policy Learning with Evaluations and Corrections. In Companion Publication of the 2021 International Conference on Multimodal Interaction, pp. 192–193, 2021. [bibtex] [pdf] [doi] green open access

Bart Bootsma, Giovanni Franzese, and Jens Kober. Interactive Learning of Sensor Policy Fusion. In 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), pp. 665–670, 2021. [bibtex] [pdf] [doi] green open access

Jan Scholten, Daan Wout, Carlos E. Celemin, and Jens Kober. Deep Reinforcement Learning with Feedback-based Exploration. In IEEE Conference on Decision and Control (CDC), pp. 803–808, 2019. [bibtex] [pdf] [doi] [code] green open access

Daan Wout, Jan Scholten, Carlos E. Celemin, and Jens Kober. Learning Gaussian Policies from Corrective Human Feedback. arXiv:1903.05216 [cs.LG], 2019. [bibtex] [pdf] [doi] [code] bronze open access