Human multi-robot workspace sharing

In this work a solution to human multi-robot safe coexistence is presented in which multiple mobile manipulators are in charge of performing a cooperative task in a workspace shared with human operators. The safety of the interaction is assessed by a safety field that takes into account the whole system and is general enough concerning its expression. Based on the value of this field, the cooperative task trajectory is properly modified so as to ensure a safe interaction while trying to preserve as much as possible the nominal task, which is instead completely aborted whenever the safety of the interaction cannot be guaranteed. The solution is first designed within a centralized architecture and, then, upon this, a distributed implementation is presented which, in general, aims to exhibit the same performance as the centralized counterpart. Finally, both simulations and experiments on two Comau SmartSix robots corroborate the designed solution.

Contact classification and reaction in human-robot physical interaction

In this scenario a robot is able to both carry out autonomous tasks and to physically interact with a human operator to achieve a common objective. Since human and robot share the same workspace both accidental and intentional contacts between them might arise. Therefore, a solution based on Recurrent Neural Networks (RNNs) is proposed to detect and classify the nature of the contact with the human, even in the case the robot is interacting with the environment because of its own task. Then, reaction strategies are defined depending on the nature of contact: human avoid- ance with evasive action in the case of accidental interaction, and admittance control in the case of intentional interaction. In regard to the latter, Control Barrier Functions (CBFs) are considered to guarantee the satisfaction of robot constraints, while endowing the robot with a compatible compliant behavior.

Human multi-robot physical interaction

The objective is to devise a general framework to allow a human operator to physically interact with an object manipulated by a multi-manipulator system in a distributed setting. A two layer solution is devised. At the top layer an optimal Linear Quadratic Tracking problem is solved where both the human and robots' intentions are taken into account, being the former online estimated by Recursive Least Squares (RLS) technique. The output of this layer is a desired trajectory of the object which is the input of the bottom layer and from which desired trajectories for the robot end effectors are computed based on the closed-chain constraints. Each robot, then, implements a time-varying gain adaptive control law so as to take into account model uncertainty and internal wrenches. Simulations with three 6-DOFs serial chain manipulators mounted on mobile platforms corroborate the theoretical findings.

Visual action planning for rigid and deformable object manipulation

In this video, we present a framework for visual action planning of complex manipulation tasks with high-dimensional state-spaces, such as manipulation of deformable objects. Planning is performed in a low-dimensional latent state space that embeds images. We define and implement a Latent Space Roadmap (LSR) which is a graph-based structure that globally captures the latent system dynamics. Our framework consists of two main components: a Visual Foresight Module (VFM) that generates a visual plan as a sequence of images, and an Action Proposal Network (APN) that predicts the actions between them. We show the effectiveness of the method on a simulated box stacking taskas well as a T-shirt folding task performed with a Baxter robot.

Benchmarking bimanual cloth manipulation

Cloth manipulation is a challenging task that, despite its importance, has received relatively little attention compared to rigid object manipulation. In this letter, we provide three benchmarks for evaluation and comparison of different approaches towards three basic tasks in cloth manipulation: spreading a tablecloth over a table, folding a towel, and dressing. The tasks can be executed on any bimanual robotic platform and the objects involved in the tasks are standardized and easy to acquire. We provide several complexity levels for each task, and describe the quality measures to evaluate task execution. Furthermore, we provide baseline solutions for all the tasks and evaluate them according to the proposed metrics.

Distributed fault detection and isolation

The problem of defining a Distributed Fault Detection and Isolation strategy for a team of mobile manipulators performing a cooperative mission is addressed. The overall system relies on an observer-controller scheme, where each robot estimates the global state of the team through a distributed observer requiring information exchange with the direct neighbor robots; then, the global state estimate is used by each robot to compute the estimated local input so as to achieve a specific task. The observer-controller scheme also allows to define a set of residual vectors that can be used by the robots to detect and isolate faults affecting anyone of the teammates, even if not in direct communication, without increasing the computational burden and the information exchange. The approach is validated with a heterogeneous team of three robots perform a cooperative task.