Table 1 List of papers
Refs. | Aim | Task | Robot | Sensors used | Control strategy | Accuracy | Method |
|---|---|---|---|---|---|---|---|
Lui al. [25] | Manipulating deformable objects./The robot untangles various ropes | Cloth folding or untangling or coveraging | PR2 | Stereo camera pairs Mono-cameras pressure sensor arrays | SL | 76.7% for intersection graph inference and 89.2% for node inference | Max-margin learning |
Bersch et al. [24] | Transforming an item from a random crumpled configuration into a folded state./A PR2 robot folds a towel | Cloth folding or untangling or coveraging | PR2 robot | Stereo camera pairs Mono-cameras pressure sensor arrays | SL | 90% | SVM |
Corona et al. [28] | The approach of this paper uses a hierarchy of three CNNs./Grasp a garment using a robot | Cloth folding or untangling or coveraging | Wam robot | Kinect | SL | 96.85% | CNNs |
Yang et al. [23] | A machine-learning-based humanoid robot that can work as a production line worker and fold clothes./A dual arm robot folds a cloth | Cloth folding or untangling or coveraging | Nextage Open Robot | A camera image whose resolution is 112 × 112 × 3chs (37,632 dimensions, RGB) | SL | Trained Cloth Grabbed 88.9% Grabbed + Folded 88.9% Untrained Cloth Grabbed 77.8% Grabbed + Folded 66.7% Total Grabbed 83.3% Grabbed + Folded 77.8% | DL |
Hu et al. [26] | A general approach to automatically visual servo-control the position and shape of a deformable object/Manipulation of a garment by a dual arm robot | Cloth folding or untangling or coveraging | ABB robot | Two 3D cameras | SL | 70% for the peg-in-hole task and 90% for other tasks | Fast Online Gaussian Process Regression |
Tanaka et al. [14] | The paper presents a motion planning method for automatic operation of cloth products using a robot./The robot folds a garment | Cloth folding or untangling or coveraging | HIRO robot | Kinect | SL | – | NN |
Jia et al. [27] | An approach for manipulating high-DOF deformable objects using a random-forest-based controller./A dual-armed robot and a human is holding four corners of a cloth | Cloth folding or untangling or coveraging | ABB YuMi | Realsense camera | SL | – | Random-Forest-Based Imitation Learning |
Sannapaneni [29] | Teaching a manipulator using LfD technique to fold clothes./The ADAM robot folds a cloth | Cloth folding or untangling or coveraging | Amrita Dual Anthropo-morphic Manipulator | Logitech camera | LfD | – | Dynamic Movement Primitives (DMP), HMM |
Wu et al. [35] | A problem of deformable object manipulation through model-free visual RL./A PR2 manipulating clothes and ropes | Cloth folding or untangling or coveraging | PR2 | Kinect | RL | – | Used Maximal Value under Placing (MVP) to select the pick point that has the maximum value |
Balaguer et al. [30] | A learning algorithm that combines imitation and RL to perform towel folding tasks./A bimanual manipulator folds towels | Cloth folding or untangling or coveraging | Robotic manipulator | Motion capture system | RL | – | Control policy PoWER |
Yaqiang et al. [33] | Describing folding behaviour acquisition of a shirt by a dual-arm robot./The dual arm robot folds a t- shirt | Cloth folding or untangling or coveraging | Baxter | Kinect | RL | – | PILCO algorithm |
Balaguer et al. [30] | A learning algorithm is proposed that combines imitation and RL./A dual arm robot folds a garment | Cloth folding or untangling or coveraging | Barrett Arms | Kinect | RL | 81.66% | |
Koganti et al. (2015) | The authors propose the offline learning of a cloth dynamics model./A dual arm robot puts a shirt on a mannequin | Putting a cloth on user’s arm | WAM | RealSense camera | SL | – | Gaussian Process Latent Variable Model |
Zhang et al. (2017) | This paper uses a hierarchical multi-task control strategy to automatically adapt the robot motion and minimize the force applied between the user and the robot caused by user movements. Moreover, there is the online update of the dressing trajectory based on the user movement limitations modelled with the Gaussian Process Latent Variable Model in a latent space, and the density information extracted from such latent space./A dual arm robot puts on a shirt on a person’s arm | Putting a cloth on user’s arm | Baxter | Kinect | SL | – | Gaussian Process Latent Variable Model in a latent space |
Chance et al. [37] | A robot was used to dress a jacket onto a mannequin and human participants considering several combinations of user pose and clothing type, while recording dynamic data from the robot, a load cell, and an IMU./Putting a jacket on a mannequin | Putting a cloth on user’s arm | Baxter | Kinect | SL | – | SVM |
Stria et al. [38] | A classification of garment categories and a focus particularly on garments being held in a hanging state by a robotic arm./The dual arm puts a shirt on the person’s arm | Putting a cloth on user’s arm | – | ASUS Xtion | SL | 82% | SVM |
Erickson et al. [39] | A multidimensional capacitive sensing technique that estimates the local pose of a human limb in real time./A PR2 robot pulls a hospital gown onto a participant’s arm | Putting a cloth on user’s arm | PR2 | Stereo camera pairs Mono-cameras pressure sensor arrays | SL | – | Fully CNN |
Gao et al. [40] | An end-to-end approach for home-environment assistive humanoid robots to provide personalized assistance through a dressing/A Baxter robot assists two users to wear a sleeveless jacket | Putting a cloth on user’s arm | Baxter | Kinect | SL | – | Unsupervised expectation-minimization (EM) algorithm is used to learn Gaussian mixture models (GMMs) |
Kapusta et al. [43] | A data-driven haptic perception can be used to infer relationships between clothing and the human body during robot-assisted dressing./The robot puts a hospital gown on the patient's arm | Putting a cloth on user’s arm | Festo linear actuator driven by an Animatics Smart Motor | Force-torque sensor | LfD | – | Hidden Markov models (HMM) |
Pignat et al. [41] | A programming by demonstration method to efficiently learn how to dress a person./The dual arm robot puts a jacket on a person’ s arm | Putting a cloth on user’s arm | Baxter Robot | Kinect Sensor motors | LfD | – | Hidden semi-Markov model (HSMM) |
Clegg et al. (2018) | Incorporate cloth simulation in the deep RL framework to learn a robust dressing control policy./A simulation of a virtual human wearing a hospital gown | Putting a cloth on user’s arm | – | Haptic sensors | Deep RL | – | Trust Region Policy Optimization (TRPO) |
Clegg et al. [71] | The application of haptic aware feedback control and deep RL to robot assisted dressing in simulation./A simulation of putting a hospital gown on a person’s arm | Putting a cloth on user’s arm | Simulated Kuka | Haptic, capacitive, force/torque sensor | RL | 100% success rate of the right arm, but a 96% success rate of the left arm | Markov Decision Process (POMDP) |
Erickson et al. [39] | A deep recurrent model that, when given a proposed action by the robot, predicts the forces a garment will apply to a person’s body./A PR2 pulls a hospital gown onto a participant’s arm | Putting a cloth on user’s arm | PR2 | Haptic, force/torque sensor Kinect | Other methods | – | Deep Haptic Model Predictive Control |
Chance et al. [47] | A dressing task using a compliant robotic arm on a mannequin./Getting the arm into the sleeve of the jacket | Putting a cloth on user’s arm | Baxter | Force, wireless accelerometer, gyroscope and joint torque sensors | Other methods | – | No learning methods |
Koganti et al. [49] | Aa data-efficient representation to encode task-specific motor-skills of the robot using Bayesian nonparametric latent variable models./The dual arm puts a shirt on the user | Putting a cloth on user’s head | Baxter | Kinect | SL | – | Gaussian Process Latent Variable Model |
Saxena et al. [50] | A Deep Learning framework for garment recognition and grasping point detection./Putting a t-shirt on a person | Putting a cloth on user’s head | Baxter | Kinect | SL | Sleeveless T-shirt 96% and 100% of the times in Single View and Multi-view, Full-Sleeved T-shirt 90% and 94% of the times in Single and Multi-View. In the case of Unseen Clothes, Sleeveless T-shirt 76% and 96% of the times in Single and Multi-View, Full-Sleeved T-shirt 40% and 72% of the time in Single and Multi-View | DL |
Joshi et al. (2019) | A framework for robotic clothing assistance by imitation learning from a human demonstration to a compliant dual-arm robot./The dual arm robot puts a shirt on the user's head | Putting a cloth on user’s head | Baxter | Kinect | LfD | – | – |
Umali et al. (2017) | The authors segment the doffing procedure into a sequence of human–robot actions such that the robot only assists when necessary and the human performs the more intricate parts of the procedure./Robot-Assisted doffing using transfer motions | Putting a cloth on user’s head | Baxter | Two webcams Audio-visual camera Kinect | LfD | – | TrajOpt method |
Matsubara et al. (2013) | A novel RL framework for learning motor skills that interacts with non-rigid materials./The robot manipulates a shirt | Putting a cloth on user’s head | WAM | Two stereo cameras | RL | 90% | Reward function design with topology coordinates |
Koganti et al. [52] | A novel method for the real-time estimation of Human-Cloth relationship./The robot puts on a shirt on a mannequin head | Putting a cloth on user’s head | WAM robot | A Senz3D time of flight (ToF) sensor | RL | – | – |
Twardon et al. [53] | The authors consider a robot that learns to put a knit cap on a styrofoam head/A dual arm robot puts a cap on a mannequin’s head | Putting a cloth on user’s head | Mitsubishi PA-10 arms | Kinect | RL | – | Direct policy with gradient free policy |
Tamei et al. [54] | This study uses RL to perform the task of clothing assistance where the robot learns to put a mannequin head in a shirt./Putting a shirt into a mannequin | Putting a cloth on user’s head | WAM robot | A Senz3D time of flight (ToF) sensor | RL | – | Policy gradient approach |
Shinohara et al. [56] | A novel learning framework for learning motor skills interacting with non-rigid materials by RL./Robot is putting a shirt on user heads | Putting a cloth on user’s head | WAM robot | A Senz3D time of flight (ToF) sensor | RL | 90% | – |
Klee et al. [57] | An approach for a robot to provide personalized assistance for dressing a user./The manipulator successfully puts a hat on the user | Putting a cloth on user’s head | Baxter robot | Kinect | Other methods | – | Sampling-Based Motion Planning |
Canal et al. [58] | A method to perform behaviour adaptation to the user preferences, using symbolic task planning./A manipulator puts a shoe on user’s feet | Putting a cloth on user’s feet | WAM Arm | Kinect | LfD | – | MDP |
Yamazaki et al. [59] | An autonomous robot’s method of dressing a subject in clothing with the target task of dressing a person in the sitting pose./The humanoid robot puts trousers on the user | Putting a cloth on user’s leg | Humanoid robot | Xtion sensor, tactile and force sensors | Other methods | 73% | – |
Lee et al. [5] | A method for learning force-based manipulation skills from demonstrations./The robot ties a knot, folds a towel, erases a whiteboard, and ties a rope to a pipe | Multiple tasks | PR2 robot | Stereo camera Mono-camera pressure sensor arrays | LfD | – | Dynamic Movement Primitives (DMP), HMM, Point cloud |
Tsurumine et al. [60] | Deep learning method applied to two real robotic manipulation tasks./The robot has to flip a handkerchief and fold a t-shirt | Multiple tasks | NEXTAGE | Two cameras on the head of the robot | RL | 80% | Two Deep RL algorithms: Deep P-Network (DPN) and Dueling Deep P-Network (DDPN) |
Matas et al. [63] | Aa combination of state-of-the-art of deep RL algorithms to solve the problem of manipulating deformable objects./The robot folds a towel up to a mark, folds a face towel diagonally, and drapes a piece of cloth over a hanger | Multiple tasks | 7 DOF Kinova | Genius C170 web camera | RL | Diagonal folding 90% Hanging 77% Tape 86% | Deep deterministic policy gradient (DDPG) algorithm |