A fast selective grasping algorithm with deep learning and autonomous dataset creation on point cloud

Abstract

Grasping objects presents a multifaceted challenge influenced by variations in shape, perspective, material, roughness, and environmental conditions, making it a complex task. This research introduces an algorithm designed to address these challenges by employing point clouds since they allow a better notion of depth and geometry than RGB images. The proposed algorithm leverages geometric primitive estimation and lateral curvatures to identify optimal grasping regions swiftly and efficiently, where only the object geometry is used to analyze where to grasp.

To ensure the selection of desirable objects within the environment and to avoid undesirable ones, a purpose-built neural network, Point Encode Convolution (PEC), is introduced. PEC is tailored to utilize point clouds from RGB-D sensors and offers rapid execution and training times. The proposed design allows for efficient training and retraining, making it adaptable to diverse sets of objects. To expedite the training process, an autonomous dataset generation method is proposed. This method eliminates the need for manual annotation by autonomously generating data, and training is conducted within a simulation environment, such as Isaac Sim or any other simulation that allows object manipulation through scripts.

Validation of both algorithms, individually and in tandem, is conducted through the implementation of two grasp systems. The first system integrates the grasping algorithm with a neural network capable of object detection and 6D pose estimation. Initial validation occurs within the Webots and Gazebo simulations, where Gazebo was used for the visual validation and Webots for grasp validation due to its better physics handling without needing external plugins. However, due to certain limitations, the network component is excluded from subsequent experimental validation. The second system features the grasping algorithm with the neural network to be used on selective object grasping tasks.

Experimental validation is carried out using a UR5 robotic manipulator, an Intel RealSense D435 visual sensor, and a Robotiq 2F-140 gripper. The proposed neural network achieves a classification accuracy of 92.24% on a publicly available dataset. Meanwhile, the grasping algorithm attains an average success rate of 94% across all tested objects. The execution time of both algorithms is around 0.002 seconds each.