Currently, many shipyards and industrial sites are using cranes. However, if an accident occurs due to a crane collision, it will lead to large property damage and personal injury. Therefore, this study proposed a new crane collision avoidance system. It is to prevent collisions between cranes by predicting 6-DOF pose of the crane through RGB image data and deep learning model.
In this study, synthetic images generation and automatic annotation using 3D CAD model and an algorithm of the crane pose estimation are proposed. It is very difficult to get the RGB image of the crane in the industrial site, but it is replaced with a synthetic image generated by rendering using a 3D CAD model. However, when using synthetic images, there is a domain gap between train data domain and test data domain. To reduce this gap, domain randomization is applied to the train data.
Also, when training a deep learning model, there is a difficulty in that a human has to do 6-DOF pose annotation. To solve this problem, the 6-DOF pose annotation is set to be automated through image processing and various functions of Blender, a rendering program.
After training, it is necessary to verify through an test, but it is very difficult to get real crane RGB images and it is impossible to annotate the 6-DOF pose of a real crane. To solve this problem, a Crane miniature is selected as a test object. And to annotate the crane pose, Arco markers are placed under the crane miniature, and the pose of the crane miniature is annotated using the predicted marker pose.
In this study, crane pose estimation algorithm has two architectures. The first architecture predicts the UV mask using UNet and UV mask is applied to the PnP and RANSAC algorithms to obtain the 6-DOF pose of the crane. In the second architecture, the bounding box of the crane is obtained from YOLO-v3, and the cropped image is input to UNet. Also, the predicted UV mask is applied to the PnP and RANSAC algorithms to obtain the 6-DOF of the crane. The results show better accuracy in the second architecture.