Kinect based Teleoperation of a Humanoid Robot

Abstract

Teleoperation of robots has become increasingly important in a wide range of industries and applications, from manufacturing and logistics to medicine and disaster response. The ability to remotely control robots to perform complex tasks can significantly improve efficiency and safety, particularly in hazardous or hard-to-reach environments. The goal of this project is to develop a teleoperation system that can accurately replicate human movements using a Kinect sensor and a Nao robot. The system utilizes inverse kinematics to enable the robot to mimic human movements. Specifically, the system will leverage the Microsoft Kinect V2 sensor to capture 3D skeletal joint data in real time, which will be processed and used to control the movements of the Nao robot. Choosing Kinect v2 over image processing to extract human skeleton offers the benefits of dedicated depth sensing, accuracy, and ease of integration. We investigated two approaches to translate the human poses into the corresponding command on the robot joint. The first one is based on Deep Learnings and the second one is based on inverse kinematics. In the implementation, we decided to proceed with the second approach due to the time required to complete the graduation project. The Inverse Kinematics approach works on a chain level such that each Kinect skeleton chain is converted into Nao body chain. We work on 3 main chains in this project; head, arm and leg. As the result, we were able to make the Nao robot imitate human in many poses and situations. Success was greater in individual chain movements than in complex movements and poses just as walking, mermaid and seated, with small errors in terms of centimeters