Mobile Crawler: A Robot that learns by itself

Abstract

Reinforcement Learning (RL) is one machine learning algorithm that enables robots autonomously develop control laws and policies to achieve optimization goals by means of punishments and rewards. The aim of this work is to demonstrate the ability of a mobile robot to learn to perform a set of actions that allow it to move forward by itself and without explicit commands. For this purpose, we constructed a crawling robot that contains 2 degrees of freedom. The robot is controlled by Arduino-Uno microcontroller. The learning process is done by Q-learning, which is a type of RL, to make the robot perform a set of trial and error actions and learns to move forward. The robot is also equipped with an ultrasonic sensor that measures the distance between the robot and the wall and provides a reward value for the robot when as it moves forward. By allowing the robot to move its crawling arm randomly within a well-defined domain of actions and providing feedback to the Q-learning from the ultrasonic sensor, we noticed that the robot can learn the sequence of corrective actions. The experiments show that the learning process took from 5 to 10 minutes due to the random values in the Q-learning algorithm. We propose to use this project as a learning application in the artificial intelligence course because the students can observe the process of Q-learning in an interesting way.