Reinforcement Learning for Dynamical Systems

Abstract: In this project, we survey a variety of reinforcement learning (RL) techniques and study them in the context of dynamical systems; specifically, we address the cart-pole problem. In general, RL algorithms achieve their goal by observing a system’s state and executing an action based on the expected reward given by the environment. We perform a review of RL literature highlighting existing approaches. For brevity, we limit our survey to literature relevant to dynamical systems. Furthermore, we implement a series of RL algorithms using OpenAI’s Gym [1] and Pytorch[2]. Specifically, we implement 1) a basic algorithm countering pole motion, 2) policy gradient utilizing a two-layer neural network, 3) deep Q-learning backed by a three-layer neural network, and 4) model predictive control (MPC). To conclude, we compare the individual performance of the RL techniques against the results of the MPC.

This project was the final project to Georgia Tech’s Statistical Machine Learning course (ECE6254). The final .pdf write-up of the project can be read here.