NOTE: Course information changes frequently, including Methods of Instruction. Please revisit these pages periodically for the most recent and up-to-date course information.

 Most existing reinforcement learning (RL) research is in the framework of discrete-time Markov Decision Processes (MDPs). Many real world applications, however, call for RL in continuous time with possibly continuous state and action spaces, such as high frequency trading and autonomous driving. Moreover, when cast in continuous time/spaces, it is possible to provide a theoretical and interpretable foundation for RL heuristics due to the availability of many technical tools such as stochastic analysis, stochastic control and differential equations.

This PhD reading course will center around reinforcement learning in continuous time/spaces and applications especially to financial engineering.  Students will take turns to present research papers, either important ones in the literature or their own papers, on topics including but not limited to exploration via randomization, entropy regularization, Boltzmann exploration, policy evaluation, policy gradient, q-learning, Langevin diffusions and application to nonconvex optimization, and mean-variance portfolio selection.  The objective is to stimulate interest in this emerging, largely unexplored area, to motivate new problems, and to inspire innovative approaches to solve research problems.

The course is mainly for PhD students in IEOR, computer science, mathematics, statistics and business school, who have taken courses in stochastic analysis, and are familiar with optimization and differential equations. Exceptional MS students with similar training may also take the course. The grading is based on the performance in the class including presentation and participation.