Lecturer: Dr. Subramanian Ramamoorthy

Teaching Assistant : Emmanuel Kahembwe | Demonstrator : Yordan Hristov

Lecture time and locations

11:10-12pm

Tuesdays: Room LG.08, David Hume Tower

Fridays: Room 5.3, Lister Learning and Teaching Centre

Course Details

Course Specification | Course Page | Slide Notes | Course Work

Course Summary

This course is intended as a specialized course on models and techniques for decision making in autonomous robots that must function in rich interactive settings involving interactions with a dynamic environment, and other agents (e.g., people). In the first part of the course, students will learn about formal models of decision making, and computational methods for automating these decisions within robots. In the second part of the course, we will consider issues arising in practical deployments of such autonomous robots, including problems of achieving safety, explainability and trust. Students will be exposed to current thinking on models and algorithmic methods for achieving these attributes in autonomous robots.

The content of this course has connections to other courses within our existing curriculum, such as Reinforcement Learning and Algorithmic Game Theory. A noteworthy difference is that RL and AGTA are primarily focussed on broad coverage of algorithmic methods, whereas this course will emphasize issues of modelling, with some focus on problems arising in practical robotics applications.

Course Description

The course will cover the following major themes, although specific topics could vary from year to year.

• Motivation
  • Problems involving interaction: Strategically rich human-robot interaction; Multi-robot interactions
  • How have decisions been modelled in different disciplines: probability theory, machine learning, psychology and cognitive science
• Mathematics of decisions
  • The utility maximization framework, Bayesian choice models
  • Causality, Causal learning
  • Bandit problems, Markov Decision Processes, and associated analysis methods
  • Dynamic programming principle, and associated approximation and learning algorithms
  • Incomplete information, Game theoretic models and solution concepts
• Computer science of decisions
  • Representations for planning – tradeoffs in modelling hierarchy, uncertainty, etc.
  • Safety and trust in autonomous systems
  • Explainability in AI
  • Bounded rationality and cognitive biases

Relevant QAA Computing Curriculum Sections: Artificial Intelligence, Intelligent Information Systems Technologies

Prerequisites

This is a second course at the postgraduate level.

This course is open to all Informatics students including those on joint degrees. However, students will benefit from prior exposure to robotics at the level of the Robotics: Science and Systems or Intelligent Autonomous Robotics.

The recommended requirements are as follows:

• Proficiency in Python, familiarity with C/C++
  • Class assignments will be in Python 3.6 (and will use numpy, gym and jupyter notebooks)
• Mathematics:
  • Multivariate Calculus
  • Linear Algebra
  • Probability & Stochastic Processes
  • Differential Equations
  • Principles of Optimization (linear programming, gradient descent)

Reading List

There is no single textbook for this course.

The instructor will provide lecture notes/slides, which will be complemented by readings from books and research articles.

Readings indicative of the course content include:

• B. Christian, T. Griffiths, Algorithms to Live By, William Collins Press, 2016.
• W.B. Powell, Approximate Dynamic Programming, Wiley, 2011.
• I. Gilboa, Theory of Decision under Uncertainty, Cambridge University Press, 2009.
• R.D. Luce, H. Raiffa, Games and Decisions, Dover Publications, 1989.
• M. Mitzenmacher, E. Upfal, Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis, Cambridge University Press, 2017.