# Robotics, Vision And Control Fundamental Algorithms In MATLAB 21

AimThe aim of this course is to understand how to build robot systems that can sense their environment to then perform an action. To do this, robots must have the ability to receive information from their environment but also an understanding of themselves within it. Importantly, how can we capture this information greatly influences our subsequent actions both in terms of the availability of actionable data but also the processing required to interpret. By exploring these trade-offs we will explore in depth the sensory systems used to capture information and in turn develop models that enable robots to cooperate with its environment in different ways. In this course, we will discuss a range of sensing and cognitive control strategies. This theory will then be used to design robotic systems to perform manipulation tasks to cope with unstructured environments such as localizing objects for grasping.Learning Outcomes Understand the main concepts related to robotic manipulation and sensing.

Develop methods for tackling uncertainty in robotic manipulation systems.

Read scientific literature in robotics to choose approaches for a particular problem.

Implement state-of-the-art algorithms on simulated manipulators and sensors.

Theory Prerequisites A working knowledge of linear algebra: a linear algebra refresher (Khan Academy lecture) are Vector intro for linear algebra

Introduction to the matrix

Multiplying matrices

Introduction to the identity matrix

How to transform vectors using a transformation matrix

Introduction to eigenvalues and eigenvectors

Topics Representing Poses and Kinematics for Robot Manipulation

Visual & other Exteroceptive Sensing

Visual pose estimation under uncertainty

Force/Torque & other Proprioceptive Sensing

Sensing-based Grasping

Pick-and-Place Methods

Navigation Among Movable Objects (NAMO)

Human-Robot Interaction and Collaboration

Reinforcement Learning for Grasping

Syllabus Structure One 2-hours lecture will discuss background theory.

One 1-hour lecture for research reading and presentation from the students.

One 1-hour lab session will focus on simulated experiments on manipulators and sensors for grasping purposes.

Coding Prerequisites Previous programming experience: C++, ROS, PCL.

Motivation to work hard.

An introductory course on ROS from ETH can be found here.

Textbooks Lecture 1-2: P. Corke, "Robotics, Vision and Control: Fundamental Algorithms in Matlab, 2nd ed", Springer Tracts in Advanced Robotics, 2017.

Lecture 1-2: J. Craig, "Introduction to Robotics: Mechanics and Control", Global Edition, 3rd Edition, Pearson.

Lecture 5-6:I. Goodfellow, Y. Bengio, and A. Courville, "Deep Learning", The MIT Press.

Lecture 5-6:G. Strang, "Linear Algebra and Learning from Data", Wellesley-Cambridge Press.

Lecture 9:R. Murphy, "Introduction to AI Robotics", Second Edition, The MIT Press.

Lecture 10:R. Sutton and A. Barto, "Reinforcement Learning", Second Edition, The MIT Press.

Supplementary (Kalman Fusion):S. Thrun, W. Burgard, and D. Fox, "Probabilistic Robotics", The MIT Press.

Supplementary (Data Structures):Cormen, Leiserson, Rivest, Stein, "Introduction to Algorithms", The MIT Press.

Supplementary (Perception):Siegwart, Nourbakhsh, Scaramuzza, "Introduction to Autonomous Mobile Robots", Second Edition, The MIT Press.

Supplementary (Robot Walking):Uchida, Delp, "Biomechanics of Movement, The Science of Sports, Robotics, and Rehabilitation", The MIT Press.

AnnouncementsThe Moodle page for the course: here.

## Robotics, Vision and Control Fundamental Algorithms in MATLAB 21

Abstract:Today, computer vision algorithms are very important for different fields and applications, such as closed-circuit television security, health status monitoring, and recognizing a specific person or object and robotics. Regarding this topic, the present paper deals with a recent review of the literature on computer vision algorithms (recognition and tracking of faces, bodies, and objects) oriented towards socially assistive robot applications. The performance, frames per second (FPS) processing speed, and hardware implemented to run the algorithms are highlighted by comparing the available solutions. Moreover, this paper provides general information for researchers interested in knowing which vision algorithms are available, enabling them to select the one that is most suitable to include in their robotic system applications.Keywords: trustworthy HRI; robot artificial cognition; HRIs in real-world settings