3D Vision

Second semester 2020

Andrés Marrugo, PhD
Universidad Tecnológica de Bolívar

Aims and Scope

This course covers the main techniques of 3D data acquisition, both passive (stereoscopic vision and multiple views) and active (active triangulation by structured light). The camera calibration process and the use of the geometric constraints of stereo systems are studied in detail in order to obtain adequate correspondences. Finally, the post-processing stages of three-dimensional vision systems are discussed: the representation of 3D images (e.g. through meshes, depth maps etc.), the extraction of characteristics and the recognition of 3D objects.

In this course the student will obtain proficiency in the fundamentals of stereo vision systems, the calibration of cameras, the geometric restrictions of stereoscopic systems (epipolar line, fundamental matrix, etc.) and carrying out the software implementation in simple applications. In addition, the student will learn the main active techniques of 3D image acquisition (active triangulation by structured light, and the basics of kinect, among others). Finally, different forms of representation of 3D images and their use in problems of computer vision will be presented.

Prior knowledge of this course includes probability, linear algebra, and calculus. Programming experience in MATLAB is desirable, but not required.

Useful Resources

Tutorials, review materials

MATLAB reference

Outline

This is a new course, this website will be updated as we go along.

Lecture 1: Introduction and Linear Algebra Resfresher

We will be discussing the main aspects and motivation for 3D computer vision. Also a brief overview of the Linear Algebra involved in the course.

Assignment 1

A summary of Linear Algebra and Probability exercises. Due date: 2020-08-21.

Assignment 1

Lecture 2: Sensors and Image Formation

We will be studying the main aspects about image formation, the pinhole camera model and image sensors.

Reading

Assignment 2

Matlab basics and image formation. Due date: 2020-08-28.

Lecture 3: 2D-2D Coordinate transforms

In this lecture we introduce 2D coordinate transforms.

Lecture 4: 3D-3D Coordinate transforms

In this lecture we introduce 3D coordinate transforms.

Code

Reading

Quiz

TBD Quiz on Coordinate transforms.

Assignment 3

Coordinate transforms. Due date: TBD.

Lecture 5: 3D-2D Coordinate Transforms

In this lecture we recall perspective projection, this time from a matrix multiplication perspective.

Lecture 6: Image Filtering

Linear filters, correlation, convolution kernel, smoothing and edge detection via the gradient.

Reading

Lecture 7: Binary Images

Manipulating Binary or logical images.

Assignment 4

Coordinate transforms. Due date: TBD.

Lecture 7: 2D Image transforms

2D Image transformations and least squares fitting.

Lecture 8: Alignment using Nonlinear Least Squares

Alignment using Newton’s method.

Lecture 9: Pose estimation

Reading

Assignment 5

Correspondence problem. Due date: TBD.

Lecture 11: Corner detection

We now describe how to detect features or salient points from an image.

Reading

Lecture 12: SIFT

In this lecture we will discuss Scale-Invariant Keypoints.

Reading

Lecture 13: Linear Pose Estimation

In this lecture we will discuss the direct estimation of a camera pose using matrix algebra.

Reading

Projects

In this course you are required to complete a short project, similar to the assignments, but you are free to choose the approach and the implementation. You will work in teams of two and you will deliver a project report in the IEEE paper format and a 15 minute presentation.

The project is due December 15th.

Assignment 6

Correspondence problem. Due date: TBD.

Lecture 14: Stereo

In this lecture we will discuss stereo vision.

Lecture 15: Stereo

In this lecture we will discuss stereo vision.

Lecture 16: Structure from Motion

In this lecture we will discuss structure from motion.

Lecture 17: Camera calibration

In this lecture we will discuss how to calculate the camera intrinsic parameters.

Reading

Lecture 18: Structured Light

In this lecture we will discuss structured light systems.

Final exam

The exam is 3 hours and you may use any resources, (MATLAB, Books, slides, etc.). However, provide full answers. These may be done by hand and scanned. Please make everything as clear as possible.