3D Vision
Second semester 2020
Andrés Marrugo, PhD
Universidad Tecnológica de Bolívar
Aims and Scope
This course covers the main techniques of 3D data acquisition, both passive (stereoscopic vision and multiple views) and active (active triangulation by structured light). The camera calibration process and the use of the geometric constraints of stereo systems are studied in detail in order to obtain adequate correspondences. Finally, the post-processing stages of three-dimensional vision systems are discussed: the representation of 3D images (e.g. through meshes, depth maps etc.), the extraction of characteristics and the recognition of 3D objects.
In this course the student will obtain proficiency in the fundamentals of stereo vision systems, the calibration of cameras, the geometric restrictions of stereoscopic systems (epipolar line, fundamental matrix, etc.) and carrying out the software implementation in simple applications. In addition, the student will learn the main active techniques of 3D image acquisition (active triangulation by structured light, and the basics of kinect, among others). Finally, different forms of representation of 3D images and their use in problems of computer vision will be presented.
Prior knowledge of this course includes probability, linear algebra, and calculus. Programming experience in MATLAB is desirable, but not required.
Useful Resources
Tutorials, review materials
- MATLAB tutorial
- More MATLAB tutorials: basic operations, programming, working with images
- Linear algebra review
- Random variables review
MATLAB reference
Outline
This is a new course, this website will be updated as we go along.
Lecture 1: Introduction and Linear Algebra Resfresher
We will be discussing the main aspects and motivation for 3D computer vision. Also a brief overview of the Linear Algebra involved in the course.
Assignment 1
A summary of Linear Algebra and Probability exercises. Due date: 2020-08-21.
- Send by email.
Lecture 2: Sensors and Image Formation
We will be studying the main aspects about image formation, the pinhole camera model and image sensors.
Reading
Assignment 2
Matlab basics and image formation. Due date: 2020-08-28.
- Assignment 2
- Send by email.
Lecture 3: 2D-2D Coordinate transforms
In this lecture we introduce 2D coordinate transforms.
Lecture 4: 3D-3D Coordinate transforms
In this lecture we introduce 3D coordinate transforms.
Code
Reading
Quiz
TBD Quiz on Coordinate transforms.
Assignment 3
Coordinate transforms. Due date: TBD.
Lecture 5: 3D-2D Coordinate Transforms
In this lecture we recall perspective projection, this time from a matrix multiplication perspective.
Lecture 6: Image Filtering
Linear filters, correlation, convolution kernel, smoothing and edge detection via the gradient.
Reading
Lecture 7: Binary Images
Manipulating Binary or logical images.
Assignment 4
Coordinate transforms. Due date: TBD.
- Assignment 4
- Send by email.
Lecture 7: 2D Image transforms
2D Image transformations and least squares fitting.
Lecture 8: Alignment using Nonlinear Least Squares
Alignment using Newton’s method.
Lecture 9: Pose estimation
Reading
Assignment 5
Correspondence problem. Due date: TBD.
- robot1.jpg
- robot2.jpg
- Assignment 5
- Send by email.
Lecture 11: Corner detection
We now describe how to detect features or salient points from an image.
Reading
Lecture 12: SIFT
In this lecture we will discuss Scale-Invariant Keypoints.
Reading
- Read “Distinctive Image Features from Scale-Invariant Keypoints” by D. Lowe.
Lecture 13: Linear Pose Estimation
In this lecture we will discuss the direct estimation of a camera pose using matrix algebra.
Reading
Projects
In this course you are required to complete a short project, similar to the assignments, but you are free to choose the approach and the implementation. You will work in teams of two and you will deliver a project report in the IEEE paper format and a 15 minute presentation.
The project is due December 15th.
Assignment 6
Correspondence problem. Due date: TBD.
Lecture 14: Stereo
In this lecture we will discuss stereo vision.
- Lecture 14 slides
- pentagonLeft.png
- pentagonRight.png
- left.png
- right.png
- stereo_ball.m
- Lecture 13 examples
Lecture 15: Stereo
In this lecture we will discuss stereo vision.
Lecture 16: Structure from Motion
In this lecture we will discuss structure from motion.
Lecture 17: Camera calibration
In this lecture we will discuss how to calculate the camera intrinsic parameters.
Reading
Lecture 18: Structured Light
In this lecture we will discuss structured light systems.
Final exam
The exam is 3 hours and you may use any resources, (MATLAB, Books, slides, etc.). However, provide full answers. These may be done by hand and scanned. Please make everything as clear as possible.