Roland Memisevic
Goethe University Frankfurt
Higher-Order Feature Learning:
Building a Computer Vision "Swiss Army Knife"
In many vision tasks, good performance is all about the right representation. Learning of image features (AKA Sparse Coding or Dictionary Learning) has therefore become a standard approach to many recognition, de-noising and other vision tasks.
While standard feature learning works well on static images, most interesting tasks go beyond these: Problems like video and motion understanding, stereo vision, invariant recognition, etc. do not come in the form of unordered, static images. Instead, it is the relationship between images that carries the relevant information.
Recently, Higher-order Sparse Coding models have emerged to address this issue, and many of these models are currently the best performing methods in tasks involving videos, stereo data, or image pairs. Many of the models were introduced independently and for various different tasks, but they are all based on the same core idea: Sparse codes can act like "gates", that modulate the connections between the other variables in a model. This allows them to represent changes in images and it turns model parameters into "stereo", "mapping" or "spatio-temporal" features.
The tutorial will show how Higher Order Features allow us to learn to "relate" images. It will discuss efficient learning and inference methods and it will present a tour of recent applications. The tutorial will also discuss some connections of these models to biological models of simple and complex cells and to multi-layer and recent deep learning methods.
Content:
- Introduction
- Sparse Coding, Feature Learning and Natural Images
- Learning how Images Change
- Examples From Stereo, Video and Motion Modeling
- Models and Methods
- Multiplicative Gating
- Multiplication and Phase Information
- Relation to Biological Models
- Relation to Feature Pooling and Deep Learning
- Inference And Learning
- Gated Inference
- Learning Higher Order Features
- Effient Spatio-Temporal Learning
- Historical Perspective
- A Tour of Higher order Features in Practice
- Video and Action Understanding
- Image Matching
- Learning Within-Image Correlations
- Learning for Invariant Classification
- Learning Stereo Vision
