変形・照明不変の特徴量

DaLI descriptors are local image patch representations that have been shown to be robust to deformation and strong illumination changes. These descriptors are constructed by treating the image patch as a 3D surface and then simulating the diffusion of heat along the surface for different intervals of time. Small time intervals represent local deformation properties while large time intervals represent global deformation properties. Additionally, by performing a logarithmic sampling and then a Fast Fourier Transform, it is possible to obtain robustness against non-linear illumination changes. We have created the first feature point dataset that focuses on deformation and illumination changes of real world objects in order to perform evaluation, where we show the DaLI descriptors outperform all the widely used descriptors.

論文 モデル データセット

This research consisted in improving the computational efficiency, improving performance and looking at more specific details at why this descriptor is able to obtain results. We make both the descriptor code and the dataset presented in the paper publicly available. For more details please refer to the journal paper.

Flowchart of the process of creating the DaLI descriptor
Flowchart of the process of creating the DaLI descriptor.

The DaLI is a descriptor built by embedding the image patch as a surface and then using the Heat Kernel Signature to calculate the heat diffusion along the surface. In order to become invariant to illumination changes, logarithmic sampling along with the Fast Fourier Transform is employed. A flowchart of how the descriptor and its compressed (PCA) variant is constructed is shown above. The percentages indicate the computation time spent on each step. An example of a DaLI descriptor is shown below.

  • Frequency 1
  • Frequency 2
  • Frequency 3
  • Frequency 4
  • Frequency 5
  • Frequency 6

DaLI Dataset

  • Ref.
  • Target
  • LIOP
  • SIFT
  • DAISY
  • DaLI
  • DaLI-PCA

We make the full dataset that appears in the paper available to anyone who wishes to use it. We include the code to reproduce the main figures in the paper. This dataset consists of 192 unique 640x480 grayscale images corresponding to 12 different objects. Points of interest obtained by the Difference of Gaussians (DoG) detector are provided and matched across different pairs of images.

Please refer to the dataset page for full details.

Notes

  • All results in the paper were obtained on a x86_64 machine running Ubuntu 12.04 LTS using Matlab 7.14.0.739 (R2012a) and Octave 3.2. On other platforms and software versions results may differ from the paper.

  • The GIH descriptor is not provided as the only available implementation can only be run on the 32 bit Windows operating system.

  • There is an additional deformation level that does not appear in the paper and is disabled by default.

Video Sequence

We have also evaluated the descriptor on two real world sequences taken from:

Francesc Moreno-Noguer, Pascal Fua
Stochastic Exploration of Ambiguities for Nonrigid Shape Recovery
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013

The results are shown in the paper. As a supplement to the paper we show the full sequence with the relative improvement of DaLI over DAISY below:

論文

2015年

DaLI: Deformation and Light Invariant Descriptor
DaLI: Deformation and Light Invariant Descriptor
Edgar Simo-Serra, Carme Torras, Francesc Moreno-Noguer
International Journal of Computer Vision (IJCV) 115(2):135-154, 2015
PDF プロジェクトサイト ソフトウェア DOI
Recent advances in 3D shape analysis and recognition have shown that heat diffusion theory can be effectively used to describe local features of deforming and scaling surfaces. In this paper, we show how this description can be used to characterize 2D image patches, and introduce DaLI, a novel feature point descriptor with high resilience to non-rigid image transformations and illumination changes. In order to build the descriptor, 2D image patches are initially treated as 3D surfaces. Patches are then described in terms of a heat kernel signature, which captures both local and global information, and shows a high degree of invariance to non-linear image warps. In addition, by further applying a logarithmic sampling and a Fourier transform, invariance to photometric changes is achieved. Finally, the descriptor is compacted by mapping it onto a low dimensional subspace computed using Principal Component Analysis, allowing for an efficient matching. A thorough experimental validation demonstrates that DaLI is significantly more discriminative and robust to illuminations changes and image transformations than state of the art descriptors, even those specifically designed to describe non-rigid deformations. 
@Article{SimoSerraIJCV2015,
   author    = {Edgar Simo-Serra and Carme Torras and Francesc Moreno Noguer},
   title     = {{DaLI: Deformation and Light Invariant Descriptor}},
   journal   = {International Journal of Computer Vision (IJCV)},
   volume    = {115},
   number    = {2},
   pages     = {136--154},
   year      = 2015,
}

2011年

Deformation and Illumination Invariant Feature Point Descriptor
Deformation and Illumination Invariant Feature Point Descriptor
Francesc Moreno-Noguer
Conference in Computer Vision and Pattern Recognition (CVPR), 2011
PDF プロジェクトサイト
Recent advances in 3D shape recognition have shown that kernels based on diffusion geometry can be effectively used to describe local features of deforming surfaces. In this paper, we introduce a new framework that allows using these kernels on 2D local patches, yielding a novel feature point descriptor that is both invariant to non-rigid image deformations and illumination changes. In order to build the descriptor, 2D image patches are embedded as 3D surfaces, by multiplying the intensity level by an arbitrarily large and constant weight that favors anisotropic diffusion and retains the gradient magnitude information. Patches are then described in terms of a heat kernel signature, which is made invariant to intensity changes, rotation and scaling. The resulting feature point descriptor is proven to be significantly more discriminative than state of the art ones, even those which are specifically designed for describing non-rigid image deformations. 
@InProceedings{MorenoNoguerCVPR2011,
   author    = {Francesc Moreno-Noguer},
   title     = {{Deformation and Illumination Invariant Feature Point Descriptor}},
   booktitle = "Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR)",
   year      = 2011,
}

ソフトウェア

DaLI
DaLI, 1.0 (2015年01月)
Deformation and Light Invariant feature point descriptor
詳細 github ダウンロード
This is an implementation of the Deformation and Light Invariant (DaLI) descriptor. The core of the library is written in C. Additionally a Matlab/Octave interface is provided.
ceigs
ceigs, 1.1 (2012年01月)
C Wrapper for the ARPACK (Arnoldi Iteration) Library
詳細 github ダウンロード
This is a simple C frontend for ARPACK. This allows easy access to calculating a subset of eigenvectors and eigenvalues of sparse matrices. Specifically it can solve two problems: - Av = vd - Av = Mvd Where A, M are sparse matrices, v is the subset of eigenvectors and d is the diagonal matrix of eigenvalues.