DaLI Dataset

We present a dataset for the evaluation of deformation and illumination invariance of local image patch feature point descriptors. The dataset consists of 192 unique 640x480 grayscale images corresponding to 12 different objects. Points of interest are obtained by the Difference of Gaussians (DoG) detected and were manually matched in order to obtain a ground truth mapping.

  • Ref.
  • Target
  • LIOP
  • SIFT
  • DAISY
  • DaLI
  • DaLI-PCA

We make the full dataset that appears in the paper available to anyone who wishes to use it. We include the code to reproduce the main figures in the paper. This dataset consists of 192 unique 640x480 grayscale images corresponding to 12 different objects. Points of interest obtained by the Difference of Gaussians (DoG) detector are provided and matched across different pairs of images. We provide three different evaluations:

  1. Deformation only (test_deform.m)
  2. Illumination only (test_illum.m)
  3. Both deformation and illumination (test_both.m)

To reproduce the results in the paper start by downloading and unpacking the dataset:

wget https://esslab.jp/~ess/data/dali_dataset.tar.bz2
tar xvjf dali_dataset.tar.bz2
cd dali_dataset

Next run matlab and from matlab call:

test_both % or test_deform, test_illum depending on what you wish to evaluate

This will extract the patches and start evaluating different descriptors. However, it will not evaluate the DaLI descriptor as it is unable to be run under matlab due to library conflicts. In order to also compute the DaLI descriptor please run the same command from octave after uncommenting main_desc_dali in init_final_desc.m. Note that once the results are obtained they can also be loaded under matlab.

In order to calculate the DaLI-PCA descriptor it is necessary to compute the DaLI descriptor first, then run the dalipca.m script from matlab. This will generate the cached descriptor files for the descriptor. It is then possible to uncomment main_desc_dalipca in init_final_desc.m which will allow obtaining results with both matlab and octave.

To summarize all the steps:

  1. Download and unpack dataset
  2. Run test_both, test_illum and test_deform from matlab
  3. Uncomment main_desc_dali from init_final_desc.m
  4. Run test_both, test_illum and test_deform from octave
  5. Run dalipca
  6. Uncomment main_desc_dalipca from init_final_desc.m
  7. Run test_both, test_illum and test_deform from matlab

As we know this is fairly complicated we also provide the results files for all the descriptors, which allow you to directly uncomment all the descriptors in init_final_desc.m directly after unpackaging this. It will also allow you to avoid having to spend many hours computing descriptors.

To compare against your descriptor, please look at descriptors/main_desc_dali.m as an example of how to create the descriptor structure. It is then as simple as adding a new line to init_final_desc.m.

Notes

  • All results in the paper were obtained on a x86_64 machine running Ubuntu 12.04 LTS using Matlab 7.14.0.739 (R2012a) and Octave 3.2. On other platforms and software versions results may differ from the paper.

  • The GIH descriptor is not provided as the only available implementation can only be run on the 32 bit Windows operating system.

  • There is an additional deformation level that does not appear in the paper and is disabled by default.

Citing

If you use this dataset, please cite

@Article{SimoSerraIJCV2015,
  author    = {Edgar Simo-Serra and Carme Torras and Francesc Moreno Noguer},
  title     = {{DaLI: Deformation and Light Invariant Descriptor}},
  journal   = {International Journal of Computer Vision (IJCV)},
  volume    = {115},
  number    = {2},
  pages     = {136--154},
  year      = 2015,
}