Online Multi-Modal Distance Metric Learning with Application to Image Retrieval

0
748
Online Multi-Modal Distance Metric Learning with Application to Image Retrieval

Online Multi-Modal Distance Metric Learning with Application to Image Retrieval

Abstract

Online Multi-Modal Distance Metric Learning with Application to Image Retrieval,Distance metric learning (DML) is an important technique to improve similarity search in content-based image retrieval. Despite being studied extensively, most existing DML approaches typically adopt a single-modal learning framework that learns the distance metric on either a single feature type or a combined feature space where multiple types of features are simply concatenated. Such single-modal DML methods suffer from some critical limitations: (i) some type of features may significantly dominate the others in the DML task due to diverse feature representations; and (ii) learning a distance metric on the combined high-dimensional feature space can be extremely time-consuming using the naive feature concatenation approach. To address these limitations, in this paper, we investigate a novel scheme of online multi-modal distance metric learning (OMDML), which explores a unified two-level online learning scheme: (i) it learns to optimize a distance metric on each individual feature space; and (ii) then it learns to find the optimal combination of diverse types of features. To further reduce the expensive cost of DML on high-dimensional feature space, we propose a low-rank OMDML algorithm which not only significantly reduces the computational cost but also retains highly competing or even better learning accuracy. We conduct extensive experiments to evaluate the performance of the proposed algorithms for multi-modal image retrieval, in which encouraging results validate the effectiveness of the proposed technique.

Conclusions

This  paper investigated a novel family of online multi-modal distance metric learning algorithms for CBIR tasks by exploiting multiple types of features. We pinpointed some major limitations of traditional DML approaches in practice, and presented the online multi-modal DML method which simultaneously learns both the optimal distance metric on each individual feature space and the optimal combination of multiple metrics on different types of features. Further, we proposed the low-rank online multi-modal DML algorithm, which not only runs more efficiently and scalably, but also achieves the state-of-the-art performance among the competing algorithms in our experiments. Future work can extend our framework in resolving other types of multimodal data analytics tasks beyond image retrieval.