Huevos

Traditional image hashing methods can be generally classified into two domains: data- independent and data-dependent. A representative data-independent method termed Locality Sensitive Hashing (LSH) [22] produces the compact binary codes by us- ing random projections on original high-dimensional features along with thresholding schemes. Then an extended version of LSH named Kernelized Locality Sensitive Hashing (KLSH) [92] is proposed, which builds the hash functions with the kernel- ized pairwise representations. However, these methods that are adopting a random projection paradigm usually yield unappreciated performance when using short codes, which makes them unsuitable for hashing applications that require compact binary codes.

Consequently, data-dependent methods are widely developed, where advanced sta- tistical learning strategies are utilized in building the hashing functions to produce compact yet effective binary codes. As mentioned above, those hashing methods can be further divided into supervised and unsupervised ones. In supervised hashing, dedicated prior knowledge is involved in the code learning. For example, Binary Re- constructive Embedding (BRE) [91] builds the kernel hash function via minimizing the squared reconstruction errors between the original and the Hamming distances of the to-be-learned binary embeddings. However, the proposed coordinate descent optimization is not efficient and the retrieval performance is not satisfactory. Kernel Supervised Hashing (KSH) [117] approximates the Hamming distances between data samples by utilizing the pairwise similarity calculating from category information. The drawback of KSH is that the intra-class distance is not optimized in the training process, which lowers the binary code quality dramatically when dealing with various categories within one dataset. In [177], Linear Discriminant Analysis hashing (LDA- Hash) is proposed to alleviate this issue by minimizing the intra-class and maximize the inter-class variations of binary codes simultaneously. Shen et al. [153] propose Su- pervised Discrete Hashing (SDH) that solves the binary codes via applying Discrete

2.1. SINGLE-MODALITY SIMILARITY SEARCH 25

Cyclic Coordinate descent (DCC) method, where the label information is used to approximate the binary code via linear classifiers. However, Support Vector Machine (SVM) [27] is trained during each iteration to generate auxiliary variables, which is not efficient when tackling large-scale datasets.

Recently, breakthrough performance has been achieved by the combination of deep learning techniques and hashing in the large-scale image retrieval tasks [43, 94, 219], where deep Convolutional Neural Network (CNN) [7, 90] is widely deployed in those works. In [219], Convolution Neural Network Hashing (CNNH) is proposed to decom- pose the hash function learning into two stages: hash code learning and deep network fine-tuning, where the similarity between data pairs is preserved by minimizing the loss between the inner product of binary code and the similarity matrix. While Deep Neural Network Hashing (DNNH) in [94], which can be treated as an updated version of CNNH, outperforms the former framework by jointly conducting in-depth feature and hash code learning within the deep structure. Despite many supervised deep hashing frameworks [101, 111, 192, 198, 248] have been proposed for better retrieval performance, they generally incorporate either labels or pairwise/triplet similarity in the training process and obtaining such supervision information is usually label- extensive and expensive, which are not appreciated in the real-world applications.

In unsupervised hashing, the hash function is formulated and built on the training of unlabeled data. Some representative methods are briefly introduced here. For in- stance, SPectral hashing (SP) [211] is proposed to build the hash function via solving the eigenfunctions, where the smallest eigenvalues are thresholded to zero to obtain the binary codes. However, SP works under the assumption that the data structure follows a uniform distribution, which impedes it from the complex applications that involve uncertain distributions. In [71], Spherical Hashing (SpH) learns a series of spherical functions and the binary codes are generated by quantizing the distances between the original feature representations and their corresponding centers. The limitation of SpH is that the optimization is not performed in the binary space, which yields low code quality. Moreover, the iterative center learning is computation- ally expensive and time-consuming. Iterative Quantization (ITQ) [51] is proposed to learn the binary codes via minimizing the quantization errors between the target codes and the product of their original representations and an orthogonal rotation matrix. Principal Component Analysis (PCA) [213] or Canonical Correlation Analy- sis (CCA) [67] can be applied to reduce the high dimensionality of the original feature before the discrete optimization. While an updated version of ITQ is proposed in [57] termed ITQ+ with both robustness and generalization capability enhanced by using

2.1. SINGLE-MODALITY SIMILARITY SEARCH 26

a`p-norm distance in the discrete optimization. Some related works of ITQ are also

proposed: Isotropic Hashing (IsoH) [88] and Harmonious Hashing (HamH) [221]. To be specific, IsoH utilizes a rotation to balance the variance by minimizing the recon- struction error of the covariance matrix and a diagonal matrix, while HamH minimizes the distance between the rotated data and a perfectly balanced matrix. However, the reconstruction on a small covariance matrix in IsoH is unstable in large-scale and high-dimensional experiments. The strict requirements of HamH and its non- iterative optimization algorithm may fail to find a good solution. Consequently, Liu et al. [118] propose Anchor Graph Hashing (AGH) for the large-scale image retrieval, where a small number of anchors are learned to represent the similarities between data pairs in the training set, thus preserving the neighborhood structure inherently in an efficient way. Discrete Graph Hashing (DGH) [116] is further developed to improve retrieval performance, where the hash codes are directly optimized in such graph-based hashing. The idea of using anchor points is also introduced in K-Means hashing (KMH) [69], which generates effective hash codes via minimizing the Ham- ming distances between anchors after binarization. However, the hash code quality from both the aforementioned methods heavily relies on the anchor point selection, where the selection process is sometimes arbitrary and computationally expensive in most cases.

Consistent with supervised hashing, deep neural networks have been further in- volved in the recent algorithms of unsupervised hashing, which enables to handle the data distributions with the nonlinearity nature in the real-world application sce- narios [193]. In [43], Deep Hashing (DH) is developed to learn the hash function with multiple hierarchical nonlinear transformations, where independent bits in bi- nary codes with even distribution can be achieved. Subsequently, an unsupervised learning-based deep hashing framework is proposed in [122] that trains the deep neu- ral network by minimizing the compact real-valued codes and their binary codes. They further extend this work to supervised deep hashing and multi-label supervised deep hashing, which aims at generating more discriminative binary codes with the aid of supervision information. In [23], Nonlinear Discrete Hashing (NDH) is proposed for scalable image search, where the binary codes are optimized with discrete quanti- zation and the reconstruction errors are minimized between the learned binary codes and the original data, respectively. The above deep-based methods generally adopt shallow deep networks as the backbone in the hash code learning, which weakens the feature representation ability to some extent. Consequently, more advanced deep net- works like Generative Adversarial Network (GAN) [52] have been incorporated in the

2.1. SINGLE-MODALITY SIMILARITY SEARCH 27

hashing framework, such as HashGAN [49], which improves the retrieval performance significantly because of the more powerful deep model.