adding in sub-headings for CV, NLP and multi-modal (#6)

* Add subheadings for CV, NLP and multi-modal.

* Add ANN, some papers and new dataset.

* Fix language style and some wordings.

Co-authored-by: M. Yusuf Sarıgöz <[email protected]>

Author: stephenleo

README.md

@@ -120,7 +120,34 @@ recommender algorithms.</summary>
 > It supports incorporating user and item features to the traditional matrix factorization. It represents users and items as a sum of the latent representations of their features, thus achieving a better generalization.
 </details>
 
+## Approximate Nearest Neighbors ⚡
+<details>
+<summary><a href="https://github.com/erikbern/ann-benchmarks">ANN Benchmarks</a> - Benchmarking various ANN implementations for different metrics.</summary>
+
+> It provides benchmarking of 20+ ANN algorithms on nine standard datasets with support to bring your dataset. ([Medium Post](https://medium.com/towards-artificial-intelligence/how-to-choose-the-best-nearest-neighbors-algorithm-8d75d42b16ab?sk=889bc0006f5ff773e3a30fa283d91ee7))
+</details>
+
+<details>
+<summary><a href="https://github.com/facebookresearch/faiss">FAISS</a> - Efficient similarity search and clustering of dense vectors that possibly do not fit in RAM</summary>
+
+> It is not the fastest ANN algorithm but achieves memory efficiency thanks to various quantization and indexing methods such as IVF, PQ, and IVF-PQ. ([Tutorial](https://www.pinecone.io/learn/faiss-tutorial/))
+</details>
+
+<details>
+<summary><a href="https://github.com/nmslib/hnswlib">HNSW</a> - Hierarchical Navigable Small World graphs</summary>
+
+> It is still one of the fastest ANN algorithms out there, requiring relatively a higher memory usage. (Paper: [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://arxiv.org/abs/1603.09320))
+</details>
+
+<details>
+<summary><a href="https://github.com/google-research/google-research/tree/master/scann">Google's SCANN</a> - The technology behind vector search at Google</summary>
+
+> Paper: [Accelerating Large-Scale Inference with Anisotropic Vector Quantization](https://arxiv.org/abs/1908.10396)
+</details>
+
+
 ## Papers 🔬
+### Loss Functions
 <details>
 <summary><a href="http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf">Dimensionality Reduction by 
 Learning an Invariant Mapping</a> - First appearance of Contrastive Loss.</summary>
@@ -176,6 +203,35 @@ Self-Supervised Learning</a> - Better regularization for high-dimensional embedd
 
 </details>
 
+### Computer Vision
+<details>
+<summary><a href="http://arxiv.org/abs/2002.05709">SimCLR: A Simple Framework for Contrastive Learning of Visual Representations</a> - Self-Supervised method comparing two differently augmented versions of the same image with Contrastive Loss</summary>
+
+> It demonstrates among other things that
+> - composition of data augmentations plays a critical role - Random Crop + Random Color distortion provides the best downstream classifier accuracy,
+> - introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations,
+> - and Contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning.
+</details>
+
+### Natural Language Processing
+<details>
+<summary><a href="https://aclanthology.org/2021.emnlp-main.552">SimCSE: Simple Contrastive Learning of Sentence Embeddings</a> - An unsupervised approach, which takes an input sentence and predicts itself in a contrastive objective, with only standard dropout used as noise.
+</summary>
+
+> They also incorporates annotated pairs from natural language inference datasets into their contrastive learning framework in a supervised setting, showing that contrastive learning objective regularizes pre-trained embeddings’ anisotropic space to be more uniform, and it better aligns positive pairs when supervised signals are available.
+</details>
+
+### Multi-Modal
+<details>
+<summary><a href="http://arxiv.org/abs/2103.00020">Learning Transferable Visual Models From Natural Language Supervision</a> - The paper that introduced CLIP: Training a unified vector embedding for image and text.
+</summary>
+</details>
+
+<details>
+<summary><a href="http://arxiv.org/abs/2102.05918">Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision</a> - Google's answer to CLIP: Training a unified vector embedding for image and text but using noisy text instead of a carefully curated dataset.
+</summary>
+</details>
+
 
 ## Datasets ℹ️
 > Practitioners can use any labeled or unlabelled data for metric learning with an appropriate method chosen. However, some datasets are particularly important in the literature for benchmarking or other ways, and we list them in this section.
@@ -210,3 +266,9 @@ serving as a useful benchmark.</summary>
 
 > The dataset is published along with ["Deep Metric Learning via Lifted Structured Feature Embedding"](https://github.com/rksltnl/Deep-Metric-Learning-CVPR16) paper.
 </details>
+
+<details>
+<summary><a href="https://www.drivendata.org/competitions/79/">MetaAI's 2021 Image Similarity Dataset and Challenge</a> - dataset has 1M Reference image set, 1M Training image set, 50K Dev query image set and 50K Test query image set</summary>
+
+> The dataset is published along with ["The 2021 Image Similarity Dataset and Challenge"](http://arxiv.org/abs/2106.09672) paper.
+</details>