Tutorials and Examples for the SimilaritySearch.jl package

Here you will find several examples for the SimilaritySearch.jl package.

Tutorials and Documentation

For step-by-step API tutorials and guides, please visit the official documentation: - SimilaritySearch.jl Documentation and Tutorial - TextSearch.jl Documentation

The demonstrations on this website focus on richer and larger datasets (using vision, full-text Wikipedia, manifolds, and prime factorization).

Installation

You need a modern Julia installation (Julia \(\ge 1.10\), including Julia \(v1.12\)).

To run and explore the examples locally, clone the repository and instantiate the environment:

git clone https://github.com/sadit/SimilaritySearchDemos.git
cd SimilaritySearchDemos
julia -t auto --project=. -e 'using Pkg; Pkg.instantiate()'

Problem statement

Given a finite dataset, \(S \subseteq U\) where \(n = |S|\), and a metric distance function \(d\) working with any pair of elements in \(U\), the similarity search problem consists on retrieving similar items to a given query \(q\), for example, the \(k\) most similar items to \(q\) in \(S\) (\(k\) nearest neighbors).

At first glance, the problem is simple since it can be solved using an exhaustive evaluation of all possible results \(d(u_1, q), \cdots, d(u_n, q)\) (that is, for all \(u_i \in S\)) and select those \(k\) items \(\{u_i\}\) having the least distance to \(q\). This solution is impractical when \(n\) is large or when the number of expected queries is high. In these cases, it is necessary to create a data structure that preprocess the dataset and reduce the cost of solving queries, it is often called a metric index. When the dataset is pretty large or the metric space is quite complex, sometimes we can loose the ability of retrieving the exact solution to gain speed, clearly, the approximation quality becomes a major concern and these approximate methods require a lot of knowledge to trade speed retrieval process also kept high the solution’s quality. Additionally, the amount of memory used by the index and the construction time are also concerns whenever \(n\) is big.

The SearchGraph index in the SimilaritySearch.jl package is a competitive alternative for solving search queries that automatically tune search speed and quality and also remains very competitive in memory and construction costs. Here we show some demostrations of how using SimilaritySearch.jl in several synthetic and real problems.