Using with ManifoldLearning

by: Eric S. Téllez

This demonstration is about using SimilaritySearch and ManifoldLearning methods through SimSearchManifoldLearning.

SimSearchManifoldLearning 0.4 note: The fit(Isomap, X, nntype=...) API is unchanged; this release updates the SimilaritySearch v1.2 backend internally. ApproxEuclidean, ApproxManhattan, and ApproxChebyshev selectors work exactly as before.

using SimilaritySearch, SimilaritySearch.Dist, SimSearchManifoldLearning, ManifoldLearning, Primes, Plots, StatsPlots, StatsBase, LinearAlgebra, Markdown, Random

SCurve example

X, L = ManifoldLearning.scurve(segments=5)

scatter(X[1, :], X[2, :], X[3, :], color=L, alpha=0.5)

SimilaritySearch support exact and approximate algorithms to solve k nearest neighbors. Also, it supports different metrics. For instance, let see how the selection of the distance function modifies the projection.

Manhattan distance (\(L_1\))

let Y = predict(fit(Isomap, X, nntype=ApproxManhattan))
    scatter(Y[1,:], Y[2,:], color=L, alpha=0.5)
end

LOG add! sp=1 ep=512 BeamSearch(bsize=4, Δ=1.0, maxvisits=1000000) n.size-quantiles=[0.0, 3.0, 3.0, 4.0, 6.0] mem=427MB max-rss=1388MB 2026-08-27T12:36:51.482

Euclidean distance (\(L_2\))

let
    E = predict(fit(Isomap, X, nntype=ApproxEuclidean))
    scatter(E[1,:], E[2,:], color=L, alpha=0.5)
end

LOG add! sp=1 ep=512 BeamSearch(bsize=4, Δ=1.0, maxvisits=1000000) n.size-quantiles=[0.0, 3.0, 3.0, 3.0, 4.0] mem=480MB max-rss=1493MB 2026-08-27T12:37:01.939

Chebyshev distance (\(L_\infty\))

let
    Ch = predict(fit(Isomap, X, nntype=ApproxChebyshev))
    scatter(Ch[1,:], Ch[2,:], color=L, alpha=0.5)
end

LOG add! sp=1 ep=512 BeamSearch(bsize=4, Δ=1.0, maxvisits=1000000) n.size-quantiles=[0.0, 3.0, 3.0, 4.0, 6.0] mem=479MB max-rss=1585MB 2026-08-27T12:37:09.973

Visualizing prime gaps

The difference between contiguous prime numbers is called a Prime gap. We use this series of values as a time series example due to its interesting behavior and since it can be computed without downloading more than the necessary packages.

This example shows how to generate the dataset and index it. We will use the ManifoldLearning for generating the 2d visualization.

Generation of the dataset

The time series is represented with windows of size w, we also take log of gaps to reduce variance in gap values. We create a matrix to avoid redefinition of the knn interface for ManifoldLearning.

function create_database_primes_diff(n, w)
    T = log2.(diff(primes(n)))
    M = Matrix{Float32}(undef, w, length(T) - w)
    @info size(M)
    for i in 1:size(M, 2)
        M[:, i] .= view(T, i:(i+w-1))
    end

    M
end


x, y = let
    P = create_database_primes_diff(3 * 10^4, 5)
    # or LLE
    primesgap = fit(Isomap, P; k=16, maxoutdim=2, nntype=ApproxEuclidean)
    
    p = predict(primesgap)
    p[1, :], p[2, :]
end

A 2D histogram

histogram2d(x, y; nbins=100)

Environment and dependencies

Julia Version 1.12.7
Commit 6d172b025e4 (2026-08-15 08:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 64 × Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, cascadelake)
  GC: Built with stock GC
Threads: 64 default, 1 interactive, 64 GC (on 64 virtual cores)
Environment:
  JULIA_PROJECT = @.
  JULIA_NUM_THREADS = auto
  JULIA_LOAD_PATH = @:@stdlib
Status `~/Research/SimilaritySearchDemos/Project.toml`
  [aaaa29a8] Clustering v0.15.8
  [944b1d66] CodecZlib v0.7.9
  [a93c6f00] DataFrames v1.8.2
  [f67ccb44] HDF5 v0.17.3
  [0f8b85d8] JSON3 v1.14.3
  [23fbe1c1] Latexify v0.16.12
  [eb30cadb] MLDatasets v0.7.21
  [06eb3307] ManifoldLearning v0.9.0
⌅ [ca7969ec] PlotlyLight v0.11.1
  [91a5bcdd] Plots v1.41.7
  [27ebfcd6] Primes v0.5.7
  [92933f4c] ProgressMeter v1.11.0
  [ca7ab67e] SimSearchManifoldLearning v0.4.0 `../SimSearchManifoldLearning.jl`
  [053f045d] SimilaritySearch v1.2.0 `../SimilaritySearch.jl`
⌅ [2913bbd2] StatsBase v0.33.21
  [f3b207a7] StatsPlots v0.15.8
  [7f6f6c8a] TextSearch v1.1.1 `../TextSearch.jl`
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated`