Visualizing MNIST database

by: Eric S. Téllez

This example creates a visualization of the MNIST images (hand written digits) using MLDatasets.jl to retrieve it.

Note: This example needs a lot of computing power; therefore you may want to set the environment variable JULIA_NUM_THREADS=auto before running julia.

using SimilaritySearch, SimilaritySearch.Dist, SimSearchManifoldLearning, Plots, StatsBase, LinearAlgebra, Markdown, MLDatasets, Random

db, y, dist = let data = MNIST(split=:train)
    T, y = data.features, data.targets
    n = size(T, 3)
    MatrixDatabase(Float32.(reshape(T, (28*28, n)))), y, Dist.SqL2()
end

Now we can create the index

1index = SearchGraph(dist, db)
ctx = SearchGraphContext(hyperparameters_callback=OptimizeParameters(MinRecall(0.99)))
2index!(index, ctx)
3optimize_index!(index, ctx, MinRecall(0.95))

1

Defines the index and the search context (caches and hyperparameters); particularly, we use a very high quality build MinRecall(0.99); high quality constructions yield to faster queries due to the underlying graph structure.

2

Actual indexing procedure using the given search context.

3

Optimizing the index to trade quality and speed.

Searching

Our index can solve queries over the entire dataset, for instance, solving synonym queries as nearest neighbor queries.

function search_and_render(index, ctx, q, res)
    res = reuse!(res)
    @time search(index, ctx, q, res)
    qinverted = 1 .- reshape(q, (28, 28))' # distinguishability
    h = hcat(qinverted, [reshape(index[p.id], (28, 28))' for p in IdDistView(res)]...)
    
    Gray.(h)
end

res = knnqueue(ctx, 12)
for _ in 1:7
    for qid in rand(1:length(index))
        display(search_and_render(index, ctx, index[qid], res))
    end
end

  0.000175 seconds (3 allocations: 64 bytes)
  0.000145 seconds (3 allocations: 64 bytes)
  0.000138 seconds (3 allocations: 64 bytes)
  0.000190 seconds (3 allocations: 64 bytes)
  0.000129 seconds (3 allocations: 64 bytes)
  0.000148 seconds (3 allocations: 64 bytes)
  0.000133 seconds (3 allocations: 64 bytes)

UMAP Visualization

function normcolors(V)
    min_, max_ = extrema(V)
    V .= (V .- min_) ./ (max_ - min_)
    V .= clamp.(V, 0, 1)
end

normcolors(@view e3[1, :])
normcolors(@view e3[2, :])
normcolors(@view e3[3, :])

let C = [RGB(c[1], c[2], c[3]) for c in eachcol(e3)],
    X = view(e2, 1, :),
    Y = view(e2, 2, :)
    scatter(X, Y, color=C, fmt=:png, alpha=0.2, size=(600, 600), ma=0.3, ms=2, msw=0, label="", yticks=nothing, xticks=nothing, xaxis=false, yaxis=false)
    for i in 1:100
        j = rand(1:length(y))
        annotate!(X[j], Y[j], text(y[j], :black, :right, 8, "noto"))
    end
end

plot!()

e2, e3 = let min_dist=0.5f0,
             k=7,
             n_epochs=75,
             neg_sample_rate=3,
             tol=1e-3,
             layout=SpectralLayout()

    @time "Compute 2D UMAP model" U2 = fit(UMAP, index; k, neg_sample_rate, layout, n_epochs, tol, min_dist)
    @time "Compute 3D UMAP model" U3 = fit(U2, 3; neg_sample_rate, n_epochs, tol)
    @time "predicting 2D embeddings" e2 = clamp.(predict(U2), -10f0, 10f0)
    @time "predicting 3D embeddings" e3 = clamp.(predict(U3), -10f0, 10f0)
    e2, e3
end

Finding Near-Duplicate MNIST Digits

Near-duplicate detection finds digits that are nearly identical — useful for dataset deduplication or identifying redundant samples.

# searchbatch returns (ids, dists) matrices; each column i holds the k neighbors of query i
ids, dists = searchbatch(index, ctx, index.db, 5)

# neardup: collect pairs (i, j) where i < j and the distance is below a threshold
threshold = 50f0  # squared-L2 threshold for "near duplicate" on 28×28 float32 images

neardup_pairs = Tuple{Int,Int}[]
for i in axes(ids, 2)
    for ki in axes(ids, 1)
        j = ids[ki, i]
        d = dists[ki, i]
        if j > i && d < threshold
            push!(neardup_pairs, (i, j))
        end
    end
end

@info "Found $(length(neardup_pairs)) near-duplicate pairs (threshold=$threshold)"

[ Info: Found 111829 near-duplicate pairs (threshold=50.0)

# Visualize the first few near-duplicate pairs
n_show = min(6, length(neardup_pairs))
plots = []
for (i, j) in neardup_pairs[1:n_show]
    img_i = Gray.(reshape(index[i], (28, 28))')
    img_j = Gray.(reshape(index[j], (28, 28))')
    push!(plots, plot(hcat(img_i, img_j), title="$(y[i]) vs $(y[j])", axis=false, ticks=false))
end
plot(plots..., layout=(2, 3), size=(600, 400))

Final notes

This example shows how to index and visualize the MNIST dataset using UMAP low dimensional projections. Low dimensional projections are made with SimSearchManifoldLearning, note that SimilaritySearch is also used for computing the all \(k\) nearest neighbors needed by the UMAP model. The neardup section demonstrates how searchbatch (which now returns an (ids, dists) tuple in SimilaritySearch v1.2) can be used to efficiently detect near-identical images. Note that this notebook should be ran with several threads to reduce time costs.

Environment and dependencies

Julia Version 1.12.7
Commit 6d172b025e4 (2026-08-15 08:05 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 64 × Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, cascadelake)
  GC: Built with stock GC
Threads: 64 default, 1 interactive, 64 GC (on 64 virtual cores)
Environment:
  JULIA_PROJECT = @.
  JULIA_NUM_THREADS = auto
  JULIA_LOAD_PATH = @:@stdlib
Status `~/Research/SimilaritySearchDemos/Project.toml`
  [aaaa29a8] Clustering v0.15.8
  [944b1d66] CodecZlib v0.7.9
  [a93c6f00] DataFrames v1.8.2
  [f67ccb44] HDF5 v0.17.3
  [0f8b85d8] JSON3 v1.14.3
  [23fbe1c1] Latexify v0.16.12
  [eb30cadb] MLDatasets v0.7.21
  [06eb3307] ManifoldLearning v0.9.0
⌅ [ca7969ec] PlotlyLight v0.11.1
  [91a5bcdd] Plots v1.41.7
  [27ebfcd6] Primes v0.5.7
  [92933f4c] ProgressMeter v1.11.0
  [ca7ab67e] SimSearchManifoldLearning v0.4.0 `../SimSearchManifoldLearning.jl`
  [053f045d] SimilaritySearch v1.2.0 `../SimilaritySearch.jl`
⌅ [2913bbd2] StatsBase v0.33.21
  [f3b207a7] StatsPlots v0.15.8
  [7f6f6c8a] TextSearch v1.1.1 `../TextSearch.jl`
Info Packages marked with ⌅ have new versions available but compatibility constraints restrict them from upgrading. To see why use `status --outdated`