Parallel construction and parallel search

by: Eric S. Téllez

using SimilaritySearch

Similarity search on very large datasets and high-dimensional datasets require high computational resources. In this example we show how to arallelize both the construction and search to be able to handle this kind of databases.

dim = 16
1db = MatrixDatabase(randn(Float32, dim, 10^5))
2queries = MatrixDatabase(randn(Float32, dim, 30))
3dist = SqL2Distance()
4ctx = SearchGraphContext(parallel_first_block=256, parallel_block=512)
5G = SearchGraph(; dist, db)
6index!(G, ctx)

1

A synthetic database of dimension 16 and \(10^5\) vectors.

2

A synthetic query set of \(30\) points.

3

The distance function.

4

The search context working with SearchGraph; a set of hyperparameters for the index.

5

The index definition.

6

The index construction.

The SearchGraph construction algorithm is incremental:

• If the index is empty, an element is inserted just inserting it into the index.
• If the index is not empty, the element is inserted and connected to its nearest neighbors (looking into the current index).

The parallel construction is made with index! or append_items!; for this matter these functions accept a parallel_block argument via the ctx context, that controls how many elements are inserted at once, i.e., looking for its nearest neighbors in parallel and connected also in parallel.

As in the sequential version, a minimum number of elements must exists to work, and therefore, the parallel_first_block argument can also be specified. By default, it is equal to parallel_block. The parallel_block argument should be set to at least the number of available threads, and perhaps multiplying it by a small constant is also a good approach.

Note that you must not call push_item!, append_items!, or index! from several threads. The default algorithm will takes advantage of the available threads using a single call.

Searching

Once the index is constructed, you can solve batches in parallel and also single queries. In contrast with append, these functions can be called in multithreading algorithms. However, you must pause the searching requests while perform insertions (parallel or sequential); mixing insertions and search produces an undefined behavior for search results.

I, D = searchbatch(G, ctx, queries, 10)

Threads.@threads for i in eachindex(queries)
    p = search(G, ctx, queries[i], KnnResult(10))
    res = p.res
    
    print("=== $i -- nearest neighbor:")
    println(res[1])
    print("=== $i -- result set:")
    println(collect(res))
    print("=== $i -- identifiers:")
    println(collect(IdView(res))) # do something with `res`
    print("=== $i -- distances:")
    println(collect(DistView(res))) # do something with `res`
end

=== 7 -- nearest neighbor:=== 19 -- nearest neighbor:=== 15 -- nearest neighbor:=== 11 -- nearest neighbor:=== 10 -- nearest neighbor:=== 27 -- nearest neighbor:=== 25 -- nearest neighbor:=== 29 -- nearest neighbor:=== 26 -- nearest neighbor:=== 5 -- nearest neighbor:=== 18 -- nearest neighbor:=== 14 -- nearest neighbor:=== 22 -- nearest neighbor:=== 12 -- nearest neighbor:=== 4 -- nearest neighbor:=== 30 -- nearest neighbor:=== 6 -- nearest neighbor:=== 13 -- nearest neighbor:=== 3 -- nearest neighbor:=== 1 -- nearest neighbor:=== 2 -- nearest neighbor:=== 16 -- nearest neighbor:=== 20 -- nearest neighbor:=== 23 -- nearest neighbor:=== 9 -- nearest neighbor:=== 21 -- nearest neighbor:=== 17 -- nearest neighbor:IdWeight(=== 24 -- nearest neighbor:=== 28 -- nearest neighbor:=== 8 -- nearest neighbor:0x00009da0, 6.675923f0)
=== 15 -- result set:IdWeight(0x0000ef27, 4.726558f0)
=== 24 -- result set:IdWeight(0x0000cd58, 4.528069f0)
=== 27 -- result set:IdWeight(0x00000ff4, 5.687612f0)
=== 3 -- result set:IdWeight(0x0000f9b6, 6.8288493f0)
IdWeight=== 9 -- result set:(0x0000c4a0, 3.047554f0)
=== 6 -- result set:IdWeight(0x000140bc, 4.0632343f0)
=== 19 -- result set:IdWeight(0x0001277e, 5.9874015f0)
IdWeight=== 11 -- result set:(0x00007e99, 5.232687f0)
=== 10 -- result set:IdWeight(0x0001778a, 5.571958f0)
=== 23 -- result set:IdWeight(0x0000083b, 2.6822302f0)
=== 29 -- result set:IdWeight(0x00009d78, 3.5820096f0)
=== 8 -- result set:IdWeight(0x000150f8, 3.9342616f0)
=== 13 -- result set:IdWeight(0x0000be28, 6.292833f0)
IdWeight=== 14 -- result set:(0x0000fe5d, 7.1279845f0)
=== 30 -- result set:IdWeight(0x00013579, 3.4584124f0)
IdWeight=== 5 -- result set:(0x000067ba, 2.8870392f0)
IdWeight=== 18 -- result set:(0x0000b0ba, 2.2960021f0)
IdWeight=== 22 -- result set:(0x0000a290, 5.2792125f0)
IdWeight=== 4 -- result set:(0x000118bf, 4.038556f0)
=== 25 -- result set:IdWeight(0x00012021, 2.6875844f0)
IdWeight=== 16 -- result set:(0x00004d1a, 3.3135195f0)
IdWeight=== 1 -- result set:(0x00002ce3, 4.406932f0)
=== 21 -- result set:IdWeight(0x0000527f, 4.530012f0)
IdWeight=== 12 -- result set:(0x0000b32d, 3.2068803f0)
=== 26 -- result set:IdWeight(0x00004736, 6.6390896f0)
=== 17 -- result set:IdWeight(0x00006e2b, 3.0433564f0)
IdWeight=== 28 -- result set:(0x0000f274, 3.063364f0)
IdWeight=== 2 -- result set:(0x00006d0c, 4.363042f0)
=== 20 -- result set:IdWeight(0x00007b9a, 2.5394197f0)
=== 7 -- result set:IdWeight[IdWeight(0x00009da0, 6.675923f0), IdWeight(0x000061a6, 7.02988f0), IdWeight(0x00011464, 7.1213803f0), IdWeight(0x0001068c, 7.567528f0), IdWeight(0x00012719, 8.113082f0), IdWeight(0x000079be, 8.482317f0), IdWeight(0x00016ac6, 8.860224f0), IdWeight(0x00000cc5, 8.981895f0), IdWeight(0x000115de, 9.03268f0), IdWeight(0x00010a2e, 9.121991f0)]
=== 15 -- identifiers:IdWeight[IdWeight(0x0000be28, 6.292833f0), IdWeight(0x0000e60e, 7.7584715f0), IdWeight(0x00000393, 8.022731f0), IdWeight(0x0000788a, 8.082609f0), IdWeight(0x0000702a, 8.101276f0), IdWeight(0x00007fde, 8.145304f0), IdWeight(0x0000264c, 8.16899f0), IdWeight(0x0001653a, 8.294217f0), IdWeight(0x0000c7f5, 8.562826f0), IdWeight(0x00007381, 8.667862f0)]
=== 14 -- identifiers:IdWeight[IdWeight(0x00002ce3, 4.406932f0), IdWeight(0x0000202e, 4.557166f0), IdWeight(0x0001762e, 4.912068f0), IdWeight(0x00014114, 4.91214f0), IdWeight(0x0000d64f, 4.965306f0), IdWeight(0x0001364f, 4.981351f0), IdWeight(0x00004b95, 4.991246f0), IdWeight(0x00013560, 5.1663833f0), IdWeight(0x00006af8, 5.600113f0), IdWeight(0x00004010, 5.64502f0)]
=== 21 -- identifiers:IdWeight[IdWeight(0x0000b0ba, 2.2960021f0), IdWeight(0x00002460, 2.8062224f0), IdWeight(0x000042f1, 3.5092864f0), IdWeight(0x0001123c, 3.6508331f0), IdWeight(0x00001b1f, 3.6789362f0), IdWeight(0x0000a461, 3.7138393f0), IdWeight(0x00015f4f, 3.8067896f0), IdWeight(0x000096ce, 3.9141073f0), IdWeight(0x000072bf, 4.0175414f0), IdWeight(0x000181bc, 4.1544557f0)]
=== 22 -- identifiers:IdWeight[IdWeight(0x00004d1a, 3.3135195f0), IdWeight(0x00016d33, 3.3848624f0), IdWeight(0x0001804d, 4.025727f0), IdWeight(0x00013024, 4.077698f0), IdWeight(0x00002896, 4.366047f0), IdWeight(0x0001727d, 4.375115f0), IdWeight(0x0000f13c, 4.681094f0), IdWeight(0x0000054c, 4.71946f0), IdWeight(0x0000f70a, 4.9946475f0), IdWeight(0x000034b9, 5.0513253f0)]
=== 1 -- identifiers:IdWeight[IdWeight(0x0001277e, 5.9874015f0), IdWeight(0x00003feb, 6.110808f0), IdWeight(0x0000b894, 6.878881f0), IdWeight(0x000008f9, 7.056838f0), IdWeight(0x00016b3d, 7.3161545f0), IdWeight(0x00001839, 7.4862347f0), IdWeight(0x00005846, 7.9547477f0), IdWeight(0x00002ea3, 8.335855f0), IdWeight(0x00002c44, 8.767622f0), IdWeight(0x00008d45, 8.830698f0)]
=== 11 -- identifiers:IdWeight[IdWeight(0x00012021, 2.6875844f0), IdWeight(0x00002bee, 4.135638f0), IdWeight(0x00017f60, 4.8320765f0), IdWeight(0x00003067, 5.0121684f0), IdWeight(0x00016933, 5.0521636f0), IdWeight(0x0000f909, 5.074626f0), IdWeight(0x0001222b, 5.080174f0), IdWeight(0x00001893, 5.138646f0), IdWeight(0x00004463, 5.2589445f0), IdWeight(0x00006bce, 5.5174394f0)]
=== 16 -- identifiers:IdWeight[IdWeight(0x00004736, 6.6390896f0), IdWeight(0x0000a9af, 6.6492963f0), IdWeight(0x00012c60, 7.055795f0), IdWeight(0x0001315a, 7.0637984f0), IdWeight(0x00012a0a, 7.361882f0), IdWeight(0x0000d076, 7.6028204f0), IdWeight(0x000170b5, 7.618756f0), IdWeight(0x0001697b, 7.7091656f0), IdWeight(0x0000b399, 7.9337654f0), IdWeight(0x0000594a, 8.160997f0)]
=== 17 -- identifiers:IdWeight[IdWeight(0x000067ba, 2.8870392f0), IdWeight(0x00007add, 4.5670953f0), IdWeight(0x00008246, 5.4409337f0), IdWeight(0x00009276, 5.633697f0), IdWeight(0x00014fab, 5.681482f0), IdWeight(0x00014e37, 5.6852694f0), IdWeight(0x0001083c, 5.819864f0), IdWeight(0x0000dfa0, 5.8213906f0), IdWeight(0x0001255f, 5.8465f0), IdWeight(0x0000d796, 5.860892f0)]
=== 18 -- identifiers:IdWeight[IdWeight(0x00013579, 3.4584124f0), IdWeight(0x0000dbdc, 4.4805098f0), IdWeight(0x00005e87, 4.483524f0), IdWeight(0x00000624, 4.515228f0), IdWeight(0x00016492, 4.5867596f0), IdWeight(0x0000f4ce, 4.593042f0), IdWeight(0x00005d3a, 4.647873f0), IdWeight(0x0000f096, 4.8091617f0), IdWeight(0x00010555, 4.95529f0), IdWeight(0x0000e257, 4.956506f0)]
=== 5 -- identifiers:IdWeight[IdWeight(0x0000ef27, 4.726558f0), IdWeight(0x0000afc0, 5.0613613f0), IdWeight(0x00000c0a, 5.0821133f0), IdWeight(0x000158a3, 5.130314f0), IdWeight(0x00008272, 5.1979394f0), IdWeight(0x000172a9, 5.204519f0), IdWeight(0x000060bf, 5.3080554f0), IdWeight(0x000021b0, 5.5862293f0), IdWeight(0x000110d8, 5.617485f0), IdWeight(0x000085d1, 5.620654f0)]
=== 24 -- identifiers:IdWeight[IdWeight(0x0000f274, 3.063364f0), IdWeight(0x0001405f, 3.3474534f0), IdWeight(0x000004ce, 3.7191494f0), IdWeight(0x000112b2, 4.0377135f0), IdWeight(0x0000832b, 4.174683f0), IdWeight(0x000043f0, 4.3115225f0), IdWeight(0x00013d5a, 4.5141444f0), IdWeight(0x000185d1, 4.5508885f0), IdWeight(0x00008b03, 4.661785f0), IdWeight(0x0001672d, 4.676373f0)]
=== 2 -- identifiers:IdWeight[IdWeight(0x0000f9b6, 6.8288493f0), IdWeight(0x00015bfc, 6.891539f0), IdWeight(0x000080fd, 7.030101f0), IdWeight(0x0001534c, 7.301273f0), IdWeight(0x000066a1, 7.85197f0), IdWeight(0x0000a779, 8.320555f0), IdWeight(0x00000307, 8.374487f0), IdWeight(0x0000fee4, 8.527804f0), IdWeight(0x00013c46, 8.647624f0), IdWeight(0x00002cc2, 8.679995f0)]
=== 9 -- identifiers:IdWeight[IdWeight(0x0001778a, 5.571958f0), IdWeight(0x00017934, 6.118064f0), IdWeight(0x000056c5, 6.317239f0), IdWeight(0x0000d4a1, 6.4048085f0), IdWeight(0x0000cc8e, 6.5301356f0), IdWeight(0x00004d68, 6.7262073f0), IdWeight(0x00013e8d, 6.735403f0), IdWeight(0x0000afab, 7.560175f0), IdWeight(0x00006c7f, 7.578321f0), IdWeight(0x0000b04a, 8.011299f0)]
=== 23 -- identifiers:IdWeight[IdWeight(0x00007b9a, 2.5394197f0), IdWeight(0x0000a209, 2.905683f0), IdWeight(0x000121e2, 2.9237483f0), IdWeight(0x00008fe3, 3.4052174f0), IdWeight(0x0000cddd, 3.5397158f0), IdWeight(0x00014e81, 4.1171403f0), IdWeight(0x0001805e, 4.145492f0), IdWeight(0x0000f98a, 4.1695886f0), IdWeight(0x000057ca, 4.2271905f0), IdWeight(0x00003862, 4.251796f0)]
=== 7 -- identifiers:IdWeight[IdWeight(0x00009d78, 3.5820096f0), IdWeight(0x0000257d, 4.370799f0), IdWeight(0x00010017, 4.6067004f0), IdWeight(0x0000dca0, 4.69836f0), IdWeight(0x000118ad, 4.802515f0), IdWeight(0x0000d212, 4.8357935f0), IdWeight(0x0001779a, 5.011992f0), IdWeight(0x000025a3, 5.0458636f0), IdWeight(0x0001713f, 5.076128f0), IdWeight(0x00014ce3, 5.5756445f0)]
=== 8 -- identifiers:IdWeight[IdWeight(0x00006e2b, 3.0433564f0), IdWeight(0x00009bb3, 3.3318112f0), IdWeight(0x0000cd92, 3.7610397f0), IdWeight(0x00000165, 3.90755f0), IdWeight(0x000130a6, 4.0177884f0), IdWeight(0x00017a04, 4.163202f0), IdWeight(0x000057b7, 4.3299117f0), IdWeight(0x00001e74, 4.3771844f0), IdWeight(0x00008b59, 4.3801303f0), IdWeight(0x0000a688, 4.3804317f0)]
=== 28 -- identifiers:IdWeight[IdWeight(0x0000527f, 4.530012f0), IdWeight(0x000000bc, 5.515502f0), IdWeight(0x00006ca3, 5.545234f0), IdWeight(0x0000ca92, 5.5717845f0), IdWeight(0x00006b09, 5.626886f0), IdWeight(0x0000eb0d, 5.6659746f0), IdWeight(0x00003612, 5.763579f0), IdWeight(0x000128f2, 5.8228397f0), IdWeight(0x0000dfd2, 5.8642907f0), IdWeight(0x00007765, 5.9204035f0)]
=== 12 -- identifiers:IdWeight[IdWeight(0x00007e99, 5.232687f0), IdWeight(0x00009213, 5.4548883f0), IdWeight(0x00003847, 5.5799065f0), IdWeight(0x000098b6, 5.674197f0), IdWeight(0x000049be, 5.8413153f0), IdWeight(0x0001069e, 5.9631867f0), IdWeight(0x0000c190, 5.973559f0), IdWeight(0x00016230, 6.1401963f0), IdWeight(0x0000f28a, 6.173872f0), IdWeight(0x0000418b, 6.268172f0)]
=== 10 -- identifiers:IdWeight[IdWeight(0x0000fe5d, 7.1279845f0), IdWeight(0x0000eb72, 7.6273737f0), IdWeight(0x0000c7b9, 8.354789f0), IdWeight(0x00016f5b, 8.671972f0), IdWeight(0x000152ea, 9.068197f0), IdWeight(0x000026e5, 9.246054f0), IdWeight(0x00005e52, 9.471013f0), IdWeight(0x000054f8, 10.126999f0), IdWeight(0x000105d6, 10.231085f0), IdWeight(0x000098de, 10.311321f0)]
=== 30 -- identifiers:IdWeight[IdWeight(0x0000c4a0, 3.047554f0), IdWeight(0x000165ac, 3.065462f0), IdWeight(0x00003efe, 3.508696f0), IdWeight(0x0001758a, 3.91457f0), IdWeight(0x00005003, 4.1052475f0), IdWeight(0x00011d3b, 4.380745f0), IdWeight(0x0000daad, 4.5479016f0), IdWeight(0x00017966, 4.8038964f0), IdWeight(0x0000caa7, 4.816485f0), IdWeight(0x0000680b, 4.827343f0)]
=== 6 -- identifiers:IdWeight[IdWeight(0x000118bf, 4.038556f0), IdWeight(0x0000baea, 6.57015f0), IdWeight(0x000008e0, 7.129302f0), IdWeight(0x000178e4, 7.193729f0), IdWeight(0x00011fa6, 7.254283f0), IdWeight(0x00007fcf, 7.290628f0), IdWeight(0x0000b426, 7.707114f0), IdWeight(0x000014a6, 7.8612556f0), IdWeight(0x00000414, 7.9704895f0), IdWeight(0x00009b67, 7.97945f0)]
=== 25 -- identifiers:IdWeight[IdWeight(0x0000083b, 2.6822302f0), IdWeight(0x000062f5, 3.4622371f0), IdWeight(0x0000339d, 3.473967f0), IdWeight(0x000147ab, 3.6882677f0), IdWeight(0x000009da, 3.7322772f0), IdWeight(0x00012156, 3.752961f0), IdWeight(0x00013551, 3.7754538f0), IdWeight(0x000126eb, 3.80498f0), IdWeight(0x00006613, 3.8093596f0), IdWeight(0x000111c4, 3.8456643f0)]
=== 29 -- identifiers:IdWeight[IdWeight(0x0000a290, 5.2792125f0), IdWeight(0x000082e2, 6.2465644f0), IdWeight(0x00012a73, 6.36628f0), IdWeight(0x000104cd, 6.4137673f0), IdWeight(0x00008685, 6.467493f0), IdWeight(0x0000beff, 6.672741f0), IdWeight(0x00014529, 6.7045493f0), IdWeight(0x0000b3f2, 7.1916842f0), IdWeight(0x00001157, 7.236992f0), IdWeight(0x00011323, 7.3473973f0)]
=== 4 -- identifiers:IdWeight[IdWeight(0x0000b32d, 3.2068803f0), IdWeight(0x00006779, 4.1885586f0), IdWeight(0x00007a3a, 4.882552f0), IdWeight(0x00014814, 4.9450045f0), IdWeight(0x0000e40b, 5.3147616f0), IdWeight(0x0000c85e, 5.6501484f0), IdWeight(0x0000a4a5, 5.699605f0), IdWeight(0x000021ce, 5.800843f0), IdWeight(0x00007606, 5.985986f0), IdWeight(0x000085d2, 6.112025f0)]
=== 26 -- identifiers:IdWeight[IdWeight(0x00000ff4, 5.687612f0), IdWeight(0x000086cd, 5.834033f0), IdWeight(0x0001189a, 5.8992977f0), IdWeight(0x0000d54c, 6.2110023f0), IdWeight(0x00006a7e, 6.269251f0), IdWeight(0x00004236, 6.3195744f0), IdWeight(0x00013269, 6.4085903f0), IdWeight(0x00014113, 6.5205083f0), IdWeight(0x00001f82, 6.882298f0), IdWeight(0x00009281, 6.8878827f0)]
=== 3 -- identifiers:IdWeight[IdWeight(0x000140bc, 4.0632343f0), IdWeight(0x000030a4, 4.0751514f0), IdWeight(0x00010539, 4.5265074f0), IdWeight(0x0001618b, 5.203409f0), IdWeight(0x0001048b, 5.2985888f0), IdWeight(0x0000a539, 5.338418f0), IdWeight(0x0000622d, 5.59543f0), IdWeight(0x00007504, 5.601231f0), IdWeight(0x00016a5c, 6.0236363f0), IdWeight(0x000009ef, 6.0780196f0)]
=== 19 -- identifiers:IdWeight[IdWeight(0x00006d0c, 4.363042f0), IdWeight(0x000009c3, 5.530006f0), IdWeight(0x000151f7, 6.2204013f0), IdWeight(0x00001269, 6.285384f0), IdWeight(0x00003112, 6.423761f0), IdWeight(0x0000e36e, 6.5526094f0), IdWeight(0x0000f849, 6.797353f0), IdWeight(0x000082c1, 6.897246f0), IdWeight(0x00007eab, 6.8972917f0), IdWeight(0x00017f90, 6.9653015f0)]
=== 20 -- identifiers:IdWeight[IdWeight(0x0000cd58, 4.528069f0), IdWeight(0x0000bc5f, 4.689286f0), IdWeight(0x000177f6, 5.6855125f0), IdWeight(0x000090a2, 6.0342574f0), IdWeight(0x00005c29, 6.218595f0), IdWeight(0x0001492d, 6.4577646f0), IdWeight(0x0000ee86, 6.4820256f0), IdWeight(0x0000d766, 6.558745f0), IdWeight(0x000000fd, 6.6225147f0), IdWeight(0x00000996, 6.811672f0)]
=== 27 -- identifiers:UInt32[0x00007b9a, 0x0000a209, 0x000121e2, 0x00008fe3, 0x0000cddd, 0x00014e81, 0x0001805e, 0x0000f98a, 0x000057ca, 0x00003862]
=== 7 -- distances:UInt32[0x0001277e, 0x00003feb, 0x0000b894, 0x000008f9, 0x00016b3d, 0x00001839, 0x00005846, 0x00002ea3, 0x00002c44, 0x00008d45]
=== 11 -- distances:UInt32[0x0000083b, 0x000062f5, 0x0000339d, 0x000147ab, 0x000009da, 0x00012156, 0x00013551, 0x000126eb, 0x00006613, 0x000111c4]
=== 29 -- distances:UInt32[0x0000ef27, 0x0000afc0, 0x00000c0a, 0x000158a3, 0x00008272, 0x000172a9, 0x000060bf, 0x000021b0, 0x000110d8, 0x000085d1]
=== 24 -- distances:UInt32[0x00012021, 0x00002bee, 0x00017f60, 0x00003067, 0x00016933, 0x0000f909, 0x0001222b, 0x00001893, 0x00004463, 0x00006bce]
=== 16 -- distances:UInt32[0x00007e99, 0x00009213, 0x00003847, 0x000098b6, 0x000049be, 0x0001069e, 0x0000c190, 0x00016230, 0x0000f28a, 0x0000418b]
=== 10 -- distances:UInt32[0x00006d0c, 0x000009c3, 0x000151f7, 0x00001269, 0x00003112, 0x0000e36e, 0x0000f849, 0x000082c1, 0x00007eab, 0x00017f90]
=== 20 -- distances:UInt32[0x0000fe5d, 0x0000eb72, 0x0000c7b9, 0x00016f5b, 0x000152ea, 0x000026e5, 0x00005e52, 0x000054f8, 0x000105d6, 0x000098de]
=== 30 -- distances:UInt32[0x0000b0ba, 0x00002460, 0x000042f1, 0x0001123c, 0x00001b1f, 0x0000a461, 0x00015f4f, 0x000096ce, 0x000072bf, 0x000181bc]
=== 22 -- distances:UInt32[0x0000cd58, 0x0000bc5f, 0x000177f6, 0x000090a2, 0x00005c29, 0x0001492d, 0x0000ee86, 0x0000d766, 0x000000fd, 0x00000996]
=== 27 -- distances:UInt32[0x000067ba, 0x00007add, 0x00008246, 0x00009276, 0x00014fab, 0x00014e37, 0x0001083c, 0x0000dfa0, 0x0001255f, 0x0000d796]
=== 18 -- distances:UInt32[0x00004736, 0x0000a9af, 0x00012c60, 0x0001315a, 0x00012a0a, 0x0000d076, 0x000170b5, 0x0001697b, 0x0000b399, 0x0000594a]
=== 17 -- distances:UInt32[0x0000c4a0, 0x000165ac, 0x00003efe, 0x0001758a, 0x00005003, 0x00011d3b, 0x0000daad, 0x00017966, 0x0000caa7, 0x0000680b]
=== 6 -- distances:IdWeight[IdWeight(0x000150f8, 3.9342616f0), IdWeight(0x000084d2, 4.1718383f0), IdWeight(0x0000b885, 5.519068f0), IdWeight(0x00016c45, 6.1695642f0), IdWeight(0x00016dfc, 6.542225f0), IdWeight(0x00008d2d, 6.8909006f0), IdWeight(0x0000fc9c, 6.907578f0), IdWeight(0x00008c8f, 7.01647f0), IdWeight(0x0001223f, 7.148072f0), IdWeight(0x00012cbc, 7.1572924f0)]
=== 13 -- identifiers:UInt32[0x000118bf, 0x0000baea, 0x000008e0, 0x000178e4, 0x00011fa6, 0x00007fcf, 0x0000b426, 0x000014a6, 0x00000414, 0x00009b67]
=== 25 -- distances:UInt32[0x0000be28, 0x0000e60e, 0x00000393, 0x0000788a, 0x0000702a, 0x00007fde, 0x0000264c, 0x0001653a, 0x0000c7f5, 0x00007381]
=== 14 -- distances:UInt32[0x00004d1a, 0x00016d33, 0x0001804d, 0x00013024, 0x00002896, 0x0001727d, 0x0000f13c, 0x0000054c, 0x0000f70a, 0x000034b9]
=== 1 -- distances:UInt32[0x00000ff4, 0x000086cd, 0x0001189a, 0x0000d54c, 0x00006a7e, 0x00004236, 0x00013269, 0x00014113, 0x00001f82, 0x00009281]
=== 3 -- distances:UInt32[0x00006e2b, 0x00009bb3, 0x0000cd92, 0x00000165, 0x000130a6, 0x00017a04, 0x000057b7, 0x00001e74, 0x00008b59, 0x0000a688]
=== 28 -- distances:UInt32[0x00009d78, 0x0000257d, 0x00010017, 0x0000dca0, 0x000118ad, 0x0000d212, 0x0001779a, 0x000025a3, 0x0001713f, 0x00014ce3]
=== 8 -- distances:UInt32[0x000140bc, 0x000030a4, 0x00010539, 0x0001618b, 0x0001048b, 0x0000a539, 0x0000622d, 0x00007504, 0x00016a5c, 0x000009ef]
=== 19 -- distances:UInt32[0x0000f274, 0x0001405f, 0x000004ce, 0x000112b2, 0x0000832b, 0x000043f0, 0x00013d5a, 0x000185d1, 0x00008b03, 0x0001672d]
=== 2 -- distances:UInt32[0x0000f9b6, 0x00015bfc, 0x000080fd, 0x0001534c, 0x000066a1, 0x0000a779, 0x00000307, 0x0000fee4, 0x00013c46, 0x00002cc2]
=== 9 -- distances:UInt32[0x000150f8, 0x000084d2, 0x0000b885, 0x00016c45, 0x00016dfc, 0x00008d2d, 0x0000fc9c, 0x00008c8f, 0x0001223f, 0x00012cbc]
=== 13 -- distances:UInt32[0x0001778a, 0x00017934, 0x000056c5, 0x0000d4a1, 0x0000cc8e, 0x00004d68, 0x00013e8d, 0x0000afab, 0x00006c7f, 0x0000b04a]
=== 23 -- distances:UInt32[0x0000b32d, 0x00006779, 0x00007a3a, 0x00014814, 0x0000e40b, 0x0000c85e, 0x0000a4a5, 0x000021ce, 0x00007606, 0x000085d2]
=== 26 -- distances:UInt32[0x0000a290, 0x000082e2, 0x00012a73, 0x000104cd, 0x00008685, 0x0000beff, 0x00014529, 0x0000b3f2, 0x00001157, 0x00011323]
=== 4 -- distances:UInt32[0x00002ce3, 0x0000202e, 0x0001762e, 0x00014114, 0x0000d64f, 0x0001364f, 0x00004b95, 0x00013560, 0x00006af8, 0x00004010]
=== 21 -- distances:UInt32[0x00009da0, 0x000061a6, 0x00011464, 0x0001068c, 0x00012719, 0x000079be, 0x00016ac6, 0x00000cc5, 0x000115de, 0x00010a2e]
=== 15 -- distances:UInt32[0x0000527f, 0x000000bc, 0x00006ca3, 0x0000ca92, 0x00006b09, 0x0000eb0d, 0x00003612, 0x000128f2, 0x0000dfd2, 0x00007765]
=== 12 -- distances:UInt32[0x00013579, 0x0000dbdc, 0x00005e87, 0x00000624, 0x00016492, 0x0000f4ce, 0x00005d3a, 0x0000f096, 0x00010555, 0x0000e257]
=== 5 -- distances:Float32[2.5394197, 2.905683, 2.9237483, 3.4052174, 3.5397158, 4.1171403, 4.145492, 4.1695886, 4.2271905, 4.251796]
Float32[5.2792125, 6.2465644, 6.36628, 6.4137673, 6.467493, 6.672741, 6.7045493, 7.1916842, 7.236992, 7.3473973]
Float32[3.5820096, 4.370799, 4.6067004, 4.69836, 4.802515, 4.8357935, 5.011992, 5.0458636, 5.076128, 5.5756445]
Float32[3.0433564, 3.3318112, 3.7610397, 3.90755, 4.0177884, 4.163202, 4.3299117, 4.3771844, 4.3801303, 4.3804317]
Float32[2.2960021, 2.8062224, 3.5092864, 3.6508331, 3.6789362, 3.7138393, 3.8067896, 3.9141073, 4.0175414, 4.1544557]
Float32[4.726558, 5.0613613, 5.0821133, 5.130314, 5.1979394, 5.204519, 5.3080554, 5.5862293, 5.617485, 5.620654]
Float32[4.530012, 5.515502, 5.545234, 5.5717845, 5.626886, 5.6659746, 5.763579, 5.8228397, 5.8642907, 5.9204035]
Float32[6.675923, 7.02988, 7.1213803, 7.567528, 8.113082, 8.482317, 8.860224, 8.981895, 9.03268, 9.121991]
Float32[4.528069, 4.689286, 5.6855125, 6.0342574, 6.218595, 6.4577646, 6.4820256, 6.558745, 6.6225147, 6.811672]
Float32[7.1279845, 7.6273737, 8.354789, 8.671972, 9.068197, 9.246054, 9.471013, 10.126999, 10.231085, 10.311321]
Float32[4.0632343, 4.0751514, 4.5265074, 5.203409, 5.2985888, 5.338418, 5.59543, 5.601231, 6.0236363, 6.0780196]
Float32[6.292833, 7.7584715, 8.022731, 8.082609, 8.101276, 8.145304, 8.16899, 8.294217, 8.562826, 8.667862]
Float32[6.8288493, 6.891539, 7.030101, 7.301273, 7.85197, 8.320555, 8.374487, 8.527804, 8.647624, 8.679995]
Float32[3.3135195, 3.3848624, 4.025727, 4.077698, 4.366047, 4.375115, 4.681094, 4.71946, 4.9946475, 5.0513253]
Float32[5.232687, 5.4548883, 5.5799065, 5.674197, 5.8413153, 5.9631867, 5.973559, 6.1401963, 6.173872, 6.268172]
Float32[3.2068803, 4.1885586, 4.882552, 4.9450045, 5.3147616, 5.6501484, 5.699605, 5.800843, 5.985986, 6.112025]
Float32[4.038556, 6.57015, 7.129302, 7.193729, 7.254283, 7.290628, 7.707114, 7.8612556, 7.9704895, 7.97945]
Float32[5.571958, 6.118064, 6.317239, 6.4048085, 6.5301356, 6.7262073, 6.735403, 7.560175, 7.578321, 8.011299]
Float32[3.9342616, 4.1718383, 5.519068, 6.1695642, 6.542225, 6.8909006, 6.907578, 7.01647, 7.148072, 7.1572924]
Float32[5.9874015, 6.110808, 6.878881, 7.056838, 7.3161545, 7.4862347, 7.9547477, 8.335855, 8.767622, 8.830698]
Float32[3.4584124, 4.4805098, 4.483524, 4.515228, 4.5867596, 4.593042, 4.647873, 4.8091617, 4.95529, 4.956506]
Float32[3.063364, 3.3474534, 3.7191494, 4.0377135, 4.174683, 4.3115225, 4.5141444, 4.5508885, 4.661785, 4.676373]
Float32[2.8870392, 4.5670953, 5.4409337, 5.633697, 5.681482, 5.6852694, 5.819864, 5.8213906, 5.8465, 5.860892]
Float32[4.363042, 5.530006, 6.2204013, 6.285384, 6.423761, 6.5526094, 6.797353, 6.897246, 6.8972917, 6.9653015]
Float32[3.047554, 3.065462, 3.508696, 3.91457, 4.1052475, 4.380745, 4.5479016, 4.8038964, 4.816485, 4.827343]
Float32[4.406932, 4.557166, 4.912068, 4.91214, 4.965306, 4.981351, 4.991246, 5.1663833, 5.600113, 5.64502]
Float32[5.687612, 5.834033, 5.8992977, 6.2110023, 6.269251, 6.3195744, 6.4085903, 6.5205083, 6.882298, 6.8878827]
Float32[2.6875844, 4.135638, 4.8320765, 5.0121684, 5.0521636, 5.074626, 5.080174, 5.138646, 5.2589445, 5.5174394]
Float32[6.6390896, 6.6492963, 7.055795, 7.0637984, 7.361882, 7.6028204, 7.618756, 7.7091656, 7.9337654, 8.160997]
Float32[2.6822302, 3.4622371, 3.473967, 3.6882677, 3.7322772, 3.752961, 3.7754538, 3.80498, 3.8093596, 3.8456643]

Environment and dependencies

Julia Version 1.10.9
Commit 5595d20a287 (2025-03-10 12:51 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 64 × Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, cascadelake)
Threads: 64 default, 0 interactive, 32 GC (on 64 virtual cores)
Environment:
  JULIA_PROJECT = .
  JULIA_NUM_THREADS = auto
  JULIA_LOAD_PATH = @:@stdlib
Status `~/sites/SimilaritySearchDemos/Project.toml`
  [aaaa29a8] Clustering v0.15.8
  [944b1d66] CodecZlib v0.7.8
  [a93c6f00] DataFrames v1.7.0
  [c5bfea45] Embeddings v0.4.6
  [f67ccb44] HDF5 v0.17.2
  [b20bd276] InvertedFiles v0.8.0 `~/.julia/dev/InvertedFiles`
  [682c06a0] JSON v0.21.4
  [23fbe1c1] Latexify v0.16.6
  [eb30cadb] MLDatasets v0.7.18
  [06eb3307] ManifoldLearning v0.9.0
⌃ [ca7969ec] PlotlyLight v0.11.0
  [91a5bcdd] Plots v1.40.11
  [27ebfcd6] Primes v0.5.7
  [ca7ab67e] SimSearchManifoldLearning v0.3.0 `~/.julia/dev/SimSearchManifoldLearning`
  [053f045d] SimilaritySearch v0.12.0 `~/.julia/dev/SimilaritySearch`
⌅ [2913bbd2] StatsBase v0.33.21
  [f3b207a7] StatsPlots v0.15.7
  [7f6f6c8a] TextSearch v0.19.0 `~/.julia/dev/TextSearch`
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated`