Tier 2 API Reference

RocksDBException(msg)

Thrown whenever a RocksDB C API call reports an error (i.e. sets a non-NULL errptr).

db = RocksDB.opendb("mydb")
close(db)
try
    RocksDB.opendb("mydb"; create_if_missing = false, error_if_exists = true)
catch e
    e isa RocksDB.RocksDBException && println("failed: ", e.msg)
end

Options(; kwargs...)

Wraps rocksdb_options_t with defaults that work well for most uses: zstd compression, a 10-bits-per-key bloom filter, an 8 MiB block cache, and create_if_missing = true.

Keyword arguments: create_if_missing, error_if_exists, create_missing_column_families (like create_if_missing, but for column families named in opendb's column_families list that don't exist yet), compression (a Symbol in keys(RocksDB.COMPRESSION_TYPES) or an integer compression code), max_open_files, max_background_jobs, write_buffer_size, bloom_bits_per_key (set to nothing to disable the bloom filter), block_cache_size (bytes; set to nothing to disable the block cache), env, parallelism (passed to rocksdb_options_increase_parallelism).

Examples

opts = RocksDB.Options(; compression = :lz4, max_open_files = 256, bloom_bits_per_key = nothing)
db = RocksDB.opendb("mydb"; options = opts)
close(db)

Most of the time you don't need to build an Options yourself at all – opendb/RocksDBDict forward their keyword arguments straight into one for you:

db = RocksDB.opendb("mydb2"; compression = :lz4, max_open_files = 256)
close(db)

Cache(capacity_bytes)

An LRU block cache. You don't normally construct this yourself: passing block_cache_size to Options (or to opendb/ RocksDBDict, which forward it) builds one of these internally and attaches it for you.

opts = RocksDB.Options(; block_cache_size = 64 * 1024^2)  # what block_cache_size builds
opts.cache            # the Cache RocksDB.Options just created

There is currently no public way to attach a Cache you constructed yourself to more than one Options (each Options always builds its own); this type is exposed mainly so opts.cache above is a real, documented object rather than an opaque internal detail.

FilterPolicy(; bits_per_key=10.0)

A bloom filter policy, used to skip reads that would otherwise touch disk for a key that isn't present. As with Cache, you don't normally construct this yourself: Options(; bloom_bits_per_key=...) builds one internally.

opts = RocksDB.Options(; bloom_bits_per_key = 10.0)  # what bloom_bits_per_key builds
opts.filter           # the FilterPolicy RocksDB.Options just created

Pass bloom_bits_per_key = nothing to Options to disable the bloom filter entirely (opts.filter is then nothing).

Unlike Cache (which is stored as a shared_ptr copy, safe to destroy our handle to independently of RocksDB's use of it), a FilterPolicy handed to rocksdb_block_based_options_set_filter_policy has its ownership transferred: RocksDB does options.filter_policy.reset(ptr), taking sole ownership of that exact C++ object. So once a FilterPolicy has been attached to a table's options, our finalizer must NOT also destroy it (that would be a double free) – the owned field tracks this internally.

Env(; background_threads=nothing, high_priority_background_threads=nothing)

RocksDB's OS-facing environment: background compaction/flush thread pool sizing, primarily. Unlike Cache/FilterPolicy, you can build one yourself and share it across several Options (and therefore several databases), which is the main reason to construct one directly rather than leaving it to Options's defaults.

env = RocksDB.Env(; background_threads = 4)
opts = RocksDB.Options(; env = env)
db = RocksDB.opendb("mydb"; options = opts)
close(db)

DB

A handle to an open RocksDB database. Construct with opendb; close with close (or let the finalizer do it). Supports the do-block form:

RocksDB.opendb(path) do db
    put!(db, "k", "v")
end

opendb(path; kwargs...) -> DB
opendb(f, path; kwargs...)

Open (creating if needed, by default) the RocksDB database at path. Keyword arguments not related to column families are forwarded to Options unless an options::Options is given directly. Pass column_families = ["default", "other", ...] to open multiple column families at once (all opened with the same Options unless you pass options yourself). Pass read_only=true to open an existing database without acquiring the (exclusive) write lock – useful for inspecting a database another process still has open for writing.

Pass ttl to open a database whose entries expire after roughly that many seconds. Either a single number of seconds (applied to "default", or to every column family if column_families is given), or (only together with column_families) a vector of per-column-family TTLs, same length and order as column_families. read_only cannot be combined with ttl – RocksDB's C API has no such combination.

Expiry is enforced only during compaction, not on every read: get and iteration can still return an entry after its TTL has elapsed, until some compaction happens to run over it (RocksDB makes no promise about when that is – non-positive/absent ttl means "never expires", not "as soon as possible"). Force it deterministically with compact! if you need to observe expiry without waiting for a background compaction.

A database ever opened with ttl must always be reopened with ttl (any value, even a different one – RocksDB allows changing it across opens) afterwards: entries carry a hidden 4-byte timestamp that a ttl-less open would otherwise expose as if it were part of the value.

The do-block form guarantees the database is closed even if an exception is thrown.

put!(db, key, value; cf=nothing, options=db.default_write_options)

key and value may be any AbstractString or byte vector. cf is nothing (the default column family), a ColumnFamily, or the column family's name as a String.

Examples

db = RocksDB.opendb("mydb")
put!(db, "hello", "world")
String(get(db, "hello"))    # "world"
haskey(db, "hello")         # true
delete!(db, "hello")
get(db, "hello")            # nothing
close(db)

get(db, key; cf=nothing, options=db.default_read_options) -> Union{Vector{UInt8},Nothing}

Returns nothing if key doesn't exist. See put! for a combined put!/get/delete!/haskey example.

delete!(db, key; cf=nothing, options=db.default_write_options)

Deleting a key that doesn't exist is not an error. See put! for a combined example.

haskey(db::DB, key) -> Bool

Note: unlike put!/get/delete!, this method does not accept a cf/options keyword – it always checks the default column family with the default read options. To check a specific column family, call get(db, key; cf=...) !== nothing directly (this is exactly what RocksDBDict's own haskey does).

WriteOptions(; sync=false)

Options for a single put!/delete!/write! call. sync=true waits for the write to be flushed to disk (fsync) before returning – slower, but safe across a power loss/crash; the default (false) only guarantees the write is visible to the process, not yet durable on disk.

db = RocksDB.opendb("mydb")
put!(db, "critical", "value"; options = RocksDB.WriteOptions(; sync = true))
close(db)

ReadOptions(; snapshot=nothing)

Options for a single get/DBIterator read. Its main use is pairing with Snapshot for a point-in-time-consistent read, unaffected by writes made after the snapshot:

db = RocksDB.opendb("mydb")
put!(db, "k", "before")
snap = RocksDB.Snapshot(db)
put!(db, "k", "after")

ro = RocksDB.ReadOptions(; snapshot = snap)
String(get(db, "k"; options = ro))   # "before"
String(get(db, "k"))                 # "after" -- default options see the live DB
close(db)

WriteBatch()

An in-memory batch of writes; apply it atomically to a DB with write! – either all of the batch's writes take effect, or (if the process crashes/errors before write! returns) none of them do.

db = RocksDB.opendb("mydb")
wb = RocksDB.WriteBatch()
put!(wb, "a", "1")
put!(wb, "b", "2")
delete!(wb, "a")
write!(db, wb)              # "a" and "b" appear/disappear together
String(get(db, "b"))        # "2"
get(db, "a")                # nothing

empty!(wb)                  # reuse the same batch for a second round of writes
close(db)

batch(f, db) (documented below, alongside write!) is a do-block convenience that creates the batch, runs f on it, and commits with write! automatically (only if f returns normally).

put!(batch::WriteBatch, key, value; cf=nothing)

Stage a write in batch (not applied until write!). See WriteBatch for a full example.

delete!(batch::WriteBatch, key; cf=nothing)

Stage a deletion in batch (not applied until write!). See WriteBatch for a full example.

empty!(batch::WriteBatch)

Discard all staged writes/deletes in batch, so it can be reused.

write!(db, batch; options=db.default_write_options)

Atomically apply batch to db.

batch(f, db::DB; options=db.default_write_options)

do-block convenience: create a WriteBatch, run f on it, and write! it to db – only if f returns normally. If f throws, the batch is discarded and nothing is written (same all-or-nothing guarantee as building a WriteBatch by hand). The RocksDBDict equivalent is RocksDB.batch(f, dict).

db = RocksDB.opendb("mydb")
RocksDB.batch(db) do wb
    put!(wb, "a", "1")
    put!(wb, "b", "2")
end
String(get(db, "a"))   # "1"
close(db)

DBIterator(db; cf=nothing, options=ReadOptions())

Iterates (key, value) pairs (as Vector{UInt8}) in ascending key order. Pass a snapshot-bound options (see Snapshot) for a point-in-time-consistent scan.

db = RocksDB.opendb("mydb")
put!(db, "b", "2"); put!(db, "a", "1"); put!(db, "c", "3")
for (k, v) in RocksDB.DBIterator(db)
    println(String(k), " => ", String(v))   # a => 1, b => 2, c => 3, in that order
end
close(db)

For manual control (seeking to a specific key, range/prefix scans, etc.) rather than iterating from the start, use seek_to_first!/ seek_to_last!/seek! with valid/key/ value directly – this is exactly what DBIterator's own Base.iterate method is built from. For example, a prefix scan:

db = RocksDB.opendb("mydb2")
put!(db, "user:1", "a"); put!(db, "user:2", "b"); put!(db, "other", "c")

it = RocksDB.DBIterator(db)
RocksDB.seek!(it, "user:")   # jump straight to the first key >= "user:"
while RocksDB.valid(it) && startswith(String(RocksDB.key(it)), "user:")
    println(String(RocksDB.key(it)), " => ", String(RocksDB.value(it)))
    RocksDB.advance!(it)
end
close(db)

seek_to_first!(it::DBIterator)

Position it at the first (smallest) key. Returns it.

seek_to_last!(it::DBIterator)

Position it at the last (largest) key. Returns it.

seek!(it::DBIterator, key)

Position it at the first key greater than or equal to key (key need not exist) – the standard way to start a range/prefix scan. Returns it.

advance!(it::DBIterator)

Move it to the next key in ascending order. Returns it. Check valid afterward before calling key/value.

valid(it::DBIterator) -> Bool

Whether it is currently positioned at a real entry. false after running off either end (via repeated advance!) or if it was never positioned (call seek_to_first!/seek_to_last!/ seek! first).

key(it::DBIterator) -> Vector{UInt8}

The key at it's current position. Only valid when valid(it).

value(it::DBIterator) -> Vector{UInt8}

The value at it's current position. Only valid when valid(it).

ColumnFamily

A handle to one of a DB's open column families, returned by create_column_family and stored in db.column_families (see also column_families). Pass it (or just the column family name as a String) as the cf keyword to put!/get/ delete!/DBIterator.

db = RocksDB.opendb("mydb")
cf = create_column_family(db, "users")
put!(db, "alice", "admin"; cf = cf)
String(get(db, "alice"; cf = cf))
close(db)

create_column_family(db, name; options=db.options) -> ColumnFamily

Create (and register on db) a new column family.

db = RocksDB.opendb("mydb")
cf = create_column_family(db, "users")
put!(db, "alice", "admin"; cf = cf)
close(db)

To reopen a database that already has extra column families, pass column_families=["default", "users", ...] to opendb listing every one of them (RocksDB requires this) – see opendb's docstring.

column_families(db::DB) -> Dict{String,ColumnFamily}

The column families currently open on db (always includes "default").

Snapshot(db)

Snapshot(f, db)

A consistent point-in-time read view. Release it (or let the finalizer do so) when done; pass it to ReadOptions(; snapshot=...) for reads that should be unaffected by writes made after the snapshot. RocksDB.RocksDBSnapshotView wraps this (plus a ReadOptions) into a full read-only AbstractDict, which most users want instead of using Snapshot directly.

db = RocksDB.opendb("mydb")
put!(db, "k", "before")
snap = RocksDB.Snapshot(db)
put!(db, "k", "after")
String(get(db, "k"; options = RocksDB.ReadOptions(; snapshot = snap)))   # "before"
close(db)

The do-block form releases the snapshot automatically, even if the block throws:

db = RocksDB.opendb("mydb2")
put!(db, "k", "before")
RocksDB.Snapshot(db) do snap
    put!(db, "k", "after")
    ro = RocksDB.ReadOptions(; snapshot = snap)
    String(get(db, "k"; options = ro))   # "before"
end
close(db)

property(db, name; cf=nothing) -> Union{String,Nothing}

Read a RocksDB "property" – a string-valued statistic/introspection value RocksDB tracks internally, scoped to cf (nothing meaning the default column family, as elsewhere in this package). RocksDB tracks these per column family, so pass the right cf or you'll silently get the default column family's value instead. "rocksdb.estimate-num-keys" (an approximate key count) is the one this package's Base.length methods use internally; see the RocksDB property reference for the full list (e.g. "rocksdb.stats", "rocksdb.num-files-at-level0").

db = RocksDB.opendb("mydb"; column_families = ["default", "users"], create_missing_column_families = true)
put!(db, "a", "1")
put!(db, "alice", "admin"; cf = "users")
property(db, "rocksdb.estimate-num-keys")             # "1" -- default column family only
property(db, "rocksdb.estimate-num-keys"; cf = "users")  # "1" -- the "users" column family
property(db, "rocksdb.no-such-property")               # nothing
close(db)

compact!(db; cf=nothing, from=nothing, to=nothing, exclusive=nothing,
         change_level=nothing, target_level=nothing,
         bottommost_level_compaction=nothing, max_subcompactions=nothing)

Force a manual compaction over [from, to] (both nothing means the whole key range), scoped to cf (nothing meaning the default column family, as elsewhere in this package). Useful whenever you need compaction-driven effects to happen now rather than whenever RocksDB would otherwise get to them – most notably to observe opendb's ttl option actually remove expired entries, since expiry is enforced only during compaction.

The keyword arguments besides cf/from/to map directly to RocksDB's CompactRangeOptions and are only built into one (at a small extra cost) if any of them is given; leave them all nothing for the common case.

db = RocksDB.opendb(mktempdir() * "/mydb"; ttl = 1)
put!(db, "a", "1")
sleep(2)
compact!(db)
get(db, "a")   # nothing -- expired and swept by the compaction
close(db)

BackupEngine(backup_dir; kwargs...)

A handle to a RocksDB backup engine rooted at backup_dir. Construct with BackupEngine, close with close (or let the finalizer do it); supports the do-block form. Use create_backup! to take backups of a DB, backup_info/verify_backup to inspect them, and restore_backup!/restore_latest_backup! to restore one back into a database directory.

Keyword arguments (all optional, matching RocksDB's own defaults except where noted): env (an Env for the backup engine to use, e.g. one with rate-limited background threads; defaults to a fresh, plain Env if not given – RocksDB's backup engine requires one, unlike opendb where it's optional), share_table_files (default true), sync (default true), backup_log_files (default true), backup_rate_limit, restore_rate_limit, max_background_operations, max_valid_backups_to_open, share_files_with_checksum_naming, destroy_old_data.

Examples

db = RocksDB.opendb(mktempdir() * "/mydb")
put!(db, "a", "1")

be = RocksDB.BackupEngine(mktempdir() * "/backups")
create_backup!(be, db)
close(db)
close(be)

BackupEngine(f, backup_dir; kwargs...)

do-block form: opens the backup engine, runs f, and closes it even if f throws.

db = RocksDB.opendb(mktempdir() * "/mydb2")
put!(db, "a", "1")
RocksDB.BackupEngine(mktempdir() * "/backups2") do be
    create_backup!(be, db)
end
close(db)

BackupInfo

One backup's metadata, as returned by backup_info: backup_id, timestamp (seconds since epoch), size (bytes), and num_files.

create_backup!(be::BackupEngine, db::DB; flush_before_backup=false)

Take a new backup of db into be's backup directory.

db = RocksDB.opendb(mktempdir() * "/mydb3")
put!(db, "a", "1")
be = RocksDB.BackupEngine(mktempdir() * "/backups3")
create_backup!(be, db)
length(backup_info(be))   # 1
close(db)
close(be)

backup_info(be::BackupEngine) -> Vector{BackupInfo}

The metadata (BackupInfo) of every backup currently in be's backup directory, oldest first.

db = RocksDB.opendb(mktempdir() * "/mydb4")
put!(db, "a", "1")
be = RocksDB.BackupEngine(mktempdir() * "/backups4")
create_backup!(be, db)
infos = backup_info(be)
infos[1].backup_id   # 1
close(db)
close(be)

verify_backup(be::BackupEngine, backup_id::Integer)

Verify that the backup identified by backup_id is valid (all its files are present and checksums match). Throws RocksDBException if not; returns nothing on success.

db = RocksDB.opendb(mktempdir() * "/mydb5")
put!(db, "a", "1")
be = RocksDB.BackupEngine(mktempdir() * "/backups5")
create_backup!(be, db)
verify_backup(be, 1)   # nothing -- valid
close(db)
close(be)

purge_old_backups!(be::BackupEngine, num_backups_to_keep::Integer)

Delete all but the num_backups_to_keep most recent backups.

db = RocksDB.opendb(mktempdir() * "/mydb6")
put!(db, "a", "1")
be = RocksDB.BackupEngine(mktempdir() * "/backups6")
create_backup!(be, db)
put!(db, "b", "2")
create_backup!(be, db)
purge_old_backups!(be, 1)
length(backup_info(be))   # 1
close(db)
close(be)

restore_backup!(be::BackupEngine, backup_id::Integer, db_dir; wal_dir=db_dir, keep_log_files=false)

Restore the backup identified by backup_id into db_dir (and wal_dir for its write-ahead log), overwriting whatever is already there. Does not require a DB to already be open on db_dir.

src = RocksDB.opendb(mktempdir() * "/mydb7")
put!(src, "a", "1")
be = RocksDB.BackupEngine(mktempdir() * "/backups7")
create_backup!(be, src)
close(src)

restored = mktempdir() * "/restored7"
restore_backup!(be, 1, restored)
close(be)

db2 = RocksDB.opendb(restored; create_if_missing = false)
String(get(db2, "a"))   # "1"
close(db2)

restore_latest_backup!(be::BackupEngine, db_dir; wal_dir=db_dir, keep_log_files=false)

Restore the most recent backup into db_dir (and wal_dir for its write-ahead log), overwriting whatever is already there. Does not require a DB to already be open on db_dir.

src = RocksDB.opendb(mktempdir() * "/mydb8")
put!(src, "a", "1")
be = RocksDB.BackupEngine(mktempdir() * "/backups8")
create_backup!(be, src)
close(src)

restored = mktempdir() * "/restored8"
restore_latest_backup!(be, restored)
close(be)

db2 = RocksDB.opendb(restored; create_if_missing = false)
String(get(db2, "a"))   # "1"
close(db2)