Public API Change

• Support dynamically change delete_obsolete_files_period_micros option via SetDBOptions().
• Added EventListener::OnExternalFileIngested which will be called when IngestExternalFile() add a file successfully.
• BackupEngine::Open and BackupEngineReadOnly::Open now always return error statuses matching those of the backup Env.

Bug Fixes

• Fix the bug that if 2PC is enabled, checkpoints may loss some recent transactions.
• When file copying is needed when creating checkpoints or bulk loading files, fsync the file after the file copying.

Public API Change

• Options::max_bytes_for_level_multiplier is now a double along with all getters and setters.
• Support dynamically change delayed_write_rate and max_total_wal_size options via SetDBOptions().
• Introduce DB::DeleteRange for optimized deletion of large ranges of contiguous keys.
• Support dynamically change delayed_write_rate option via SetDBOptions().
• Options::allow_concurrent_memtable_write and Options::enable_write_thread_adaptive_yield are now true by default.
• Remove Tickers::SEQUENCE_NUMBER to avoid confusion if statistics object is shared among RocksDB instance. Alternatively DB::GetLatestSequenceNumber() can be used to get the same value.
• Options.level0_stop_writes_trigger default value changes from 24 to 32.
• New compaction filter API: CompactionFilter::FilterV2(). Allows to drop ranges of keys.
• Removed flashcache support.
• DB::AddFile() is deprecated and is replaced with DB::IngestExternalFile(). DB::IngestExternalFile() remove all the restrictions that existed for DB::AddFile.

New Features

• Add avoid_flush_during_shutdown option, which speeds up DB shutdown by not flushing unpersisted data (i.e. with disableWAL = true). Unpersisted data will be lost. The options is dynamically changeable via SetDBOptions().
• Add memtable_insert_with_hint_prefix_extractor option. The option is mean to reduce CPU usage for inserting keys into memtable, if keys can be group by prefix and insert for each prefix are sequential or almost sequential. See include/rocksdb/options.h for more details.
• Add LuaCompactionFilter in utilities. This allows developers to write compaction filters in Lua. To use this feature, LUA_PATH needs to be set to the root directory of Lua.
• No longer populate “LATEST_BACKUP” file in backup directory, which formerly contained the number of the latest backup. The latest backup can be determined by finding the highest numbered file in the “meta/” subdirectory.

We abandoned release candidates 4.10.x and directly go to 4.11.2 from 4.9, to make sure the latest release is stable. In 4.11.2, we fixed several data corruption related bugs introduced in 4.9.0.

4.11.2 (9/15/2016)

Bug fixes

• Segfault when failing to open an SST file for read-ahead iterators.
• WAL without data for all CFs is not deleted after recovery.

4.8.0 (5/2/2016)

Public API Change

• Allow preset compression dictionary for improved compression of block-based tables. This is supported for zlib, zstd, and lz4. The compression dictionary’s size is configurable via CompressionOptions::max_dict_bytes.
• Delete deprecated classes for creating backups (BackupableDB) and restoring from backups (RestoreBackupableDB). Now, BackupEngine should be used for creating backups, and BackupEngineReadOnly should be used for restorations. For more details, see https://github.com/facebook/rocksdb/wiki/How-to-backup-RocksDB%3F
• Expose estimate of per-level compression ratio via DB property: “rocksdb.compression-ratio-at-levelN”.
• Added EventListener::OnTableFileCreationStarted. EventListener::OnTableFileCreated will be called on failure case. User can check creation status via TableFileCreationInfo::status.

New Features

• Add ReadOptions::readahead_size. If non-zero, NewIterator will create a new table reader which performs reads of the given size.

4.5.1 (3/25/2016)

Bug Fixes

•  Fix failures caused by the destorying order of singleton objects.

4.5.0 (2/5/2016)

Public API Changes

• Add a new perf context level between kEnableCount and kEnableTime. Level 2 now does not include timers for mutexes.
• Statistics of mutex operation durations will not be measured by default. If you want to have them enabled, you need to set Statistics::stats_level_ to kAll.
• DBOptions::delete_scheduler and NewDeleteScheduler() are removed, please use DBOptions::sst_file_manager and NewSstFileManager() instead

New Features

• ldb tool now supports operations to non-default column families.
• Add kPersistedTier to ReadTier. This option allows Get and MultiGet to read only the persited data and skip mem-tables if writes were done with disableWAL = true.
• Add DBOptions::sst_file_manager. Use NewSstFileManager() in include/rocksdb/sst_file_manager.h to create a SstFileManager that can be used to track the total size of SST files and control the SST files deletion rate.

In RocksDB 4.3, we added a new set of features that makes managing RocksDB options easier. Specifically:

• Persisting Options Automatically: Each RocksDB database will now automatically persist its current set of options into an INI file on every successful call of DB::Open(), SetOptions(), and CreateColumnFamily() / DropColumnFamily().

• Load Options from File: We added LoadLatestOptions() / LoadOptionsFromFile() that enables developers to construct RocksDB options object from an options file.

• Sanity Check Options: We added CheckOptionsCompatibility that performs compatibility check on two sets of RocksDB options.

RocksDB developers are doing a Reddit Ask-Me-Anything now at 10AM – 11AM PDT! We welcome you to stop by and ask any RocksDB related questions, including existing / upcoming features, tuning tips, or database design.

Here are some enhancements that we’d like to focus on over the next six months:

• 2-Phase Commit
• Lua support in some custom functions
• Backup and repair tools
• Direct I/O to bypass OS cache
• RocksDB Java API

https://www.reddit.com/r/IAmA/comments/47k1si/we_are_rocksdb_developers_ask_us_anything/

New RocksDB release - 4.2!

New Features

1. Introduce CreateLoggerFromOptions(), this function create a Logger for provided DBOptions.

2. Add GetAggregatedIntProperty(), which returns the sum of the GetIntProperty of all the column families.

3. Add MemoryUtil in rocksdb/utilities/memory.h. It currently offers a way to get the memory usage by type from a list rocksdb instances.

The most popular compaction style of RocksDB is level-based compaction, which is an improved version of LevelDB’s compaction algorithm. Page 9- 16 of this slides gives an illustrated introduction of this compaction style. The basic idea that: data is organized by multiple levels with exponential increasing target size. Except a special level 0, every level is key-range partitioned into many files. When size of a level exceeds its target size, we pick one or more of its files, and merge the file into the next level.

In many use cases of RocksDB, people rely on OS page cache for caching compressed data. With this approach, verifying effective of the OS page caching is challenging, because file system is a black box to users.

As an example, a user can tune the DB as following: use level-based compaction, with L1 - L4 sizes to be 1GB, 10GB, 100GB and 1TB. And they reserve about 20GB memory as OS page cache, expecting level 0, 1 and 2 are mostly cached in memory, leaving only reads from level 3 and 4 requiring disk I/Os. However, in practice, it’s not easy to verify whether OS page cache does exactly what we expect. For example, if we end up with doing 4 instead of 2 I/Os per query, it’s not easy for users to figure out whether the it’s because of efficiency of OS page cache or reading multiple blocks for a level. Analysis like it is especially important if users run RocksDB on hard drive disks, for the gap of latency between hard drives and memory is much higher than flash-based SSDs.

Checkpoint is a feature in RocksDB which provides the ability to take a snapshot of a running RocksDB database in a separate directory. Checkpoints can be used as a point in time snapshot, which can be opened Read-only to query rows as of the point in time or as a Writeable snapshot by opening it Read-Write. Checkpoints can be used for both full and incremental backups.

We recently added a new API, called GetThreadList(), that exposes the RocksDB background thread activity. With this feature, developers will be able to obtain the real-time information about the currently running compactions and flushes such as the input / output size, elapsed time, the number of bytes it has written. Below is an example output of GetThreadList. To better illustrate the example, we have put a sample output of GetThreadList into a table where each column represents a thread status:

In this article, we follow up on the first part of an answer to one of the questions in our AMA, the dynamic level size in level-based compaction.

Over the past 6 months we have seen a number of use cases where RocksDB is successfully used by the community and various companies to achieve high throughput and volume in a modern server environment.

We at Microsoft Bing could not be left behind. As a result we are happy to announce the availability of the Windows Port created here at Microsoft which we intend to use as a storage option for one of our key/value data stores.

About a year ago, there was a need to develop a spatial database at Facebook. We needed to store and index Earth’s map data. Before building our own, we looked at the existing spatial databases. They were all very good technology, but also general purpose. We could sacrifice a general-purpose API, so we thought we could build a more performant database, since it would be specifically designed for our use-case. Furthermore, we decided to build the spatial database on top of RocksDB, because we have a lot of operational experience with running and tuning RocksDB at a large scale.

Every 6 months, RocksDB team gets together to prioritize the work ahead of us. We just went through this exercise and we wanted to share the results with the community. Here’s what RocksDB team will be focusing on for the next 6 months:

Check out this blog post by Mike Arpaia and Ted Reed about how osquery leverages RocksDB to build an embedded pub-sub system. This article is a great read and contains insights on how to properly use RocksDB.

Over the last couple of years, we have been busy integrating RocksDB with various services here at Facebook that needed to store key-value pairs locally. We have also seen other companies using RocksDB as local storage components of their distributed systems.

RocksDB can be used as a storage engine of a higher level database. In fact, we are currently plugging RocksDB into MySQL and MongoDB as one of their storage engines. RocksDB can help with guaranteeing some of the ACID properties: durability is guaranteed by RocksDB by design; while consistency and isolation need to be enforced by concurrency controls on top of RocksDB; Atomicity can be implemented by committing a transaction’s writes with one write batch to RocksDB in the end.

RocksDB options can be provided using a file or any string to RocksDB. The format is straightforward: write_buffer_size=1024;max_write_buffer_number=2. Any whitespace around = and ; is OK. Moreover, options can be nested as necessary. For example BlockBasedTableOptions can be nested as follows: write_buffer_size=1024; max_write_buffer_number=2; block_based_table_factory={block_size=4k};. Similarly any white space around { or } is ok. Here is what it looks like in code:

If you have an existing application that uses LevelDB and would like to migrate to using RocksDB, one problem you need to overcome is to map the options for LevelDB to proper options for RocksDB. As of release 3.9 this can be automatically done by using our option conversion utility found in rocksdb/utilities/leveldb_options.h. What is needed, is to first replace leveldb::Options with rocksdb::LevelDBOptions. Then, use rocksdb::ConvertOptions( ) to convert the LevelDBOptions struct into appropriate RocksDB options. Here is an example:

New RocksDB release - 3.5!

New Features

1. Add include/utilities/write_batch_with_index.h, providing a utility class to query data out of WriteBatch when building it.

2. new ReadOptions.total_order_seek to force total order seek when block-based table is built with hash index.

Introduction

In this post, we are introducing “full filter block” — a new bloom filter format for block based table. This could bring about 40% of improvement for key query under in-memory (all data stored in memory, files stored in tmpfs/ramfs, an example workload. The main idea behind is to generate a big filter that covers all the keys in SST file to avoid lots of unnecessary memory look ups.

Introduction

We recently introduced a new Cuckoo Hashing based SST file format which is optimized for fast point lookups. The new format was built for applications which require very high point lookup rates (~4Mqps) in read only mode but do not use operations like range scan, merge operator, etc. But, the existing RocksDB file formats were built to support range scan and other operations and the current best point lookup in RocksDB is 1.2 Mqps given by PlainTable format. This prompted a hashing based file format, which we present here. The new table format uses a cache friendly version of Cuckoo Hashing algorithm with only 1 or 2 memory accesses per lookup.

Check out new RocksDB release on GitHub!

New Features in RocksDB 3.3:

• JSON API prototype.

• Performance improvement on HashLinkList: We addressed performance outlier of HashLinkList caused by skewed bucket by switching data in the bucket from linked list to skip list. Add parameter threshold_use_skiplist in NewHashLinkListRepFactory().

Check out new RocksDB release on GitHub!

New Features in RocksDB 3.2:

• PlainTable now supports a new key encoding: for keys of the same prefix, the prefix is only written once. It can be enabled through encoding_type paramter of NewPlainTableFactory()

• Add AdaptiveTableFactory, which is used to convert from a DB of PlainTable to BlockBasedTabe, or vise versa. It can be created using NewAdaptiveTableFactory()

As promised in the previous blog post!

RocksDB employs a multiversion concurrency control strategy. Before reading data, it needs to grab the current version, which is encapsulated in a data structure called SuperVersion.

In this post, we are introducing “PlainTable” – a file format we designed for RocksDB, initially to satisfy a production use case at Facebook.

Design goals:

1. All data stored in memory, in files stored in tmpfs/ramfs. Support DBs larger than 100GB (may be sharded across multiple RocksDB instance).
2. Optimize for prefix hashing
3. Less than or around 1 micro-second average latency for single Get() or Seek().
4. Minimize memory consumption.
5. Queries efficiently return empty results

Check out the new release on Github!

New features in RocksDB 3.1:

• Materialized hash index

• FIFO compaction style

We released 3.1 so fast after 3.0 because one of our internal customers needed materialized hash index.

Check out new RocksDB release on Github!

New features in RocksDB 3.0:

• Column Family support

• Ability to chose different checksum function

• Deprecated ReadOptions::prefix_seek and ReadOptions::prefix

In this post, we briefly introduce the recent improvements we did to RocksDB to improve the issue of lock contention costs.

RocksDB has a simple thread synchronization mechanism (See RocksDB Architecture Guide  to understand terms used below, like SST tables or mem tables). SST tables are immutable after being written and mem tables are lock-free data structures supporting single writer and multiple readers. There is only one single major lock, the DB mutex (DBImpl.mutex_) protecting all the meta operations, including:

For a Get() request, RocksDB goes through mutable memtable, list of immutable memtables, and SST files to look up the target key. SST files are organized in levels.

On level 0, files are sorted based on the time they are flushed. Their key range (as defined by FileMetaData.smallest and FileMetaData.largest) are mostly overlapped with each other. So it needs to look up every L0 file.

Check out the new RocksDB 2.8 release on Github.

RocksDB 2.8. is mostly focused on improving performance for in-memory workloads. We are seeing read QPS as high as 5M (we will write a separate blog post on this).

On Mar 27, 2014, RocksDB team @ Facebook held the 1st RocksDB local meetup in FB HQ (Menlo Park, California). We invited around 80 guests from 20+ local companies, including LinkedIn, Twitter, Dropbox, Square, Pinterest, MapR, Microsoft and IBM. Finally around 50 guests showed up, totaling around 60% show-up rate.

In recent months, we have focused on optimizing RocksDB for in-memory workloads. With growing RAM sizes and strict low-latency requirements, lots of applications decide to keep their entire data in memory. Running in-memory database with RocksDB is easy – just mount your RocksDB directory on tmpfs or ramfs [1]. Even if the process crashes, RocksDB can recover all of your data from in-memory filesystem. However, what happens if the machine reboots?

In RocksDB, we have implemented an easy way to backup your DB. Here is a simple example:

1
2
3
4
5
6
7
8
9
10

#include "rocksdb/db.h"
#include "utilities/backupable_db.h"
using namespace rocksdb;

DB* db;
DB::Open(Options(), "/tmp/rocksdb", &db);
BackupableDB* backupable_db = new BackupableDB(db, BackupableDBOptions("/tmp/rocksdb_backup"));
backupable_db->Put(...); // do your thing
backupable_db->CreateNewBackup();
delete backupable_db; // no need to also delete db