Uploaded image for project: 'Indy Node'
  1. Indy Node
  2. INDY-1205

Use RocksDB as a key-value storage

    XMLWordPrintable

Details

    • Story
    • Status: Complete
    • Medium
    • Resolution: Done
    • None
    • None
    • None
    • None
    • Sprint 18.05, 18.06

    Description

      Taken into account our agreement about refactoring with breaking changes, I believe we have a good time for migration from LevelDB to RocksDB because of the following:

      1. Migration to RocksDB requires migration (modification) of the ledger. We are going to migrate the ledger in any case (hopefully the last time) for refactoring, so we can combine it with RocksDB support, so no more migrations are needed in future.
      2. I thing migration to RocksDB should not take much time, since Dmitry did a PoC, and everything worked. We just need to merge the old branch rocksdb-integration.
      3. We have new resources for this work.
      4. It will fix bugs like INDY-1182, since it's possible to access RocksDB from other process in read only mode (it's not possible for Leveldb, that's why read_ledger script has to create a tmp copy of the ledger which is ugly, will not work for big ledgers, and has some problems for NFS mounted systems).
      5. RocksDB has snapshots support, which can help us in more efficient ledger catch-up.
      6. RocksDB works well on Windows

      What needs to be done:

      • Implement RocksDB implementation of key-value storage
      • Integrate RocksDB into CI and CD
      • Create migration script from leveldb to rocksdb
      • Adapt `read_ledger` script to work with RocksDB and avoid creation of the ledger's copy

      Attachments

        Issue Links

          Activity

            People

              VladimirWork Vladimir Shishkin
              ashcherbakov Alexander Shcherbakov
              Alexander Shcherbakov, Sergey Shilov, Vladimir Shishkin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: