Uploaded image for project: 'Fabric'
  1. Fabric
  2. FAB-15865

block/tx indexing should be optimized to improve performance

    XMLWordPrintable

Details

    • Task
    • Status: Closed
    • Medium
    • Resolution: Duplicate
    • v1.4.0
    • None
    • fabric-ledger

    Description

      With the current fabric implementation, committing peer will receive new blocks from orderer or other peers in a fabric network. After receiving new blocks, committers will verify the new blocks and transactions mainly in two steps:

      • 1). Verify tx envelope, signature, endorsement policy, txId, txID duplicate in the same block(if this capacity is enabled) and txIds of already committed txs. This step verification is mostly done with multiple goroutines now. The result of duplicate txIds check is stored in block metadata
      • 2). MVCC check; save Block into filesystem; Store Block/tx indexes; etc... Indexing txIds in this step will check duplicate txIds again by querying committed txIds in goleveldb instead of reusing the txIds duplicate check result in step 1. code here

      Checking duplicate txIds by querying goleveldb sometimes is not efficient, especially when there are a lot of, say 2000, txs in a single Block. That means we need to do 2000 queries against goleveldb to verify duplicate txIds, assuming there's no duplicate txId in the same Block. There's no need to hit goleveldb in step 2 as we can simply get the duplicate check result from block metadata in step 1.

      I did a preliminary test to verify the impact of duplicate check in step 2 and the results are listed below.
      maxMessageCount: 2000
      1. markDuplicateTxids by querying goleveldb
      txs_per_block:2000,blocks:1313,block_commit_total:314ms,state_validation:99ms,block_commit:214ms,state_commit:0ms
      2. markDuplicateTxids by reusing the block metadata
      txs_per_block:2000,blocks:1368,block_commit_total:239ms,state_validation:101ms,block_commit:137ms,state_commit:0ms

      The duration metrics were extracted from log messages printed by fabric code. The elapsed time for block commit is decreased by ~24%. And the txs committing rate on the committer is increased by ~23%-33% with sufficient cpu resource.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              xixuejia Xi Xue Jia
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: