Uploaded image for project: 'Fabric'
  1. Fabric
  2. FAB-18297

CouchDB retry not sending information to invoking chaincode

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Medium
    • Resolution: Invalid: Test Error
    • None
    • None
    • fabric-ledger
    • Hide
      Using caliper 0.4.1, and the available caliper samples, run the `get-asset` benchmark (https://github.com/hyperledger/caliper-benchmarks/tree/master/benchmarks/api/fabric/contract), having deployed goLang "fixed-asset" contract.

      Observe failure under load.

      Using a limited resource machine, and the configuration:

        workers:
          type: local
          number: 5
        rateControl:
          type: fixed-load
          opts:
              transactionLoad: 200

      The error is 100% reproducible
      Show
      Using caliper 0.4.1, and the available caliper samples, run the `get-asset` benchmark ( https://github.com/hyperledger/caliper-benchmarks/tree/master/benchmarks/api/fabric/contract), having deployed goLang "fixed-asset" contract. Observe failure under load. Using a limited resource machine, and the configuration:   workers:     type: local     number: 5   rateControl:     type: fixed-load     opts:         transactionLoad: 200 The error is 100% reproducible

    Description

      Scenario

      • go contract performing getState operation via bytes, err := ctx.GetStub().GetState(uuid) 
      • CouchDB world state
      • 2org network (v2 peers, one peer per org)
      • High continuous load scenario, driven via Caliper

       

      Actual Outcome
      Failures occur because returned bytes are empty and err is nil. This is following a CouchDB timeout error, resulting in a peer retry, but no information going back to the invoking chaincode.

      Flow appears to be:
      1) CouchDB error (taken from couchDB logs)

      [error] 2020-10-27T13:36:55.542041Z nonode@nohost <0.21077.235> 1baec15e75 rexi_server: from: nonode@nohost(<0.21935.233>) mfa: fabric_rpc:open_doc/3 exit:{timeout,{gen_server,call,[<0.21014.233>,{{

      {pread_iolist,2360084}

      }}] [{gen_server,call,2,[}}{{

      {file,"gen_server.erl"}

      }},{line,206}]},{couch_file,pread_iolist,2,[{{

      {file,"src/couch_file.erl"},{line,177}]},{couch_file,pread_binary,2,[{file,"src/couch_file.erl"}

      }},{line,172}]},{couch_file,pread_term,2,[{{

      {file,"src/couch_file.erl"}

      }},{line,160}]},{couch_btree,get_node,2,[{{

      {file,"src/couch_btree.erl"},{line,435}]},{couch_btree,lookup,3,[{file,"src/couch_btree.erl"}

      }},{line,286}]},{couch_btree,lookup,2,[{{

      {file,"src/couch_btree.erl"}

      }},{line,276}]},{couch_bt_engine,open_docs,2,[{{

      {file,"src/couch_bt_engine.erl"}

      }},{line,399}]}]


      2) Peer warning (taken from peer logs)

      {{ 2020-10-27 13:36:55.543 UTC [couchdb] handleRequest -> WARN 8a66e Attempt 1 of 4 returned Couch DB Error:error,  Status Code:500 Internal Server Error  Reason:read_failure. Retrying couchdb request in 125ms}}

      3) Chaincode continues execution with no error, and no content returned from GetState call. Result is an unmarshal operation will fail, and if appropriate logging is put in place, you will observe:

      {{ Error performing json.Unmarshal: unexpected end of JSON input}}
      {{ Error performing json.Unmarshal on bytes:}}

      ie, an error is thrown, and if trying to log out the returned bytes, nothing is present.

      Expected Outcome

      Either:
      a) Result of couchDB retry is sent to invoking chaincode
      b) Chaincode is sent an error

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            nkl199 Nick Lincoln
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: