Uploaded image for project: 'Indy Node'
  1. Indy Node
  2. INDY-911

Pool stopped taking transactions after sending 1,000 simultaneous transactions

    Details

    • Type: Bug
    • Status: Complete
    • Priority: High
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Epic Link:
    • Sprint:
      INDY 17.24: Node Perf, INDY 17.25

      Description

      I have a 7 node pool.
      I have an additional 5 machines setup running libindy with some performance generating scripts.

      On each of the 5 machines the script runs 200 threads sending 30 transactions each one at a time. So across the 5 machines I am sending 1,000 transactions at a time.

      The pool received and processed about 8 transactions and then stopped taking any new transactions. I stopped the scripts and tried from the CLI which also failed to add a new transaction.

      The pool functioned properly after restarting the nodes.

      Each node in the pool shows the same error in its log at about the same time.
      commit failed for batch request, error IndexError('list index out of range'

      I have seen this error before under a load less than 1,000 at a time.

      I am including only a portion of one of the logs since their size is too big to attach. I have all 7 logs (223 MB) if anyone needs me to send them.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                krw910 Kelly Wilson
                Watchers:
                Kelly Wilson, Nikita Spivachuk, Olga Zheregelya
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: