Uploaded image for project: 'Indy Node'
  1. Indy Node
  2. INDY-760

Pool is broken after several running of load_test.py

    Details

    • Type: Bug
    • Status: Complete
    • Priority: Medium
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Environment:

      Build Info:
        indy-node 1.0.111
        indy-anoncreds 1.0.25
        indy-plenum 1.0.97
        sovrin 1.0.23
      OS/Platform: Ubuntu 16.04.2 LTS
      Setup: performance pool (7 nodes, 2 clients)

    • Sprint:
      12

      Description

      Steps to Reproduce:

      1. Connect to pool.
      2. Using two clients, run load script several times:
      Client 1 (all tests are running one by one):
      python3 load_test.py --clients-list load_test_clients.list -t NYM -c 10 -r 100 --at-once
      python3 load_test.py --clients-list load_test_clients.list -t NYM -c 1 -r 500 --at-once
      python3 load_test.py --clients-list load_test_clients.list -t NYM -c 1 -r 500 --at-once
      python3 load_test.py --clients-list load_test_clients.list -t NYM -c 10 -r 300 --at-once
      python3 load_test_inv.py --clients-list load_test_clients.list -t NYM -c 5 -r 100 --at-once

      Client 2 in parallel with load_test_inv.py on client 1:
      python3 load_test.py --clients-list load_test_clients.list -t NYM -c 10 -r 100 --at-once
      =>test was failed because of InvalidMessageExceedingSizeException)

      Invalid test from client 1 and valid test from client 2 are stopped, new test on client 2:
      python3 load_test.py --clients-list load_test_clients.list -t NYM -c 10 -r 100 --at-once

      Actual Results:
      Pool is broken after these actions. Following messages repeatedly appear in the log:

      2017-08-24 10:42:37,450 | DEBUG | replica.py ( 311) | last_ordered_3pc | Node1:1 set last ordered as (0, 0)
      2017-08-24 10:42:37,450 | DEBUG | replica.py ( 953) | __is_next_pre_prepare | Node1:2 missing PRE-PREPAREs between 10 and 8
      2017-08-24 10:42:37,450 | DEBUG | replica.py ( 495) | _setup_for_non_master | Node1:2 Setting last ordered for non-master as (0, 0)
      2017-08-24 10:42:37,450 | DEBUG | replica.py ( 311) | last_ordered_3pc | Node1:2 set last ordered as (0, 0)
      2017-08-24 10:42:37,463 | DEBUG | replica.py ( 953) | __is_next_pre_prepare | Node1:1 missing PRE-PREPAREs between 6 and 4
      2017-08-24 10:42:37,464 | DEBUG | replica.py ( 495) | _setup_for_non_master | Node1:1 Setting last ordered for non-master as (0, 0)

      Expected Result:
      Pool should work.

      Additional Information:
      Count of transactions is equal for ledgers on all nodes:

      ubuntu@californiaPerf1:/home/sovrin/.sovrin$ sudo read_ledger --type=domain --count
      1406

        Attachments

        1. Node1.7z
          3.15 MB
        2. Node2.7z
          2.93 MB
        3. Node3.7z
          3.00 MB
        4. Node4.7z
          3.24 MB
        5. Node5.7z
          3.49 MB
        6. Node6.7z
          3.45 MB
        7. Node7.7z
          3.37 MB

          Issue Links

            Activity

              People

              • Assignee:
                danielhardman Daniel Hardman
                Reporter:
                ozheregelya Olga Zheregelya
                Watchers:
                Alexander Shcherbakov, Andrey Goncharov, Olga Zheregelya
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: