2017-11-21 13:09:42,108 | INFO | node.py (2420) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 13:09:42,189 | INFO | node.py (2420) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 13:09:42,248 | INFO | plugin_loader.py ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-11-21 13:09:42,249 | DISPLAY | replicas.py ( 36) | grow | Node2 added replica Node2:0 to instance 0 (master) 2017-11-21 13:09:42,249 | DISPLAY | replicas.py ( 36) | grow | Node2 added replica Node2:1 to instance 1 (backup) 2017-11-21 13:09:42,326 | INFO | node.py (2420) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 13:09:42,328 | INFO | stacks.py ( 84) | start | CONNECTION: Node2 listening for other nodes at 0.0.0.0:9703 2017-11-21 13:09:42,329 | INFO | node.py ( 621) | start | Node2 first time running... 2017-11-21 13:09:42,333 | INFO | zstack.py ( 590) | connect | CONNECTION: Node2 looking for Node4 at 10.0.0.5:9707 2017-11-21 13:09:42,334 | INFO | zstack.py ( 590) | connect | CONNECTION: Node2 looking for Node1 at 10.0.0.2:9701 2017-11-21 13:09:42,334 | INFO | zstack.py ( 590) | connect | CONNECTION: Node2 looking for Node3 at 10.0.0.4:9705 2017-11-21 13:09:42,393 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node4 2017-11-21 13:09:42,488 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node3 2017-11-21 13:09:42,517 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 13:09:42,517 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 13:09:42,532 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 13:09:42,532 | INFO | upgrader.py ( 151) | should_notify_about_upgrade_result | Node's 'Node2' last upgrade txn is None 2017-11-21 13:09:42,533 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 13:09:42,549 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 13:09:42,549 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 13:09:42,549 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 13:09:42,551 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-11-21 13:09:42,582 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node2:0 selected primary Node1:0 for instance 0 (view 0) 2017-11-21 13:09:42,582 | INFO | node.py ( 510) | start_participating | Node2 started participating 2017-11-21 13:09:42,582 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node2:0 declares view change 0 as completed for instance 0, new primary is Node1:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')] 2017-11-21 13:09:42,582 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node2:1 selected primary Node2:1 for instance 1 (view 0) 2017-11-21 13:09:42,582 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node2:1 declares view change 0 as completed for instance 1, new primary is Node2:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')] 2017-11-21 13:30:22,975 | INFO | propagator.py ( 148) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511271022953271) from client b'MyAS5>/6=HGPt>CF{HKRU.&xuO$0^KZ!y>e}9RLl' 2017-11-21 13:30:23,022 | INFO | replica.py (1483) | order_3pc_key | Node2:0 ordered batch request, view no 0, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root ZNYrpy1fYTShLY4hAkJtafsNJvDd6gnzpC4UKBzVd3G, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511271022953271)], discarded [] 2017-11-21 13:30:23,023 | INFO | upgrader.py ( 292) | handleUpgradeTxn | Node 'Node2' handles upgrade txn {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS', 'txnTime': 1511271022, 'seqNo': 1, 'force': False, 'justification': None, 'timeout': 10, 'action': 'start', 'auditPath': [], 'reqId': 1511271022953271, 'name': 'upgrade_to_the_latest_full', 'rootHash': 'ZNYrpy1fYTShLY4hAkJtafsNJvDd6gnzpC4UKBzVd3G', 'reinstall': False, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'version': '1.2.47', 'type': '109', 'schedule': {'8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-11-21T13:40:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-11-21T13:35:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-11-21T13:45:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-11-21T13:50:00.258870+00:00'}} 2017-11-21 13:30:23,023 | INFO | upgrader.py ( 321) | handleUpgradeTxn | Node 'Node2' schedules upgrade to 1.2.47 2017-11-21 13:30:23,023 | INFO | upgrader.py ( 359) | _scheduleUpgrade | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb's upgrader processing upgrade for version 1.2.47 2017-11-21 13:30:23,024 | INFO | node.py (2290) | executeBatch | Node2 committed batch request, view no 0, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root ZNYrpy1fYTShLY4hAkJtafsNJvDd6gnzpC4UKBzVd3G, requests: [('V4SGRU86Z58d6TV7PBUe6f', 1511271022953271)] 2017-11-21 13:30:23,032 | INFO | replica.py (1483) | order_3pc_key | Node2:1 ordered batch request, view no 0, ppSeqNo 1, ledger 2, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511271022953271)], discarded [] 2017-11-21 13:35:00,043 | INFO | propagator.py ( 148) | propagate | Node2 propagating request ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271300031735) from client Node1 2017-11-21 13:35:00,077 | INFO | replica.py (1483) | order_3pc_key | Node2:0 ordered batch request, view no 0, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root GeSC6DwsVQaSWEVVoWjmCTGE1yAdUaq3c7nE1t1FyHfG, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271300031735)], discarded [] 2017-11-21 13:35:00,078 | INFO | replica.py (1483) | order_3pc_key | Node2:1 ordered batch request, view no 0, ppSeqNo 2, ledger 2, state root None, txn root None, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271300031735)], discarded [] 2017-11-21 13:35:00,080 | INFO | node.py (2290) | executeBatch | Node2 committed batch request, view no 0, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root GeSC6DwsVQaSWEVVoWjmCTGE1yAdUaq3c7nE1t1FyHfG, requests: [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271300031735)] 2017-11-21 13:35:23,226 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node1 2017-11-21 13:35:23,226 | INFO | node.py ( 823) | onConnsChanged | Node2 lost connection to primary of master 2017-11-21 13:35:25,234 | INFO | node.py (2048) | sendInstanceChange | VIEW CHANGE: Node2 sending an instance change with view_no 1 since Primary of master protocol instance disconnected 2017-11-21 13:35:25,234 | INFO | node.py (2051) | sendInstanceChange | MONITORING: Node2 metrics for monitor: Node2 Monitor metrics:: None Delta: 0.4 Lambda: 60 Omega: 5 instances started: [1724630.458310681, 1724630.458594913] ordered request counts: {0: 2, 1: 2} ordered request durations: {0: 0.053010181756690145, 1: 0.06101995985955} master request latencies: {} client avg request latencies: [{'V4SGRU86Z58d6TV7PBUe6f': (1, 0.03158000484108925), 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': (1, 0.021430176915600896)}, {'V4SGRU86Z58d6TV7PBUe6f': (1, 0.03943368582986295), 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': (1, 0.021586274029687047)}] throughput: {0: 37.728601067239126, 1: 32.77616053179012} master throughput: 37.728601067239126 total requests: 2 avg backup throughput: 32.77616053179012 master throughput ratio: 1.1510988613399533 2017-11-21 13:35:25,234 | INFO | node.py (2108) | propose_view_change | Node2 sent view change since was disconnected from primary for too long 2017-11-21 13:35:25,250 | INFO | node.py (1954) | do_view_change_if_possible | VIEW CHANGE: Node2 initiating a view change to 1 from 0 2017-11-21 13:35:25,251 | INFO | node.py (2159) | startViewChange | VIEW CHANGE: Node2 changed to view 1, will start catchup now 2017-11-21 13:35:25,252 | INFO | message_processor.py ( 29) | discard | Node2 discarding message INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} because Received instance change request with view no 1 which is not more than its view no 1 2017-11-21 13:35:25,271 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 13:35:25,272 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 13:35:25,289 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 13:35:25,290 | INFO | upgrader.py ( 214) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:35:25,290 | INFO | upgrader.py ( 292) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:35:25,290 | INFO | upgrader.py ( 321) | handleUpgradeTxn | Node 'Node2' schedules upgrade to 1.2.47 2017-11-21 13:35:25,290 | INFO | upgrader.py ( 326) | handleUpgradeTxn | Node 'Node2' cancels previous upgrade and schedules a new one to 1.2.47 2017-11-21 13:35:25,290 | INFO | upgrader.py ( 390) | _cancelScheduledUpgrade | Cancelling upgrade of node 'Node2' to version 1.2.47 due to some reason 2017-11-21 13:35:25,290 | INFO | upgrader.py ( 359) | _scheduleUpgrade | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb's upgrader processing upgrade for version 1.2.47 2017-11-21 13:35:25,291 | INFO | upgrader.py ( 151) | should_notify_about_upgrade_result | Node's 'Node2' last upgrade txn is None 2017-11-21 13:35:25,292 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 13:35:25,322 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 13:35:25,322 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 13:35:25,323 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 13:35:25,324 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node2:0 selected primary Node2:0 for instance 0 (view 1) 2017-11-21 13:35:25,325 | INFO | node.py ( 510) | start_participating | Node2 started participating 2017-11-21 13:35:25,325 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node2:0 declares view change 1 as completed for instance 0, new primary is Node2:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 2, 'GeSC6DwsVQaSWEVVoWjmCTGE1yAdUaq3c7nE1t1FyHfG')] 2017-11-21 13:35:25,325 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node2:1 selected primary Node3:1 for instance 1 (view 1) 2017-11-21 13:35:25,326 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node2:1 declares view change 1 as completed for instance 1, new primary is Node3:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 2, 'GeSC6DwsVQaSWEVVoWjmCTGE1yAdUaq3c7nE1t1FyHfG')] 2017-11-21 13:35:29,981 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-11-21 13:35:30,007 | INFO | propagator.py ( 148) | propagate | Node2 propagating request ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271329997860) from client Node1 2017-11-21 13:35:30,077 | INFO | replica.py (1483) | order_3pc_key | Node2:1 ordered batch request, view no 1, ppSeqNo 1, ledger 2, state root None, txn root None, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271329997860)], discarded [] 2017-11-21 13:35:30,080 | INFO | replica.py (1483) | order_3pc_key | Node2:0 ordered batch request, view no 1, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root Sufjfa1upHQkafvtKeThr49DC7inH3CCejhogJdUrhD, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271329997860)], discarded [] 2017-11-21 13:35:30,081 | INFO | node.py (2290) | executeBatch | Node2 committed batch request, view no 1, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root Sufjfa1upHQkafvtKeThr49DC7inH3CCejhogJdUrhD, requests: [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1511271329997860)] 2017-11-21 13:39:59,295 | INFO | upgrader.py ( 421) | _callUpgradeAgent | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb's upgrader calling agent for upgrade 2017-11-21 13:39:59,295 | INFO | node.py ( 254) | notify_upgrade_start | Node2 is about to be upgraded, sending NODE_UPGRADE 2017-11-21 13:39:59,298 | INFO | upgrader.py ( 433) | _sendUpdateRequest | Sending message to control tool: {"version": "1.2.47"} 2017-11-21 13:39:59,299 | INFO | upgrader.py ( 452) | _sendUpdateRequest | Waiting 10 minutes for upgrade to be performed 2017-11-21 13:39:59,325 | INFO | propagator.py ( 148) | propagate | Node2 propagating request ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271599296461) from client Node2 2017-11-21 13:39:59,328 | INFO | replica.py (1884) | enqueue_prepare | Node2:1 queueing prepare due to unavailability of PRE-PREPARE. Prepare PREPARE{'viewNo': 1, 'digest': '9f0f48514ee8f76fce83e7f180237e5b00b2f97a4a62bcb2b9f1600f949d2674', 'stateRootHash': None, 'txnRootHash': None, 'ppSeqNo': 2, 'ppTime': 1511271599, 'instId': 1} from Node1:1 2017-11-21 13:39:59,333 | INFO | replica.py (1483) | order_3pc_key | Node2:1 ordered batch request, view no 1, ppSeqNo 2, ledger 2, state root None, txn root None, requests ordered [('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271599296461)], discarded [] 2017-11-21 13:39:59,349 | INFO | replica.py (1483) | order_3pc_key | Node2:0 ordered batch request, view no 1, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root GJoBk7sZCyo7GfzTzL7PE7CEdMJUCGuDkFF7KqtG1vDB, requests ordered [('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271599296461)], discarded [] 2017-11-21 13:39:59,350 | INFO | node.py (2290) | executeBatch | Node2 committed batch request, view no 1, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root GJoBk7sZCyo7GfzTzL7PE7CEdMJUCGuDkFF7KqtG1vDB, requests: [('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271599296461)] 2017-11-21 13:40:28,571 | INFO | node.py (2503) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 13:40:28,573 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node1 ordered, NYM Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv 2017-11-21 13:40:28,574 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node2 ordered, NYM 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb 2017-11-21 13:40:28,574 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node3 ordered, NYM DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya 2017-11-21 13:40:28,574 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node4 ordered, NYM 4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA 2017-11-21 13:40:28,588 | WARNING | node.py ( 623) | _create_bls_bft | BLS: Transactions will not be BLS signed by this Node, since BLS keys were not found. Please make sure that a script to init BLS keys was called (init_bls_keys), and NODE txn was sent with BLS public keys. 2017-11-21 13:40:28,593 | INFO | node.py (2503) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 13:40:28,606 | INFO | node.py ( 479) | setPoolParams | Node2 updated its pool parameters: f 1, totalNodes 4, allNodeNames {'Node4', 'Node1', 'Node2', 'Node3'}, requiredNumberOfInstances 2, minimumNodes 3, quorums {'f': 1, 'consistency_proof': Quorum(2), 'reply': Quorum(2), 'checkpoint': Quorum(2), 'propagate_primary': Quorum(2), 'ledger_status': Quorum(2), 'commit': Quorum(3), 'bls_signatures': Quorum(3), 'prepare': Quorum(2), 'propagate': Quorum(2), 'timestamp': Quorum(2), 'view_change': Quorum(3), 'same_consistency_proof': Quorum(2), 'view_change_done': Quorum(3), 'election': Quorum(3)} 2017-11-21 13:40:28,643 | INFO | plugin_loader.py ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-11-21 13:40:28,644 | DISPLAY | replicas.py ( 40) | grow | Node2 added replica Node2:0 to instance 0 (master) 2017-11-21 13:40:28,645 | DISPLAY | replicas.py ( 40) | grow | Node2 added replica Node2:1 to instance 1 (backup) 2017-11-21 13:40:28,690 | INFO | upgrader.py ( 152) | _update_upgrade_log_for_started_upgrade | Node 'Node2' successfully upgraded to version 1.2.47 2017-11-21 13:40:28,690 | INFO | node.py (2503) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 13:40:28,691 | INFO | stacks.py ( 84) | start | CONNECTION: Node2 listening for other nodes at 0.0.0.0:9703 2017-11-21 13:40:28,692 | INFO | node.py ( 680) | start | Node2 first time running... 2017-11-21 13:40:28,693 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node1 at 10.0.0.2:9701 2017-11-21 13:40:28,694 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node4 at 10.0.0.5:9707 2017-11-21 13:40:28,694 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node3 at 10.0.0.4:9705 2017-11-21 13:40:28,747 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node4 2017-11-21 13:40:28,810 | INFO | node.py (2044) | _start_view_change_if_possible | VIEW CHANGE: Node2 starting view change for 2 after 2 view change indications from other nodes 2017-11-21 13:40:28,811 | INFO | node.py (2241) | startViewChange | VIEW CHANGE: Node2 changed to view 2, will start catchup now 2017-11-21 13:40:28,811 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node3 2017-11-21 13:40:28,838 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 13:40:28,838 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 13:40:28,839 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 13:40:28,839 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:40:28,839 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:40:28,840 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 13:40:28,840 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 13:40:28,840 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 13:40:28,840 | INFO | node.py (1588) | allLedgersCaughtUp | CATCH-UP: Node2 caught up till (1, 2) 2017-11-21 13:40:28,842 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 13:40:28,843 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:0 selected primary Node3:0 for instance 0 (view 2) 2017-11-21 13:40:28,843 | INFO | node.py ( 532) | start_participating | Node2 started participating 2017-11-21 13:40:28,843 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:0 declares view change 2 as completed for instance 0, new primary is Node3:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 4, 'GJoBk7sZCyo7GfzTzL7PE7CEdMJUCGuDkFF7KqtG1vDB')] 2017-11-21 13:40:28,843 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:1 selected primary Node4:1 for instance 1 (view 2) 2017-11-21 13:40:28,844 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:1 declares view change 2 as completed for instance 1, new primary is Node4:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 4, 'GJoBk7sZCyo7GfzTzL7PE7CEdMJUCGuDkFF7KqtG1vDB')] 2017-11-21 13:40:28,844 | INFO | node.py ( 242) | acknowledge_upgrade | Node2 found the first run after upgrade, sending NODE_UPGRADE complete to version 1.2.47 2017-11-21 13:40:28,845 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 13:40:28,845 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 13:40:28,846 | INFO | node.py (1685) | no_more_catchups_needed | Node2 starting to participate since catchup is done, primaries are selected but mode was not set to participating 2017-11-21 13:40:28,846 | INFO | node.py ( 532) | start_participating | Node2 started participating 2017-11-21 13:40:28,846 | INFO | message_processor.py ( 29) | discard | Node2 discarding message INSTANCE_CHANGE{'reason': 26, 'viewNo': 2} because Received instance change request with view no 2 which is not more than its view no 2 2017-11-21 13:40:28,849 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-11-21 13:40:28,867 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271628844985) from client Node2 2017-11-21 13:40:28,901 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 2, ppSeqNo 1, ledger 2, state root None, txn root None, requests ordered [('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271628844985)], discarded [] 2017-11-21 13:40:28,905 | INFO | replica.py (1619) | order_3pc_key | Node2:0 ordered batch request, view no 2, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 44Kh4HRALU5B6nY6yWJx8rhrcMf3BvHdu1sPScXEjiEo, requests ordered [('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271628844985)], discarded [] 2017-11-21 13:40:28,906 | INFO | node.py (2374) | executeBatch | Node2 committed batch request, view no 2, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 44Kh4HRALU5B6nY6yWJx8rhrcMf3BvHdu1sPScXEjiEo, requests: [('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1511271628844985)] 2017-11-21 13:44:59,279 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271899274670) from client Node3 2017-11-21 13:44:59,317 | INFO | replica.py (1619) | order_3pc_key | Node2:0 ordered batch request, view no 2, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 3ru9mawZCCRe7brebbogXMNBwceurrrDPC1FGvNkrsks, requests ordered [('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271899274670)], discarded [] 2017-11-21 13:44:59,317 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 2, ppSeqNo 2, ledger 2, state root None, txn root None, requests ordered [('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271899274670)], discarded [] 2017-11-21 13:44:59,319 | INFO | node.py (2374) | executeBatch | Node2 committed batch request, view no 2, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 3ru9mawZCCRe7brebbogXMNBwceurrrDPC1FGvNkrsks, requests: [('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271899274670)] 2017-11-21 13:45:42,720 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node3 2017-11-21 13:45:42,720 | INFO | node.py ( 884) | onConnsChanged | Node2 lost connection to primary of master 2017-11-21 13:45:44,721 | INFO | node.py (2128) | sendInstanceChange | VIEW CHANGE: Node2 sending an instance change with view_no 3 since Primary of master protocol instance disconnected 2017-11-21 13:45:44,721 | INFO | node.py (2131) | sendInstanceChange | MONITORING: Node2 metrics for monitor: Node2 Monitor metrics:: None Delta: 0.4 Lambda: 60 Omega: 5 instances started: [1726476.853379012, 1726476.85454919] ordered request counts: {0: 2, 1: 2} ordered request durations: {0: 0.06475956388749182, 1: 0.0603007550816983} master request latencies: {} client avg request latencies: [{'8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': (1, 0.03912408300675452), 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': (1, 0.025635480880737305)}, {'8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': (1, 0.03431820310652256), 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': (1, 0.02598255197517574)}] throughput: {0: 30.883469250574986, 1: 33.167080533076344} master throughput: 30.883469250574986 total requests: 2 avg backup throughput: 33.167080533076344 master throughput ratio: 0.9311482576760414 2017-11-21 13:45:44,722 | INFO | node.py (2191) | propose_view_change | Node2 sent view change since was disconnected from primary for too long 2017-11-21 13:45:44,734 | INFO | node.py (2031) | do_view_change_if_possible | VIEW CHANGE: Node2 initiating a view change to 3 from 2 2017-11-21 13:45:44,735 | INFO | node.py (2241) | startViewChange | VIEW CHANGE: Node2 changed to view 3, will start catchup now 2017-11-21 13:45:44,756 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 13:45:44,756 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 13:45:44,771 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 13:45:44,772 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:45:44,772 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:45:44,772 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 13:45:44,772 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 13:45:44,787 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 13:45:44,788 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 13:45:44,788 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 13:45:44,815 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:0 selected primary Node4:0 for instance 0 (view 3) 2017-11-21 13:45:44,815 | INFO | node.py ( 532) | start_participating | Node2 started participating 2017-11-21 13:45:44,816 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:0 declares view change 3 as completed for instance 0, new primary is Node4:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 6, '3ru9mawZCCRe7brebbogXMNBwceurrrDPC1FGvNkrsks')] 2017-11-21 13:45:44,816 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:1 selected primary Node1:1 for instance 1 (view 3) 2017-11-21 13:45:44,816 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:1 declares view change 3 as completed for instance 1, new primary is Node1:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 6, '3ru9mawZCCRe7brebbogXMNBwceurrrDPC1FGvNkrsks')] 2017-11-21 13:45:49,746 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271949735623) from client Node3 2017-11-21 13:45:49,811 | INFO | replica.py (1619) | order_3pc_key | Node2:0 ordered batch request, view no 3, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 8rGzHyRHHaJnkbvGHDNZ2FYkrz3MgQ6jmuU3mGUjhANi, requests ordered [('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271949735623)], discarded [] 2017-11-21 13:45:49,814 | INFO | node.py (2374) | executeBatch | Node2 committed batch request, view no 3, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 8rGzHyRHHaJnkbvGHDNZ2FYkrz3MgQ6jmuU3mGUjhANi, requests: [('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271949735623)] 2017-11-21 13:45:49,829 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node3 2017-11-21 13:45:49,835 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 3, ppSeqNo 1, ledger 2, state root None, txn root None, requests ordered [('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', 1511271949735623)], discarded [] 2017-11-21 13:49:59,804 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272199791267) from client Node4 2017-11-21 13:49:59,849 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 3, ppSeqNo 2, ledger 2, state root None, txn root None, requests ordered [('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272199791267)], discarded [] 2017-11-21 13:49:59,854 | INFO | replica.py (1619) | order_3pc_key | Node2:0 ordered batch request, view no 3, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root ATezckL25jagwcHSbqTSshUXpLCExPPpkaatxvqjnfz7, requests ordered [('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272199791267)], discarded [] 2017-11-21 13:49:59,855 | INFO | node.py (2374) | executeBatch | Node2 committed batch request, view no 3, ppSeqNo 2, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root ATezckL25jagwcHSbqTSshUXpLCExPPpkaatxvqjnfz7, requests: [('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272199791267)] 2017-11-21 13:50:32,407 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node4 2017-11-21 13:50:32,408 | INFO | node.py ( 884) | onConnsChanged | Node2 lost connection to primary of master 2017-11-21 13:50:34,417 | INFO | node.py (2128) | sendInstanceChange | VIEW CHANGE: Node2 sending an instance change with view_no 4 since Primary of master protocol instance disconnected 2017-11-21 13:50:34,417 | INFO | node.py (2131) | sendInstanceChange | MONITORING: Node2 metrics for monitor: Node2 Monitor metrics:: None Delta: 0.4 Lambda: 60 Omega: 5 instances started: [1726476.853379012, 1726476.85454919] ordered request counts: {0: 2, 1: 2} ordered request durations: {0: 0.11073701782152057, 1: 0.12598506710492074} master request latencies: {} client avg request latencies: [{'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': (1, 0.05081402393989265), 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': (1, 0.05992299388162792)}, {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': (1, 0.0449202221352607), 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': (1, 0.08106484496966004)}] throughput: {0: 18.060807843168423, 1: 15.874897287107796} master throughput: 18.060807843168423 total requests: 4 avg backup throughput: 15.874897287107796 master throughput ratio: 1.1376960440453263 2017-11-21 13:50:34,418 | INFO | node.py (2031) | do_view_change_if_possible | VIEW CHANGE: Node2 initiating a view change to 4 from 3 2017-11-21 13:50:34,419 | INFO | node.py (2241) | startViewChange | VIEW CHANGE: Node2 changed to view 4, will start catchup now 2017-11-21 13:50:34,419 | INFO | node.py (2191) | propose_view_change | Node2 sent view change since was disconnected from primary for too long 2017-11-21 13:50:34,435 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 13:50:34,436 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 13:50:34,476 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 13:50:34,480 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:50:34,480 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 13:50:34,481 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 13:50:34,481 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 13:50:34,505 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 13:50:34,505 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 13:50:34,505 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 13:50:34,536 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:0 selected primary Node1:0 for instance 0 (view 4) 2017-11-21 13:50:34,536 | INFO | node.py ( 532) | start_participating | Node2 started participating 2017-11-21 13:50:34,536 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:0 declares view change 4 as completed for instance 0, new primary is Node1:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 8, 'ATezckL25jagwcHSbqTSshUXpLCExPPpkaatxvqjnfz7')] 2017-11-21 13:50:34,536 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:1 selected primary Node2:1 for instance 1 (view 4) 2017-11-21 13:50:34,537 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:1 declares view change 4 as completed for instance 1, new primary is Node2:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'TswSHbkNk6QAQJuyuqosYLYjuYEYZrRyCwuY9rGvzXx'), (2, 8, 'ATezckL25jagwcHSbqTSshUXpLCExPPpkaatxvqjnfz7')] 2017-11-21 13:50:43,994 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node4 2017-11-21 13:50:44,024 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272244012595) from client Node4 2017-11-21 13:50:44,057 | INFO | replica.py (1619) | order_3pc_key | Node2:0 ordered batch request, view no 4, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 4k5nzTnx5EMvpLGBsdBgtciuYMZMKjYhqwTUbbhDhyWW, requests ordered [('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272244012595)], discarded [] 2017-11-21 13:50:44,058 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 4, ppSeqNo 1, ledger 2, state root None, txn root None, requests ordered [('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272244012595)], discarded [] 2017-11-21 13:50:44,059 | INFO | node.py (2374) | executeBatch | Node2 committed batch request, view no 4, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 4k5nzTnx5EMvpLGBsdBgtciuYMZMKjYhqwTUbbhDhyWW, requests: [('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', 1511272244012595)] 2017-11-21 14:39:49,454 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511275189432767) from client yg4t8ejbZ%PBSioTlg:nv%PU[llx1L4:u:iAG[Hu 2017-11-21 14:39:49,494 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 4, ppSeqNo 2, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511275189432767)], discarded [] 2017-11-21 14:39:49,559 | INFO | replica.py (1619) | order_3pc_key | Node2:0 ordered batch request, view no 4, ppSeqNo 2, ledger 1, state root DneLpHxR3mGWEMP1r9d6s7KcipxQkSu7CBFqsRg2N5qv, txn root D8n7Qh8qBHg95PCbEKsjycmBnRN7MmVjZMCm16ojR3L6, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511275189432767)], discarded [] 2017-11-21 14:39:49,560 | INFO | node.py (2374) | executeBatch | Node2 committed batch request, view no 4, ppSeqNo 2, ledger 1, state root DneLpHxR3mGWEMP1r9d6s7KcipxQkSu7CBFqsRg2N5qv, txn root D8n7Qh8qBHg95PCbEKsjycmBnRN7MmVjZMCm16ojR3L6, requests: [('V4SGRU86Z58d6TV7PBUe6f', 1511275189432767)] 2017-11-21 14:42:38,348 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node1 2017-11-21 14:42:38,348 | INFO | node.py ( 884) | onConnsChanged | Node2 lost connection to primary of master 2017-11-21 14:42:41,252 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node1 ordered, NYM Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv 2017-11-21 14:42:41,252 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node2 ordered, NYM 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb 2017-11-21 14:42:41,252 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node3 ordered, NYM DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya 2017-11-21 14:42:41,253 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node4 ordered, NYM 4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA 2017-11-21 14:42:41,266 | INFO | node.py ( 617) | _create_bls_bft | BLS: BLS Signatures will be used for Node Node2 2017-11-21 14:42:41,272 | INFO | node.py ( 479) | setPoolParams | Node2 updated its pool parameters: f 1, totalNodes 4, allNodeNames {'Node1', 'Node3', 'Node2', 'Node4'}, requiredNumberOfInstances 2, minimumNodes 3, quorums {'ledger_status': Quorum(2), 'timestamp': Quorum(2), 'f': 1, 'checkpoint': Quorum(2), 'prepare': Quorum(2), 'view_change': Quorum(3), 'bls_signatures': Quorum(3), 'propagate': Quorum(2), 'consistency_proof': Quorum(2), 'propagate_primary': Quorum(2), 'view_change_done': Quorum(3), 'reply': Quorum(2), 'commit': Quorum(3), 'same_consistency_proof': Quorum(2), 'election': Quorum(3)} 2017-11-21 14:42:41,307 | INFO | plugin_loader.py ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-11-21 14:42:41,308 | DISPLAY | replicas.py ( 40) | grow | Node2 added replica Node2:0 to instance 0 (master) 2017-11-21 14:42:41,308 | DISPLAY | replicas.py ( 40) | grow | Node2 added replica Node2:1 to instance 1 (backup) 2017-11-21 14:42:41,352 | INFO | node.py (2503) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 14:42:41,353 | INFO | stacks.py ( 84) | start | CONNECTION: Node2 listening for other nodes at 0.0.0.0:9703 2017-11-21 14:42:41,354 | INFO | node.py ( 680) | start | Node2 first time running... 2017-11-21 14:42:41,355 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node1 at 10.0.0.2:9701 2017-11-21 14:42:41,356 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node3 at 10.0.0.4:9705 2017-11-21 14:42:41,365 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node4 at 10.0.0.5:9707 2017-11-21 14:42:41,501 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-11-21 14:42:42,230 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:42:42,231 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:42:42,241 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node3 2017-11-21 14:42:42,269 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:42:42,269 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:42:42,270 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:42:42,270 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:42:42,270 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:42:42,289 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:42:42,290 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:42:42,290 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 14:42:42,305 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:0 selected primary Node1:0 for instance 0 (view 0) 2017-11-21 14:42:42,306 | INFO | node.py ( 532) | start_participating | Node2 started participating 2017-11-21 14:42:42,306 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:0 declares view change 0 as completed for instance 0, new primary is Node1:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 16, 'D8n7Qh8qBHg95PCbEKsjycmBnRN7MmVjZMCm16ojR3L6'), (2, 9, '4k5nzTnx5EMvpLGBsdBgtciuYMZMKjYhqwTUbbhDhyWW')] 2017-11-21 14:42:42,306 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:1 selected primary Node2:1 for instance 1 (view 0) 2017-11-21 14:42:42,307 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:1 declares view change 0 as completed for instance 1, new primary is Node2:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 16, 'D8n7Qh8qBHg95PCbEKsjycmBnRN7MmVjZMCm16ojR3L6'), (2, 9, '4k5nzTnx5EMvpLGBsdBgtciuYMZMKjYhqwTUbbhDhyWW')] 2017-11-21 14:42:42,993 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node4 2017-11-21 14:43:58,093 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511275438071041) from client yg4t8ejbZ%PBSioTlg:nv%PU[llx1L4:u:iAG[Hu 2017-11-21 14:43:58,146 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 0, ppSeqNo 1, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511275438071041)], discarded [] 2017-11-21 14:43:58,160 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node3 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:43:58,161 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node4 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:43:58,165 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node1 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:44:55,229 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511275495206223) from client yg4t8ejbZ%PBSioTlg:nv%PU[llx1L4:u:iAG[Hu 2017-11-21 14:44:55,273 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node4 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:44:55,273 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 0, ppSeqNo 2, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511275495206223)], discarded [] 2017-11-21 14:44:55,288 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node1 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:44:55,288 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node3 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:45:05,198 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node1 2017-11-21 14:45:05,198 | INFO | node.py ( 884) | onConnsChanged | Node2 lost connection to primary of master 2017-11-21 14:45:07,200 | INFO | node.py (2128) | sendInstanceChange | VIEW CHANGE: Node2 sending an instance change with view_no 1 since Primary of master protocol instance disconnected 2017-11-21 14:45:07,200 | INFO | node.py (2131) | sendInstanceChange | MONITORING: Node2 metrics for monitor: Node2 Monitor metrics:: None Delta: 0.4 Lambda: 60 Omega: 5 instances started: [1730209.517190699, 1730209.517487361] ordered request counts: {0: 0, 1: 2} ordered request durations: {0: 0, 1: 0.09640606609173119} master request latencies: {} client avg request latencies: [{}, {'V4SGRU86Z58d6TV7PBUe6f': (2, 0.048203033045865595)}] throughput: {0: 0, 1: 20.74558252482767} master throughput: None total requests: 0 avg backup throughput: 20.74558252482767 master throughput ratio: None 2017-11-21 14:45:07,200 | INFO | node.py (2191) | propose_view_change | Node2 sent view change since was disconnected from primary for too long 2017-11-21 14:45:07,201 | INFO | node.py (2031) | do_view_change_if_possible | VIEW CHANGE: Node2 initiating a view change to 1 from 0 2017-11-21 14:45:07,202 | INFO | node.py (2241) | startViewChange | VIEW CHANGE: Node2 changed to view 1, will start catchup now 2017-11-21 14:45:07,203 | INFO | message_processor.py ( 29) | discard | Node2 discarding message INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} because Received instance change request with view no 1 which is not more than its view no 1 2017-11-21 14:45:07,210 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,210 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,220 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,220 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,220 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,221 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,221 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,238 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,238 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,238 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,253 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,253 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,284 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,284 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,285 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,285 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,285 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,303 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,304 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,304 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,343 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,343 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,361 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,362 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,362 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,363 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,363 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,364 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,364 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,364 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,389 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,389 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,412 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,413 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,413 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,413 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,414 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,414 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,415 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,415 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,450 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,451 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,480 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,480 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,481 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,481 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,481 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,482 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,482 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,484 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,511 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,512 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,512 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,512 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,513 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,513 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,514 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,514 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,515 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,515 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,539 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,540 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,541 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,541 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,541 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,542 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,542 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,543 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,543 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,544 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,586 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,587 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,588 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,588 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,588 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,589 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,589 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,589 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,590 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,590 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,600 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,602 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,602 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,603 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,603 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,604 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,604 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,604 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,606 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,606 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,617 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,617 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,648 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,649 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,649 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,649 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,650 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,650 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,650 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,652 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,659 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,661 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,661 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,661 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,662 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,664 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,664 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,664 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,665 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,665 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,674 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,675 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,676 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,676 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,676 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,677 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,677 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,678 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,679 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,680 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,691 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,692 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,693 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,693 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,694 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,694 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:07,695 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:07,695 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:07,696 | INFO | node.py (1593) | allLedgersCaughtUp | Node2 needs to catchup again 2017-11-21 14:45:07,697 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:07,703 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:07,704 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:07,706 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:07,709 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,709 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:07,709 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:09,834 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node1 ordered, NYM Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv 2017-11-21 14:45:09,834 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node2 ordered, NYM 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb 2017-11-21 14:45:09,835 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node3 ordered, NYM DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya 2017-11-21 14:45:09,835 | INFO | pool_manager.py ( 409) | _order_node | Node2 node Node4 ordered, NYM 4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA 2017-11-21 14:45:09,863 | INFO | node.py ( 617) | _create_bls_bft | BLS: BLS Signatures will be used for Node Node2 2017-11-21 14:45:09,873 | INFO | node.py ( 479) | setPoolParams | Node2 updated its pool parameters: f 1, totalNodes 4, allNodeNames {'Node2', 'Node3', 'Node1', 'Node4'}, requiredNumberOfInstances 2, minimumNodes 3, quorums {'timestamp': Quorum(2), 'reply': Quorum(2), 'election': Quorum(3), 'commit': Quorum(3), 'ledger_status': Quorum(2), 'f': 1, 'propagate_primary': Quorum(2), 'view_change_done': Quorum(3), 'checkpoint': Quorum(2), 'same_consistency_proof': Quorum(2), 'prepare': Quorum(2), 'propagate': Quorum(2), 'bls_signatures': Quorum(3), 'consistency_proof': Quorum(2), 'view_change': Quorum(3)} 2017-11-21 14:45:09,915 | INFO | plugin_loader.py ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-11-21 14:45:09,916 | DISPLAY | replicas.py ( 40) | grow | Node2 added replica Node2:0 to instance 0 (master) 2017-11-21 14:45:09,916 | DISPLAY | replicas.py ( 40) | grow | Node2 added replica Node2:1 to instance 1 (backup) 2017-11-21 14:45:10,041 | INFO | node.py (2503) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-11-21 14:45:10,044 | INFO | stacks.py ( 84) | start | CONNECTION: Node2 listening for other nodes at 0.0.0.0:9703 2017-11-21 14:45:10,044 | INFO | node.py ( 680) | start | Node2 first time running... 2017-11-21 14:45:10,046 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node3 at 10.0.0.4:9705 2017-11-21 14:45:10,047 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node1 at 10.0.0.2:9701 2017-11-21 14:45:10,048 | INFO | zstack.py ( 585) | connect | CONNECTION: Node2 looking for Node4 at 10.0.0.5:9707 2017-11-21 14:45:10,521 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:45:10,523 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:45:10,523 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:10,524 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:45:10,524 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:45:10,525 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:45:10,525 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:45:10,525 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 14:45:10,526 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:45:10,526 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 14:45:10,528 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:45:10,529 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 14:45:10,729 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-11-21 14:45:10,729 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node4 2017-11-21 14:45:10,808 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node3 2017-11-21 14:45:11,700 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node3 2017-11-21 14:45:13,211 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511275438071041) from client b'yg4t8ejbZ%PBSioTlg:nv%PU[llx1L4:u:iAG[Hu' 2017-11-21 14:45:13,219 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511275495206223) from client b'yg4t8ejbZ%PBSioTlg:nv%PU[llx1L4:u:iAG[Hu' 2017-11-21 14:45:13,921 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node4 2017-11-21 14:45:25,072 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node3 2017-11-21 14:45:25,072 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node4 2017-11-21 14:46:10,052 | INFO | node.py (2128) | sendInstanceChange | VIEW CHANGE: Node2 sending an instance change with view_no 1 since Primary of master protocol instance disconnected 2017-11-21 14:46:10,053 | INFO | node.py (2131) | sendInstanceChange | MONITORING: Node2 metrics for monitor: Node2 Monitor metrics:: None Delta: 0.4 Lambda: 60 Omega: 5 instances started: [1730358.125663602, 1730358.126018292] ordered request counts: {0: 0, 1: 0} ordered request durations: {0: 0, 1: 0} master request latencies: {} client avg request latencies: [{}, {}] throughput: {0: 0, 1: 0} master throughput: None total requests: 0 avg backup throughput: None master throughput ratio: None 2017-11-21 14:46:10,054 | INFO | node.py (2191) | propose_view_change | Node2 sent view change since was disconnected from primary for too long 2017-11-21 14:46:16,060 | INFO | node.py (2031) | do_view_change_if_possible | VIEW CHANGE: Node2 initiating a view change to 1 from 0 2017-11-21 14:46:16,061 | INFO | node.py (2241) | startViewChange | VIEW CHANGE: Node2 changed to view 1, will start catchup now 2017-11-21 14:46:16,084 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-11-21 14:46:16,084 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-11-21 14:46:16,102 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-11-21 14:46:16,103 | INFO | upgrader.py ( 224) | processLedger | 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:46:16,103 | INFO | upgrader.py ( 305) | handleUpgradeTxn | Node 'Node2' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade_to_the_latest_full'), ('reinstall', False), ('reqId', 1511271022953271), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-11-21T13:50:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-11-21T13:40:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-11-21T13:45:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-11-21T13:35:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3NRsExGbSDFBeVe8Negs1fJuWJTugu2xq1V6eSVHhGNeSULAeBdzn9VNyv4wwYSnKjhc94n4k2SSm6k7AVMoSHzS'), ('timeout', 10), ('txnTime', 1511271022), ('type', '109'), ('version', '1.2.47'), ('seqNo', 1)]) 2017-11-21 14:46:16,103 | INFO | upgrader.py ( 325) | handleUpgradeTxn | Node 'Node2' has already performed an upgrade with upgrade_id 15112710229532711. Last recorded event is ('succeeded', datetime.datetime(2017, 11, 21, 13, 40, 0, 258870, tzinfo=tzlocal()), '1.2.47', '15112710229532711') 2017-11-21 14:46:16,104 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-11-21 14:46:16,122 | INFO | node.py (1551) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-11-21 14:46:16,122 | INFO | ledger_manager.py ( 865) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-11-21 14:46:16,122 | INFO | node.py (1598) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-11-21 14:46:16,123 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:0 selected primary Node2:0 for instance 0 (view 1) 2017-11-21 14:46:16,124 | INFO | node.py ( 532) | start_participating | Node2 started participating 2017-11-21 14:46:16,124 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:0 declares view change 1 as completed for instance 0, new primary is Node2:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 16, 'D8n7Qh8qBHg95PCbEKsjycmBnRN7MmVjZMCm16ojR3L6'), (2, 9, '4k5nzTnx5EMvpLGBsdBgtciuYMZMKjYhqwTUbbhDhyWW')] 2017-11-21 14:46:16,124 | DISPLAY | primary_selector.py ( 327) | _start_selection | PRIMARY SELECTION: Node2:1 selected primary Node3:1 for instance 1 (view 1) 2017-11-21 14:46:16,124 | DISPLAY | primary_selector.py ( 351) | _start_selection | VIEW CHANGE: Node2:1 declares view change 1 as completed for instance 1, new primary is Node3:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 16, 'D8n7Qh8qBHg95PCbEKsjycmBnRN7MmVjZMCm16ojR3L6'), (2, 9, '4k5nzTnx5EMvpLGBsdBgtciuYMZMKjYhqwTUbbhDhyWW')] 2017-11-21 14:46:16,161 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node4 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:46:16,161 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node3 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:46:16,165 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node1 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 14:46:16,180 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 1, ppSeqNo 1, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511275438071041), ('V4SGRU86Z58d6TV7PBUe6f', 1511275495206223)], discarded [] 2017-11-21 15:02:31,074 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511276551030155) from client yg4t8ejbZ%PBSioTlg:nv%PU[llx1L4:u:iAG[Hu 2017-11-21 15:02:31,134 | INFO | replica.py (2025) | enqueue_prepare | Node2:1 queueing prepare due to unavailability of PRE-PREPARE. Prepare PREPARE{'ppSeqNo': 2, 'viewNo': 1, 'txnRootHash': None, 'stateRootHash': None, 'ppTime': 1511276551, 'digest': '68543da2215892272ef838cd8323a3d2298937cedc460a65f1d570f941be738a', 'instId': 1} from Node1:1 2017-11-21 15:02:31,135 | INFO | replica.py (2025) | enqueue_prepare | Node2:1 queueing prepare due to unavailability of PRE-PREPARE. Prepare PREPARE{'ppSeqNo': 2, 'viewNo': 1, 'txnRootHash': None, 'stateRootHash': None, 'ppTime': 1511276551, 'digest': '68543da2215892272ef838cd8323a3d2298937cedc460a65f1d570f941be738a', 'instId': 1} from Node4:1 2017-11-21 15:02:31,136 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 1, ppSeqNo 2, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511276551030155)], discarded [] 2017-11-21 15:02:31,153 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node3 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 15:02:31,156 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node4 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 15:02:31,170 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node1 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 15:39:59,875 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1511278799838029) from client .pQ[L6htk(fAQ76HZO.x9sRh!s:V!o894BgpH44h 2017-11-21 15:39:59,922 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node3 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 15:39:59,922 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node1 for Commit message has invalid BLS signature; suspicion code is 31 2017-11-21 15:39:59,923 | INFO | replica.py (1619) | order_3pc_key | Node2:1 ordered batch request, view no 1, ppSeqNo 3, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1511278799838029)], discarded [] 2017-11-21 15:39:59,925 | WARNING | node.py (2591) | reportSuspiciousNode | Node2 raised suspicion on node Node4 for Commit message has invalid BLS signature; suspicion code is 31