2017-10-11 16:59:19,440 | INFO | log.py ( 79) | setupRaet | Setting RAET log level 2 2017-10-11 16:59:19,454 | DEBUG | start_sovrin_node ( 39) | | You can find logs in /home/sovrin/.sovrin/Node1.log 2017-10-11 16:59:19,454 | DEBUG | start_sovrin_node ( 42) | | Sovrin related env vars: [] 2017-10-11 16:59:20,496 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: git 2017-10-11 16:59:20,520 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: hg 2017-10-11 16:59:20,568 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: svn 2017-10-11 16:59:20,569 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: bzr 2017-10-11 16:59:21,219 | DEBUG | selector_events.py ( 53) | __init__ | Using selector: EpollSelector 2017-10-11 16:59:21,219 | DEBUG | looper.py ( 123) | __init__ | Setting handler for SIGINT 2017-10-11 16:59:21,250 | DEBUG | ledger.py ( 206) | start | Starting ledger... 2017-10-11 16:59:21,251 | DEBUG | file_store.py ( 190) | appendNewLineIfReq | new line check for file: /home/sovrin/.sovrin/data/nodes/Node1/transactions_live/1 2017-10-11 16:59:21,251 | DEBUG | ledger.py ( 78) | recoverTree | Recovering tree from transaction log 2017-10-11 16:59:21,286 | DEBUG | ledger.py ( 93) | recoverTree | Recovered tree in 0.03435618383809924 seconds 2017-10-11 16:59:21,312 | DEBUG | idr_cache.py ( 25) | __init__ | Initializing identity cache Node1 2017-10-11 16:59:21,324 | INFO | node.py (2408) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 16:59:21,347 | DEBUG | ledger.py ( 206) | start | Starting ledger... 2017-10-11 16:59:21,347 | DEBUG | file_store.py ( 190) | appendNewLineIfReq | new line check for file: /home/sovrin/.sovrin/data/nodes/Node1/pool_transactions_live/1 2017-10-11 16:59:21,348 | DEBUG | ledger.py ( 78) | recoverTree | Recovering tree from transaction log 2017-10-11 16:59:21,365 | DEBUG | ledger.py ( 93) | recoverTree | Recovered tree in 0.0169037370942533 seconds 2017-10-11 16:59:21,366 | INFO | node.py (2408) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 16:59:21,419 | DEBUG | plugin_loader.py ( 95) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 16:59:21,420 | DEBUG | plugin_loader.py ( 95) | _load | skipping plugin plugin_firebase_stats_consumer[class: typing.Dict<~KT, ~VT>] because it does not have a 'pluginType' attribute 2017-10-11 16:59:21,420 | INFO | plugin_loader.py ( 116) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-10-11 16:59:21,420 | DEBUG | plugin_loader.py ( 95) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 16:59:21,421 | DEBUG | plugin_loader.py ( 95) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 16:59:21,421 | DEBUG | plugin_loader.py ( 95) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 16:59:21,421 | DEBUG | has_action_queue.py ( 77) | startRepeating | checkPerformance will be repeating every 60 seconds 2017-10-11 16:59:21,421 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 1 to run in 60 seconds 2017-10-11 16:59:21,422 | INFO | replica.py ( 300) | h | Node1:0 set watermarks as 0 300 2017-10-11 16:59:21,422 | DISPLAY | node.py (1034) | addReplica | Node1 added replica Node1:0 to instance 0 (master) 2017-10-11 16:59:21,423 | INFO | replica.py ( 300) | h | Node1:1 set watermarks as 0 300 2017-10-11 16:59:21,423 | DISPLAY | node.py (1034) | addReplica | Node1 added replica Node1:1 to instance 1 (backup) 2017-10-11 16:59:21,423 | DEBUG | has_action_queue.py ( 77) | startRepeating | checkPerformance will be repeating every 10 seconds 2017-10-11 16:59:21,423 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 1 to run in 10 seconds 2017-10-11 16:59:21,423 | DEBUG | has_action_queue.py ( 77) | startRepeating | checkNodeRequestSpike will be repeating every 60 seconds 2017-10-11 16:59:21,423 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 2 to run in 60 seconds 2017-10-11 16:59:21,423 | DEBUG | plugin_helper.py ( 24) | loadPlugins | Plugin loading started to load plugins from basedir: /home/sovrin/.sovrin 2017-10-11 16:59:21,424 | DEBUG | plugin_helper.py ( 33) | loadPlugins | Plugin directory created at: /home/sovrin/.sovrin/plugins 2017-10-11 16:59:21,424 | DEBUG | plugin_helper.py ( 67) | loadPlugins | Total plugins loaded from basedir /home/sovrin/.sovrin are : 0 2017-10-11 16:59:21,424 | DEBUG | node.py ( 325) | __init__ | total plugins loaded in node: 0 2017-10-11 16:59:21,434 | DEBUG | ledger.py ( 206) | start | Starting ledger... 2017-10-11 16:59:21,435 | DEBUG | file_store.py ( 190) | appendNewLineIfReq | new line check for file: /home/sovrin/.sovrin/data/nodes/Node1/config_transactions/1 2017-10-11 16:59:21,435 | DEBUG | ledger.py ( 78) | recoverTree | Recovering tree from transaction log 2017-10-11 16:59:21,435 | DEBUG | ledger.py ( 93) | recoverTree | Recovered tree in 0.00032229581847786903 seconds 2017-10-11 16:59:21,447 | INFO | node.py (2408) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 16:59:21,448 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from stopped to starting 2017-10-11 16:59:21,448 | DEBUG | ledger.py ( 204) | start | Ledger already started. 2017-10-11 16:59:21,448 | INFO | zstack.py ( 312) | start | Node1 starting with restricted as True and reSetupAuth as True 2017-10-11 16:59:21,449 | DEBUG | authenticator.py ( 31) | start | Starting ZAP at inproc://zeromq.zap.1 2017-10-11 16:59:21,449 | DEBUG | base.py ( 72) | allow | Allowing 0.0.0.0 2017-10-11 16:59:21,449 | DEBUG | base.py ( 112) | configure_curve | Configure curve: *[/home/sovrin/.sovrin/Node1/public_keys] 2017-10-11 16:59:21,449 | DEBUG | zstack.py ( 339) | open | Node1 will bind its listener at 9701 2017-10-11 16:59:21,450 | INFO | stacks.py ( 76) | start | Node1 listening for other nodes at 0.0.0.0:9701 2017-10-11 16:59:21,450 | INFO | zstack.py ( 312) | start | Node1C starting with restricted as False and reSetupAuth as True 2017-10-11 16:59:21,450 | DEBUG | authenticator.py ( 31) | start | Starting ZAP at inproc://zeromq.zap.2 2017-10-11 16:59:21,450 | DEBUG | base.py ( 72) | allow | Allowing 0.0.0.0 2017-10-11 16:59:21,450 | DEBUG | base.py ( 112) | configure_curve | Configure curve: *[*] 2017-10-11 16:59:21,451 | DEBUG | zstack.py ( 339) | open | Node1C will bind its listener at 9702 2017-10-11 16:59:21,451 | INFO | node.py ( 594) | start | Node1 first time running... 2017-10-11 16:59:21,455 | DEBUG | kit_zstack.py ( 96) | connectToMissing | Node1 found the following missing connections: Node3, Node4, Node2 2017-10-11 16:59:21,455 | TRACE | remote.py ( 84) | connect | connecting socket 58 24263248 to remote Node3:HA(host='10.0.0.4', port=9705) 2017-10-11 16:59:21,455 | INFO | zstack.py ( 580) | connect | Node1 looking for Node3 at 10.0.0.4:9705 2017-10-11 16:59:21,456 | DEBUG | zstack.py ( 643) | sendPingPong | Node1 will be sending in batch 2017-10-11 16:59:21,457 | TRACE | remote.py ( 84) | connect | connecting socket 59 24366864 to remote Node4:HA(host='10.0.0.5', port=9707) 2017-10-11 16:59:21,457 | INFO | zstack.py ( 580) | connect | Node1 looking for Node4 at 10.0.0.5:9707 2017-10-11 16:59:21,457 | DEBUG | zstack.py ( 643) | sendPingPong | Node1 will be sending in batch 2017-10-11 16:59:21,458 | TRACE | remote.py ( 84) | connect | connecting socket 60 24404048 to remote Node2:HA(host='10.0.0.3', port=9703) 2017-10-11 16:59:21,458 | INFO | zstack.py ( 580) | connect | Node1 looking for Node2 at 10.0.0.3:9703 2017-10-11 16:59:21,458 | DEBUG | zstack.py ( 643) | sendPingPong | Node1 will be sending in batch 2017-10-11 16:59:21,458 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 16:59:21,472 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'pi' to Node3 2017-10-11 16:59:21,473 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'pi' to Node3 2017-10-11 16:59:21,473 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'pi' to Node4 2017-10-11 16:59:21,473 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'pi' to Node4 2017-10-11 16:59:21,473 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'pi' to Node2 2017-10-11 16:59:21,473 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'pi' to Node2 2017-10-11 16:59:21,933 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:21,934 | DEBUG | zstack.py ( 652) | handlePingPong | Node1 got ping from Node2 2017-10-11 16:59:21,934 | DEBUG | zstack.py ( 643) | sendPingPong | Node1 will be sending in batch 2017-10-11 16:59:21,936 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'po' to Node2 2017-10-11 16:59:21,936 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'po' to Node2 2017-10-11 16:59:22,023 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,024 | DEBUG | zstack.py ( 658) | handlePingPong | Node1 got pong from Node2 2017-10-11 16:59:22,035 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from set() to {'Node2'} 2017-10-11 16:59:22,036 | INFO | keep_in_touch.py ( 96) | _connsChanged | Node1 now connected to Node2 2017-10-11 16:59:22,037 | DEBUG | node.py (2593) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} to 1 recipients: ['Node2'] 2017-10-11 16:59:22,039 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,040 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 16:59:22,040 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ppSeqNo': None, 'viewNo': None, 'txnSeqNo': 4, 'op': 'LEDGER_STATUS', 'ledgerId': 0}, 'Node2') 2017-10-11 16:59:22,041 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 16:59:22,042 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} from Node2 2017-10-11 16:59:22,042 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 16:59:22,042 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 16:59:22,044 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"ledgerId":0}' to Node2 2017-10-11 16:59:22,045 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"ledgerId":0}' to Node2 2017-10-11 16:59:22,673 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,674 | DEBUG | zstack.py ( 652) | handlePingPong | Node1 got ping from Node3 2017-10-11 16:59:22,674 | DEBUG | zstack.py ( 643) | sendPingPong | Node1 will be sending in batch 2017-10-11 16:59:22,676 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'po' to Node3 2017-10-11 16:59:22,676 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'po' to Node3 2017-10-11 16:59:22,736 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,737 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}']} 2017-10-11 16:59:22,737 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}']}, 'Node2') 2017-10-11 16:59:22,738 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}']} 2017-10-11 16:59:22,738 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,739 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 16:59:22,739 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,739 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,740 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 16:59:22,740 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,741 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 16:59:22,741 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 16:59:22,743 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node2 into one transmission 2017-10-11 16:59:22,743 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', b'{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}']) 2017-10-11 16:59:22,744 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node2: b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' 2017-10-11 16:59:22,745 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' to Node2 2017-10-11 16:59:22,781 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 16:59:22,782 | DEBUG | zstack.py ( 658) | handlePingPong | Node1 got pong from Node3 2017-10-11 16:59:22,787 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"primary":[],"op":"CURRENT_STATE","viewNo":0}', '{"op":"LEDGER_STATUS","txnSeqNo":4,"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","viewNo":null,"ledgerId":0,"ppSeqNo":null}']} 2017-10-11 16:59:22,787 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"primary":[],"op":"CURRENT_STATE","viewNo":0}', '{"op":"LEDGER_STATUS","txnSeqNo":4,"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","viewNo":null,"ledgerId":0,"ppSeqNo":null}']}, 'Node3') 2017-10-11 16:59:22,787 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"primary":[],"op":"CURRENT_STATE","viewNo":0}', '{"op":"LEDGER_STATUS","txnSeqNo":4,"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","viewNo":null,"ledgerId":0,"ppSeqNo":null}']} 2017-10-11 16:59:22,788 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: CURRENT_STATE{'primary': [], 'viewNo': 0} 2017-10-11 16:59:22,788 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'primary': [], 'viewNo': 0, 'op': 'CURRENT_STATE'}, 'Node3') 2017-10-11 16:59:22,788 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'primary': [], 'viewNo': 0} 2017-10-11 16:59:22,789 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 16:59:22,789 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ppSeqNo': None, 'viewNo': None, 'op': 'LEDGER_STATUS', 'txnSeqNo': 4, 'ledgerId': 0}, 'Node3') 2017-10-11 16:59:22,789 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 16:59:22,792 | DEBUG | node.py ( 884) | process_current_state_message | Node1 processing current state CURRENT_STATE{'primary': [], 'viewNo': 0} from Node3 2017-10-11 16:59:22,793 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} from Node3 2017-10-11 16:59:22,794 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 16:59:22,794 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 16:59:22,794 | DEBUG | ledger_manager.py ( 309) | processLedgerStatus | Node1 found out from {'Node3', 'Node2'} that its ledger of type 0 is latest 2017-10-11 16:59:22,794 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} that it does not need catchup 2017-10-11 16:59:22,795 | DEBUG | node.py (1537) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 16:59:22,795 | DEBUG | replica.py (2130) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 16:59:22,799 | DEBUG | node.py (1918) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 16:59:22,800 | DEBUG | replica.py (2130) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 16:59:22,801 | DEBUG | node.py (1918) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 16:59:22,801 | DEBUG | node.py (2451) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 16:59:22,802 | DEBUG | monitor.py ( 183) | reset | Monitor being reset 2017-10-11 16:59:22,802 | DEBUG | node.py (1547) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 0 2017-10-11 16:59:22,802 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 16:59:22,802 | DEBUG | node.py ( 918) | checkInstances | Node1 choosing to start election on the basis of count 2 and nodes {'Node2'} 2017-10-11 16:59:22,804 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 16:59:22,804 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node1 for ledger status of ledger 2 2017-10-11 16:59:22,804 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 16:59:22,805 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node2 for ledger status of ledger 2 2017-10-11 16:59:22,805 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,805 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node3 for ledger status of ledger 2 2017-10-11 16:59:22,805 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 16:59:22,806 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node4 for ledger status of ledger 2 2017-10-11 16:59:22,806 | DEBUG | ledger_manager.py (1006) | processStashedLedgerStatuses | Node1 going to process 0 stashed ledger statuses for ledger 2 2017-10-11 16:59:22,806 | INFO | ledger_manager.py ( 831) | catchupCompleted | Node1 completed catching up ledger 0, caught up 0 in total 2017-10-11 16:59:22,808 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node3 into one transmission 2017-10-11 16:59:22,809 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']) 2017-10-11 16:59:22,809 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node3: b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' 2017-10-11 16:59:22,809 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' to Node3 2017-10-11 16:59:22,810 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node4 into one transmission 2017-10-11 16:59:22,814 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']) 2017-10-11 16:59:22,819 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node4: b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' 2017-10-11 16:59:22,819 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' to Node4 2017-10-11 16:59:22,819 | WARNING | zstack.py ( 704) | transmit | Remote Node4 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings 2017-10-11 16:59:22,819 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node2 into one transmission 2017-10-11 16:59:22,819 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']) 2017-10-11 16:59:22,819 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node2: b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' 2017-10-11 16:59:22,820 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' to Node2 2017-10-11 16:59:22,820 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node2'} to {'Node3', 'Node2'} 2017-10-11 16:59:22,820 | INFO | keep_in_touch.py ( 96) | _connsChanged | Node1 now connected to Node3 2017-10-11 16:59:22,820 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from starting to started_hungry 2017-10-11 16:59:22,821 | DEBUG | node.py ( 918) | checkInstances | Node1 choosing to start election on the basis of count 3 and nodes {'Node3', 'Node2'} 2017-10-11 16:59:22,821 | DEBUG | primary_selector.py ( 74) | get_msgs_for_lagged_nodes | Node1 has no ViewChangeDone message to send for view 0 2017-10-11 16:59:22,821 | DEBUG | node.py ( 879) | send_current_state_to_lagging_node | Node1 sending current state CURRENT_STATE{'primary': [], 'viewNo': 0} to lagged node Node3 2017-10-11 16:59:22,821 | DEBUG | node.py (2593) | send | Node1 sending message CURRENT_STATE{'primary': [], 'viewNo': 0} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,822 | DEBUG | node.py (2593) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,822 | DEBUG | node.py (2593) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,824 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 16:59:22,824 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}']} 2017-10-11 16:59:22,824 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}']}, 'Node2') 2017-10-11 16:59:22,825 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}']} 2017-10-11 16:59:22,825 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,825 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 16:59:22,825 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,826 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,826 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 16:59:22,826 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,826 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,826 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']}, 'Node3') 2017-10-11 16:59:22,826 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,827 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,827 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}}, 'Node3') 2017-10-11 16:59:22,827 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,827 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,828 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}}, 'Node3') 2017-10-11 16:59:22,828 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,828 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 16:59:22,829 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 16:59:22,830 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} from Node3 2017-10-11 16:59:22,830 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,830 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,830 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} from Node3 2017-10-11 16:59:22,843 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,843 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,844 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 3 msgs to Node3 into one transmission 2017-10-11 16:59:22,847 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"primary":[],"viewNo":0,"op":"CURRENT_STATE"}', b'{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"ledgerId":0}', b'{"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":9,"ledgerId":1}']) 2017-10-11 16:59:22,847 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node3: b'{"op":"BATCH","signature":null,"messages":["{\\"primary\\":[],\\"viewNo\\":0,\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":4,\\"ledgerId\\":0}","{\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":9,\\"ledgerId\\":1}"]}' 2017-10-11 16:59:22,847 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"primary\\":[],\\"viewNo\\":0,\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":4,\\"ledgerId\\":0}","{\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":9,\\"ledgerId\\":1}"]}' to Node3 2017-10-11 16:59:22,847 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node2 into one transmission 2017-10-11 16:59:22,848 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', b'{"msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}']) 2017-10-11 16:59:22,848 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node2: b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' 2017-10-11 16:59:22,848 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' to Node2 2017-10-11 16:59:22,853 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,854 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}']} 2017-10-11 16:59:22,854 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}']}, 'Node2') 2017-10-11 16:59:22,854 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}']} 2017-10-11 16:59:22,855 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,856 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 16:59:22,856 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,856 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,856 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 16:59:22,856 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,867 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} from Node2 2017-10-11 16:59:22,867 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,867 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,867 | DEBUG | ledger_manager.py ( 309) | processLedgerStatus | Node1 found out from {'Node3', 'Node2'} that its ledger of type 2 is latest 2017-10-11 16:59:22,867 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} that it does not need catchup 2017-10-11 16:59:22,868 | DEBUG | node.py (1537) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 16:59:22,868 | DEBUG | replica.py (2130) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 16:59:22,868 | DEBUG | node.py (1918) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 16:59:22,868 | DEBUG | replica.py (2130) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 16:59:22,868 | DEBUG | node.py (1918) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 16:59:22,870 | DEBUG | node.py (2451) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 16:59:22,870 | DEBUG | monitor.py ( 183) | reset | Monitor being reset 2017-10-11 16:59:22,870 | DEBUG | node.py (1547) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 2 2017-10-11 16:59:22,871 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,871 | INFO | pool_config.py ( 33) | processLedger | processing config ledger for any POOL_CONFIGs 2017-10-11 16:59:22,871 | INFO | upgrader.py ( 145) | processLedger | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv processing config ledger for any upgrades 2017-10-11 16:59:22,875 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 16:59:22,876 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node1 for ledger status of ledger 1 2017-10-11 16:59:22,876 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 16:59:22,876 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node2 for ledger status of ledger 1 2017-10-11 16:59:22,876 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,876 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node3 for ledger status of ledger 1 2017-10-11 16:59:22,877 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 16:59:22,877 | DEBUG | node.py ( 832) | _ask_for_ledger_status | Node1 asking Node4 for ledger status of ledger 1 2017-10-11 16:59:22,877 | DEBUG | ledger_manager.py (1006) | processStashedLedgerStatuses | Node1 going to process 0 stashed ledger statuses for ledger 1 2017-10-11 16:59:22,877 | INFO | ledger_manager.py ( 831) | catchupCompleted | Node1 completed catching up ledger 2, caught up 0 in total 2017-10-11 16:59:22,877 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} from Node2 2017-10-11 16:59:22,878 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,878 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:22,881 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node3 into one transmission 2017-10-11 16:59:22,881 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']) 2017-10-11 16:59:22,882 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node3: b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' 2017-10-11 16:59:22,882 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' to Node3 2017-10-11 16:59:22,882 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node4 into one transmission 2017-10-11 16:59:22,882 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']) 2017-10-11 16:59:22,882 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node4: b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' 2017-10-11 16:59:22,882 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' to Node4 2017-10-11 16:59:22,885 | WARNING | zstack.py ( 704) | transmit | Remote Node4 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings 2017-10-11 16:59:22,885 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node2 into one transmission 2017-10-11 16:59:22,885 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', b'{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']) 2017-10-11 16:59:22,886 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node2: b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' 2017-10-11 16:59:22,886 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}","{\\"op\\":\\"MESSAGE_REQUEST\\",\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\"}"]}' to Node2 2017-10-11 16:59:22,888 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,889 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:22,889 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'op': 'VIEW_CHANGE_DONE'}, 'Node2') 2017-10-11 16:59:22,889 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:22,893 | DEBUG | node.py (1261) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]}, 'Node2') 2017-10-11 16:59:22,894 | DEBUG | primary_selector.py ( 103) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node2 : VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:22,894 | DEBUG | primary_selector.py ( 186) | _hasViewChangeQuorum | Node1 needs 1 ViewChangeDone messages 2017-10-11 16:59:22,895 | DEBUG | primary_selector.py ( 258) | _startSelection | Node1 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages 2017-10-11 16:59:22,896 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,897 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,897 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']}, 'Node3') 2017-10-11 16:59:22,897 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_REQUEST","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,897 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,898 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 16:59:22,898 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,898 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,898 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 16:59:22,898 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,907 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,908 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,909 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node3 into one transmission 2017-10-11 16:59:22,909 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', b'{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}']) 2017-10-11 16:59:22,909 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node3: b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' 2017-10-11 16:59:22,909 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' to Node3 2017-10-11 16:59:22,916 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,919 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,920 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']}, 'Node3') 2017-10-11 16:59:22,921 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,922 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,922 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}}, 'Node3') 2017-10-11 16:59:22,923 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,923 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,924 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}}, 'Node3') 2017-10-11 16:59:22,924 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,925 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} from Node3 2017-10-11 16:59:22,926 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,926 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,927 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} from Node3 2017-10-11 16:59:22,930 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,931 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,935 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:22,936 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}']} 2017-10-11 16:59:22,938 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}']}, 'Node2') 2017-10-11 16:59:22,939 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}']} 2017-10-11 16:59:22,940 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,940 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 16:59:22,941 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,941 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,943 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 16:59:22,944 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,945 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} from Node2 2017-10-11 16:59:22,945 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,945 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,946 | DEBUG | ledger_manager.py ( 309) | processLedgerStatus | Node1 found out from {'Node3', 'Node2'} that its ledger of type 1 is latest 2017-10-11 16:59:22,946 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} that it does not need catchup 2017-10-11 16:59:22,946 | DEBUG | node.py (1537) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 16:59:22,947 | DEBUG | replica.py (2130) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 16:59:22,947 | DEBUG | node.py (1918) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 16:59:22,947 | DEBUG | replica.py (2130) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 16:59:22,947 | DEBUG | node.py (1918) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 16:59:22,947 | DEBUG | node.py (2451) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 16:59:22,948 | DEBUG | monitor.py ( 183) | reset | Monitor being reset 2017-10-11 16:59:22,948 | DEBUG | node.py (1547) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 1 2017-10-11 16:59:22,948 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,949 | INFO | ledger_manager.py ( 831) | catchupCompleted | Node1 completed catching up ledger 1, caught up 0 in total 2017-10-11 16:59:22,949 | DEBUG | node.py (1653) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 16:59:22,950 | DEBUG | node.py (2451) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 16:59:22,951 | DEBUG | monitor.py ( 183) | reset | Monitor being reset 2017-10-11 16:59:22,951 | DEBUG | primary_selector.py ( 186) | _hasViewChangeQuorum | Node1 needs 1 ViewChangeDone messages 2017-10-11 16:59:22,951 | DEBUG | node.py (1625) | caught_up_for_current_view | Node1 does not have view change quorum for view 0 2017-10-11 16:59:22,951 | DEBUG | node.py (1608) | is_catchup_needed | Node1 is not caught up for the current view 0 2017-10-11 16:59:22,952 | DEBUG | node.py (1653) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 16:59:22,952 | DEBUG | node.py (1611) | is_catchup_needed | Node1 ordered till last prepared certificate 2017-10-11 16:59:22,952 | INFO | node.py (1593) | allLedgersCaughtUp | Node1 does not need any more catchups 2017-10-11 16:59:22,954 | DEBUG | primary_decider.py ( 131) | send | Node1's elector sending VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [(0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'), (1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')]} 2017-10-11 16:59:22,954 | INFO | primary_selector.py ( 192) | _hasViewChangeQuorum | Node1 got view change quorum (2 >= 2) 2017-10-11 16:59:22,956 | DEBUG | primary_selector.py ( 209) | has_view_change_from_primary | Node1 received ViewChangeDone from primary Node1 2017-10-11 16:59:22,956 | DEBUG | primary_selector.py ( 231) | has_sufficient_same_view_change_done_messages | Node1 found acceptable primary Node1 and ledger info ((0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'), (1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')) 2017-10-11 16:59:22,958 | DEBUG | primary_selector.py ( 272) | _startSelection | Node1 starting selection 2017-10-11 16:59:22,959 | DISPLAY | primary_selector.py ( 284) | _startSelection | Node1:0 selected primary Node1:0 for instance 0 (view 0) 2017-10-11 16:59:22,959 | INFO | node.py ( 482) | start_participating | Node1 started participating 2017-10-11 16:59:22,960 | INFO | replica.py ( 393) | primaryName | Node1:0 setting primaryName for view no 0 to: Node1:0 2017-10-11 16:59:22,960 | DEBUG | replica.py (1625) | _gc | Node1:0 cleaning up till (0, 0) 2017-10-11 16:59:22,960 | DEBUG | replica.py (1640) | _gc | Node1:0 found 0 3-phase keys to clean 2017-10-11 16:59:22,960 | DEBUG | replica.py (1642) | _gc | Node1:0 found 0 request keys to clean 2017-10-11 16:59:22,961 | INFO | replica.py ( 300) | h | Node1:0 set watermarks as 0 300 2017-10-11 16:59:22,961 | DISPLAY | primary_selector.py ( 307) | _startSelection | Node1:0 declares view change 0 as completed for instance 0, new primary is Node1:0, ledger info is [(0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'), (1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')] 2017-10-11 16:59:22,962 | DISPLAY | primary_selector.py ( 284) | _startSelection | Node1:1 selected primary Node2:1 for instance 1 (view 0) 2017-10-11 16:59:22,962 | INFO | replica.py ( 393) | primaryName | Node1:1 setting primaryName for view no 0 to: Node2:1 2017-10-11 16:59:22,963 | DEBUG | replica.py (1625) | _gc | Node1:1 cleaning up till (0, 0) 2017-10-11 16:59:22,963 | DEBUG | replica.py (1640) | _gc | Node1:1 found 0 3-phase keys to clean 2017-10-11 16:59:22,963 | DEBUG | replica.py (1642) | _gc | Node1:1 found 0 request keys to clean 2017-10-11 16:59:22,963 | INFO | replica.py ( 300) | h | Node1:1 set watermarks as 0 300 2017-10-11 16:59:22,964 | DEBUG | replica.py ( 494) | _setup_for_non_master | Node1:1 Setting last ordered for non-master as (0, 0) 2017-10-11 16:59:22,964 | DEBUG | replica.py ( 310) | last_ordered_3pc | Node1:1 set last ordered as (0, 0) 2017-10-11 16:59:22,967 | DISPLAY | primary_selector.py ( 307) | _startSelection | Node1:1 declares view change 0 as completed for instance 1, new primary is Node2:1, ledger info is [(0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'), (1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')] 2017-10-11 16:59:22,969 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} from Node2 2017-10-11 16:59:22,970 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,971 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:22,973 | DEBUG | node.py (2593) | send | Node1 sending message VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [(0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'), (1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')]} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 16:59:22,974 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"name":"Node1","viewNo":0,"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE"}' to Node3 2017-10-11 16:59:22,975 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"name":"Node1","viewNo":0,"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE"}' to Node3 2017-10-11 16:59:22,976 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"name":"Node1","viewNo":0,"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE"}' to Node4 2017-10-11 16:59:22,976 | WARNING | zstack.py ( 704) | transmit | Remote Node4 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings 2017-10-11 16:59:22,976 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"name":"Node1","viewNo":0,"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE"}' to Node4 2017-10-11 16:59:22,978 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"name":"Node1","viewNo":0,"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE"}' to Node2 2017-10-11 16:59:22,978 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"name":"Node1","viewNo":0,"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE"}' to Node2 2017-10-11 16:59:22,982 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 16:59:22,983 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,983 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']}, 'Node3') 2017-10-11 16:59:22,983 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}', '{"op":"MESSAGE_REQUEST","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS"}']} 2017-10-11 16:59:22,984 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,984 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 16:59:22,984 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,985 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,986 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 16:59:22,986 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:22,987 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:22,987 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'op': 'VIEW_CHANGE_DONE'}, 'Node3') 2017-10-11 16:59:22,988 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:22,988 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,989 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 16:59:22,990 | DEBUG | node.py (1261) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]}, 'Node3') 2017-10-11 16:59:22,993 | DEBUG | primary_selector.py ( 103) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node3 : VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:22,993 | DEBUG | message_processor.py ( 28) | discard | Node1 discarding message VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} because it already decided primary which is Node1:0 2017-10-11 16:59:22,994 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node3 into one transmission 2017-10-11 16:59:22,994 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', b'{"msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}']) 2017-10-11 16:59:22,995 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node3: b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' 2017-10-11 16:59:22,997 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' to Node3 2017-10-11 16:59:23,004 | WARNING | base_events.py (1308) | _run_once | Executing wait_for= cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.224 seconds 2017-10-11 16:59:23,264 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:23,264 | DEBUG | zstack.py ( 652) | handlePingPong | Node1 got ping from Node4 2017-10-11 16:59:23,264 | DEBUG | zstack.py ( 643) | sendPingPong | Node1 will be sending in batch 2017-10-11 16:59:23,266 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'po' to Node4 2017-10-11 16:59:23,266 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'po' to Node4 2017-10-11 16:59:23,369 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:23,369 | DEBUG | zstack.py ( 658) | handlePingPong | Node1 got pong from Node4 2017-10-11 16:59:23,370 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null}}']} 2017-10-11 16:59:23,370 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null}}']}, 'Node4') 2017-10-11 16:59:23,370 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null}}']} 2017-10-11 16:59:23,370 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,372 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}}, 'Node4') 2017-10-11 16:59:23,372 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,372 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,372 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}}, 'Node4') 2017-10-11 16:59:23,373 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,373 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,373 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}}, 'Node4') 2017-10-11 16:59:23,373 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,373 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,373 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_RESPONSE', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS', 'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}}, 'Node4') 2017-10-11 16:59:23,373 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg': {'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,374 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} from Node4 2017-10-11 16:59:23,374 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:23,375 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:23,375 | DEBUG | ledger_manager.py ( 309) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 2 is latest 2017-10-11 16:59:23,376 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} from Node4 2017-10-11 16:59:23,376 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:23,376 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 2 of size 0 with 0 2017-10-11 16:59:23,376 | DEBUG | ledger_manager.py ( 309) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 2 is latest 2017-10-11 16:59:23,377 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} from Node4 2017-10-11 16:59:23,378 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:23,378 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:23,378 | DEBUG | ledger_manager.py ( 309) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 1 is latest 2017-10-11 16:59:23,379 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} from Node4 2017-10-11 16:59:23,379 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:23,379 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 1 of size 9 with 9 2017-10-11 16:59:23,380 | DEBUG | ledger_manager.py ( 309) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 1 is latest 2017-10-11 16:59:23,382 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node3', 'Node2'} to {'Node3', 'Node4', 'Node2'} 2017-10-11 16:59:23,383 | INFO | keep_in_touch.py ( 96) | _connsChanged | Node1 now connected to Node4 2017-10-11 16:59:23,383 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from started_hungry to started 2017-10-11 16:59:23,383 | DEBUG | node.py ( 918) | checkInstances | Node1 choosing to start election on the basis of count 4 and nodes {'Node3', 'Node4', 'Node2'} 2017-10-11 16:59:23,383 | DEBUG | node.py ( 879) | send_current_state_to_lagging_node | Node1 sending current state CURRENT_STATE{'primary': [VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': ((0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'), (1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn'))}], 'viewNo': 0} to lagged node Node4 2017-10-11 16:59:23,384 | DEBUG | node.py (2593) | send | Node1 sending message CURRENT_STATE{'primary': [VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': ((0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'), (1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn'))}], 'viewNo': 0} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,384 | DEBUG | node.py (2593) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,384 | DEBUG | node.py (2593) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,385 | DEBUG | node.py (2593) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,389 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 4 msgs to Node4 into one transmission 2017-10-11 16:59:23,390 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"primary":[{"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node1","viewNo":0}],"viewNo":0,"op":"CURRENT_STATE"}', b'{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"ledgerId":0}', b'{"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":9,"ledgerId":1}', b'{"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":0,"ledgerId":2}']) 2017-10-11 16:59:23,390 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node4: b'{"op":"BATCH","signature":null,"messages":["{\\"primary\\":[{\\"ledgerInfo\\":[[0,4,\\"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA\\"],[1,9,\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\"],[2,0,\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0}],\\"viewNo\\":0,\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":4,\\"ledgerId\\":0}","{\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":9,\\"ledgerId\\":1}","{\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":0,\\"ledgerId\\":2}"]}' 2017-10-11 16:59:23,390 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"primary\\":[{\\"ledgerInfo\\":[[0,4,\\"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA\\"],[1,9,\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\"],[2,0,\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0}],\\"viewNo\\":0,\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":4,\\"ledgerId\\":0}","{\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":9,\\"ledgerId\\":1}","{\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"viewNo\\":null,\\"op\\":\\"LEDGER_STATUS\\",\\"txnSeqNo\\":0,\\"ledgerId\\":2}"]}' to Node4 2017-10-11 16:59:23,403 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:23,404 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"primary":[],"op":"CURRENT_STATE","viewNo":0}', '{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ledgerId":0,"viewNo":null,"txnSeqNo":4,"op":"LEDGER_STATUS","ppSeqNo":null}']} 2017-10-11 16:59:23,404 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"primary":[],"op":"CURRENT_STATE","viewNo":0}', '{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ledgerId":0,"viewNo":null,"txnSeqNo":4,"op":"LEDGER_STATUS","ppSeqNo":null}']}, 'Node4') 2017-10-11 16:59:23,404 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"primary":[],"op":"CURRENT_STATE","viewNo":0}', '{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ledgerId":0,"viewNo":null,"txnSeqNo":4,"op":"LEDGER_STATUS","ppSeqNo":null}']} 2017-10-11 16:59:23,404 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: CURRENT_STATE{'primary': [], 'viewNo': 0} 2017-10-11 16:59:23,404 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'primary': [], 'viewNo': 0, 'op': 'CURRENT_STATE'}, 'Node4') 2017-10-11 16:59:23,405 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'primary': [], 'viewNo': 0} 2017-10-11 16:59:23,405 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 16:59:23,405 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'LEDGER_STATUS', 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ppSeqNo': None, 'viewNo': None, 'txnSeqNo': 4, 'ledgerId': 0}, 'Node4') 2017-10-11 16:59:23,405 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 16:59:23,406 | DEBUG | node.py ( 884) | process_current_state_message | Node1 processing current state CURRENT_STATE{'primary': [], 'viewNo': 0} from Node4 2017-10-11 16:59:23,406 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} from Node4 2017-10-11 16:59:23,407 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 16:59:23,407 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 16:59:23,445 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 16:59:23,445 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE","viewNo":0,"name":"Node1"}']} 2017-10-11 16:59:23,446 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE","viewNo":0,"name":"Node1"}']}, 'Node4') 2017-10-11 16:59:23,446 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"ledgerInfo":[[0,4,"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA"],[1,9,"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"op":"VIEW_CHANGE_DONE","viewNo":0,"name":"Node1"}']} 2017-10-11 16:59:23,446 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,446 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 16:59:23,446 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,446 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,447 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 16:59:23,447 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,447 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,447 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 16:59:23,447 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,447 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,448 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 16:59:23,448 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 16:59:23,448 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:23,448 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'op': 'VIEW_CHANGE_DONE'}, 'Node4') 2017-10-11 16:59:23,448 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:23,449 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,450 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 0, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'ppSeqNo': None}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,452 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,453 | DEBUG | node.py (2593) | send | Node1 sending message MESSAGE_RESPONSE{'msg': LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 9, 'merkleRoot': '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp', 'ledgerId': 1, 'ppSeqNo': None}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 16:59:23,453 | DEBUG | node.py (1261) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]}, 'Node4') 2017-10-11 16:59:23,456 | DEBUG | primary_selector.py ( 103) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node4 : VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} 2017-10-11 16:59:23,456 | DEBUG | message_processor.py ( 28) | discard | Node1 discarding message VIEW_CHANGE_DONE{'name': 'Node1', 'viewNo': 0, 'ledgerInfo': [[0, 4, '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA'], [1, 9, '9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']]} because it already decided primary which is Node1:0 2017-10-11 16:59:23,457 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 4 msgs to Node4 into one transmission 2017-10-11 16:59:23,457 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', b'{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', b'{"msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', b'{"msg":{"ledgerId":1,"merkleRoot":"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp","ppSeqNo":null,"txnSeqNo":9,"viewNo":null},"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}']) 2017-10-11 16:59:23,457 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node4: b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' 2017-10-11 16:59:23,457 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":0,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"9Db2gWhHtb3sKcbdwCymaX3Uo82kf4wKUbTqmBVmFxAp\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":9,\\"viewNo\\":null},\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_RESPONSE\\"}"]}' to Node4 2017-10-11 16:59:31,441 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 1 2017-10-11 16:59:31,442 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 16:59:31,442 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 16:59:31,442 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 16:59:31,443 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 16:59:31,443 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 16:59:31,444 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 16:59:31,444 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 3 to run in 10 seconds 2017-10-11 16:59:36,455 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 16:59:36,456 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 16:59:36,457 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 16:59:36,458 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 16:59:41,461 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 3 2017-10-11 16:59:41,461 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 16:59:41,462 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 16:59:41,462 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 16:59:41,464 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 16:59:41,464 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 16:59:41,465 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 16:59:41,465 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 4 to run in 10 seconds 2017-10-11 16:59:51,463 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 16:59:51,464 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 16:59:51,464 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 16:59:51,464 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 16:59:51,477 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 4 2017-10-11 16:59:51,477 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 16:59:51,478 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 16:59:51,478 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 16:59:51,478 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 16:59:51,478 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 16:59:51,478 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 16:59:51,478 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 5 to run in 10 seconds 2017-10-11 17:00:01,478 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 5 2017-10-11 17:00:01,479 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:00:01,479 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:00:01,479 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:00:01,479 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:00:01,479 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:00:01,479 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:00:01,479 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 6 to run in 10 seconds 2017-10-11 17:00:06,471 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:00:06,472 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:00:06,472 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:00:06,473 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:00:11,493 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 6 2017-10-11 17:00:11,494 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:00:11,494 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:00:11,495 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:00:11,495 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:00:11,495 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:00:11,495 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:00:11,496 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 7 to run in 10 seconds 2017-10-11 17:00:21,428 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkNodeRequestSpike with id 2 2017-10-11 17:00:21,428 | DEBUG | node.py (2026) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:00:21,428 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:00:21,428 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 8 to run in 60 seconds 2017-10-11 17:00:21,428 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 1 2017-10-11 17:00:21,428 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:00:21,429 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 2 to run in 60 seconds 2017-10-11 17:00:21,478 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:00:21,479 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:00:21,480 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:00:21,480 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:00:21,508 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 7 2017-10-11 17:00:21,508 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:00:21,508 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:00:21,508 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:00:21,508 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:00:21,508 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:00:21,509 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:00:21,509 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 9 to run in 10 seconds 2017-10-11 17:00:31,510 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 9 2017-10-11 17:00:31,510 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:00:31,511 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:00:31,511 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:00:31,511 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:00:31,511 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:00:31,511 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:00:31,511 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 10 to run in 10 seconds 2017-10-11 17:00:36,495 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:00:36,496 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:00:36,496 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:00:36,497 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:00:38,009 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:00:38,010 | DEBUG | zstack.py ( 652) | handlePingPong | Node1C got ping from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:00:38,011 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'po' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:00:38,011 | DEBUG | zstack.py ( 635) | sendPingPong | Node1C ponged b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:00:38,049 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:00:38,050 | TRACE | node.py (1444) | validateClientMsg | Node1C received CLIENT message: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 17:00:38,050 | DISPLAY | node.py (1490) | processClientInBox | Node1C processing b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' request LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} 2017-10-11 17:00:38,051 | DEBUG | ledger_manager.py ( 244) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'txnSeqNo': 4, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'ledgerId': 0, 'ppSeqNo': None} from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:00:38,055 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:00:38,056 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'{"merkleRoot":"6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA","ppSeqNo":null,"viewNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"ledgerId":0}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:00:38,070 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:00:38,071 | DEBUG | ledger_manager.py ( 965) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:00:41,520 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 10 2017-10-11 17:00:41,521 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:00:41,521 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:00:41,521 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:00:41,521 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:00:41,522 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:00:41,522 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:00:41,523 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 11 to run in 10 seconds 2017-10-11 17:00:51,500 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:00:51,500 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:00:51,500 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:00:51,501 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:00:51,528 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 11 2017-10-11 17:00:51,528 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:00:51,528 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:00:51,529 | DEBUG | monitor.py ( 290) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:00:51,530 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:00:51,530 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:00:51,530 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:00:51,543 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 12 to run in 10 seconds 2017-10-11 17:00:52,114 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:00:52,115 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,116 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,117 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on request 1507741252094258 2017-10-11 17:00:52,117 | TRACE | node.py (1444) | validateClientMsg | Node1C received CLIENT message: SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:52,118 | DISPLAY | node.py (1490) | processClientInBox | Node1C processing b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' request SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:52,130 | DEBUG | node.py (1751) | processRequest | Node1 received client request: SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:00:52,130 | TRACE | propagator.py ( 159) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:52,130 | INFO | propagator.py ( 145) | propagate | Node1 propagating V4SGRU86Z58d6TV7PBUe6f request 1507741252094258 from client b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:00:52,131 | DEBUG | node.py (2593) | send | Node1 sending message PROPAGATE{'request': {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:52,132 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since not finalised 2017-10-11 17:00:52,132 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'{"op":"REQACK","reqId":1507741252094258,"identifier":"V4SGRU86Z58d6TV7PBUe6f"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:00:52,136 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh","reqId":1507741252094258,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT1"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node3 2017-10-11 17:00:52,137 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh","reqId":1507741252094258,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT1"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node3 2017-10-11 17:00:52,137 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh","reqId":1507741252094258,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT1"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node4 2017-10-11 17:00:52,138 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh","reqId":1507741252094258,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT1"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node4 2017-10-11 17:00:52,138 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh","reqId":1507741252094258,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT1"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node2 2017-10-11 17:00:52,139 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh","reqId":1507741252094258,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT1"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node2 2017-10-11 17:00:52,141 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:00:52,143 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,146 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741252094258 2017-10-11 17:00:52,151 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,152 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'op': 'PROPAGATE', 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node2') 2017-10-11 17:00:52,152 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,153 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,153 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741252094258 2017-10-11 17:00:52,153 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'request': {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,153 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'PROPAGATE', 'request': {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node4') 2017-10-11 17:00:52,154 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,154 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,156 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,157 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:52,158 | DEBUG | propagator.py ( 208) | forward | Node1 forwarding request ('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258) to 2 replicas 2017-10-11 17:00:52,163 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,163 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,163 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:52,164 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since already forwarded 2017-10-11 17:00:52,170 | INFO | replica.py ( 642) | create3PCBatch | Node1:0 creating batch 1 for ledger 1 with state root b'\xe6\x82\x0f\x99\xf6@c\xaf\x02\x05\xec\xd1\xb3m\x1f\xbb\x0c\xcd\xa2\xedv\x1ef\xa6\xa2\xec\x9f\xdfI\xa0Cw' 2017-10-11 17:00:52,172 | DISPLAY | replica.py ( 673) | create3PCBatch | Node1:0 created a PRE-PREPARE with 1 requests for ledger 1 2017-10-11 17:00:52,175 | INFO | replica.py (2064) | send | Node1:0 sending PrePrepare 2017-10-11 17:00:52,175 | TRACE | replica.py (2065) | send | Node1:0 sending PREPREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258)]} 2017-10-11 17:00:52,176 | DEBUG | replica.py ( 594) | trackBatches | Node1:0 tracking batch for PREPREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258)]} with state root b'\xe6\x82\x0f\x99\xf6@c\xaf\x02\x05\xec\xd1\xb3m\x1f\xbb\x0c\xcd\xa2\xedv\x1ef\xa6\xa2\xec\x9f\xdfI\xa0Cw' 2017-10-11 17:00:52,176 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:00:52,177 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,178 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741252094258 2017-10-11 17:00:52,178 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,178 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'PROPAGATE', 'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node3') 2017-10-11 17:00:52,178 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,190 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'reqId': 1507741252094258, 'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:52,190 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT1|type:1|reqId:1507741252094258 2017-10-11 17:00:52,191 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:52,191 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh', 'reqId': 1507741252094258, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT1'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since already forwarded 2017-10-11 17:00:52,193 | DEBUG | node.py (2593) | send | Node1 sending message PREPREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258)]} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:52,196 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741252,"txnRootHash":"2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3","ppSeqNo":1,"stateRootHash":"HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741252094258]],"discarded":1,"instId":0}' to Node3 2017-10-11 17:00:52,196 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741252,"txnRootHash":"2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3","ppSeqNo":1,"stateRootHash":"HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741252094258]],"discarded":1,"instId":0}' to Node3 2017-10-11 17:00:52,196 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741252,"txnRootHash":"2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3","ppSeqNo":1,"stateRootHash":"HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741252094258]],"discarded":1,"instId":0}' to Node4 2017-10-11 17:00:52,197 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741252,"txnRootHash":"2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3","ppSeqNo":1,"stateRootHash":"HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741252094258]],"discarded":1,"instId":0}' to Node4 2017-10-11 17:00:52,197 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741252,"txnRootHash":"2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3","ppSeqNo":1,"stateRootHash":"HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741252094258]],"discarded":1,"instId":0}' to Node2 2017-10-11 17:00:52,197 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741252,"txnRootHash":"2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3","ppSeqNo":1,"stateRootHash":"HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741252094258]],"discarded":1,"instId":0}' to Node2 2017-10-11 17:00:52,215 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:00:52,216 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'stateRootHash': None, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741252094258]]} 2017-10-11 17:00:52,216 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741252, 'op': 'PREPREPARE', 'txnRootHash': None, 'stateRootHash': None, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741252094258]], 'ledgerId': 1, 'ppSeqNo': 1}, 'Node2') 2017-10-11 17:00:52,217 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'stateRootHash': None, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741252094258]]} 2017-10-11 17:00:52,220 | DEBUG | replica.py ( 790) | processPrePrepare | Node1:1 received PRE-PREPARE(0, 1) from Node2:1 at 2270337.677546607 2017-10-11 17:00:52,220 | DEBUG | replica.py ( 903) | doPrepare | Node1:1 Sending PREPARE(0, 1) at 2270337.678161757 2017-10-11 17:00:52,221 | INFO | replica.py (2064) | send | Node1:1 sending Prepare 2017-10-11 17:00:52,221 | TRACE | replica.py (2065) | send | Node1:1 sending PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:52,221 | DEBUG | replica.py ( 886) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:52,222 | DEBUG | replica.py ( 594) | trackBatches | Node1:1 tracking batch for PREPREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'stateRootHash': None, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258)]} with state root None 2017-10-11 17:00:52,222 | DEBUG | replica.py ( 809) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(0, 1) 2017-10-11 17:00:52,227 | DEBUG | node.py (2593) | send | Node1 sending message PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:52,229 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:00:52,230 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,230 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 0, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3'}, 'Node4') 2017-10-11 17:00:52,230 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,231 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,231 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741252, 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 0, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'ppSeqNo': 1}, 'Node2') 2017-10-11 17:00:52,231 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,234 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"instId":1,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPARE","ppTime":1507741252,"ppSeqNo":1,"stateRootHash":null,"txnRootHash":null}' to Node3 2017-10-11 17:00:52,234 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"instId":1,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPARE","ppTime":1507741252,"ppSeqNo":1,"stateRootHash":null,"txnRootHash":null}' to Node3 2017-10-11 17:00:52,235 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"instId":1,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPARE","ppTime":1507741252,"ppSeqNo":1,"stateRootHash":null,"txnRootHash":null}' to Node4 2017-10-11 17:00:52,235 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"instId":1,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPARE","ppTime":1507741252,"ppSeqNo":1,"stateRootHash":null,"txnRootHash":null}' to Node4 2017-10-11 17:00:52,236 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"instId":1,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPARE","ppTime":1507741252,"ppSeqNo":1,"stateRootHash":null,"txnRootHash":null}' to Node2 2017-10-11 17:00:52,236 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"instId":1,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","viewNo":0,"op":"PREPARE","ppTime":1507741252,"ppSeqNo":1,"stateRootHash":null,"txnRootHash":null}' to Node2 2017-10-11 17:00:52,238 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 1) from Node4:0 2017-10-11 17:00:52,238 | DEBUG | replica.py ( 886) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,239 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 1) 2017-10-11 17:00:52,240 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 1) from Node2:0 2017-10-11 17:00:52,241 | DEBUG | replica.py ( 922) | doCommit | Node1:0 Sending COMMIT(0, 1) at 2270337.698428898 2017-10-11 17:00:52,241 | INFO | replica.py (2064) | send | Node1:0 sending Commit 2017-10-11 17:00:52,241 | TRACE | replica.py (2065) | send | Node1:0 sending COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} 2017-10-11 17:00:52,241 | DEBUG | replica.py ( 898) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} commits where f is 1 2017-10-11 17:00:52,242 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 1) 2017-10-11 17:00:52,243 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:00:52,243 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:52,244 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'instId': 1, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': None, 'txnRootHash': None}, 'Node4') 2017-10-11 17:00:52,244 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:52,247 | DEBUG | node.py (2593) | send | Node1 sending message COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:52,247 | DEBUG | replica.py ( 834) | processPrepare | Node1:1 received PREPARE(0, 1) from Node4:1 2017-10-11 17:00:52,247 | DEBUG | replica.py ( 922) | doCommit | Node1:1 Sending COMMIT(0, 1) at 2270337.705308745 2017-10-11 17:00:52,248 | INFO | replica.py (2064) | send | Node1:1 sending Commit 2017-10-11 17:00:52,248 | TRACE | replica.py (2065) | send | Node1:1 sending COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,248 | DEBUG | replica.py ( 898) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} commits where f is 1 2017-10-11 17:00:52,249 | DEBUG | replica.py ( 845) | processPrepare | Node1:1 processed incoming PREPARE (0, 1) 2017-10-11 17:00:52,251 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":1}' to Node3 2017-10-11 17:00:52,252 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":1}' to Node3 2017-10-11 17:00:52,252 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":1}' to Node4 2017-10-11 17:00:52,253 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":1}' to Node4 2017-10-11 17:00:52,253 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":1}' to Node2 2017-10-11 17:00:52,254 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":1}' to Node2 2017-10-11 17:00:52,256 | DEBUG | node.py (2593) | send | Node1 sending message COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:52,258 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:00:52,259 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,259 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741252, 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0, 'ppSeqNo': 1, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59'}, 'Node3') 2017-10-11 17:00:52,259 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,260 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} 2017-10-11 17:00:52,260 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 0, 'ppSeqNo': 1}, 'Node4') 2017-10-11 17:00:52,260 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} 2017-10-11 17:00:52,263 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":1}' to Node3 2017-10-11 17:00:52,263 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":1}' to Node3 2017-10-11 17:00:52,264 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":1}' to Node4 2017-10-11 17:00:52,264 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":1}' to Node4 2017-10-11 17:00:52,265 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":1}' to Node2 2017-10-11 17:00:52,265 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":1}' to Node2 2017-10-11 17:00:52,266 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 1) from Node3:0 2017-10-11 17:00:52,267 | DEBUG | replica.py ( 886) | tryCommit | Node1:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0} 2017-10-11 17:00:52,267 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 1) 2017-10-11 17:00:52,268 | DEBUG | replica.py ( 863) | processCommit | Node1:0 received COMMIT(0, 1) from Node4:0 2017-10-11 17:00:52,268 | DEBUG | replica.py ( 898) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} commits where f is 1 2017-10-11 17:00:52,268 | DEBUG | replica.py ( 874) | processCommit | Node1:0 processed incoming COMMIT(0, 1) 2017-10-11 17:00:52,269 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:00:52,270 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} 2017-10-11 17:00:52,271 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 0, 'ppSeqNo': 1}, 'Node2') 2017-10-11 17:00:52,271 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} 2017-10-11 17:00:52,272 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"COMMIT","ppSeqNo":1,"viewNo":0,"instId":0}', '{"ppSeqNo":1,"op":"PREPARE","ppTime":1507741252,"stateRootHash":null,"viewNo":0,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","txnRootHash":null,"instId":1}']} 2017-10-11 17:00:52,272 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"op":"COMMIT","ppSeqNo":1,"viewNo":0,"instId":0}', '{"ppSeqNo":1,"op":"PREPARE","ppTime":1507741252,"stateRootHash":null,"viewNo":0,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","txnRootHash":null,"instId":1}']}, 'Node3') 2017-10-11 17:00:52,272 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"op":"COMMIT","ppSeqNo":1,"viewNo":0,"instId":0}', '{"ppSeqNo":1,"op":"PREPARE","ppTime":1507741252,"stateRootHash":null,"viewNo":0,"digest":"cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59","txnRootHash":null,"instId":1}']} 2017-10-11 17:00:52,273 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} 2017-10-11 17:00:52,273 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'ppSeqNo': 1, 'instId': 0}, 'Node3') 2017-10-11 17:00:52,273 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} 2017-10-11 17:00:52,274 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:52,274 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741252, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 1, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': None, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59'}, 'Node3') 2017-10-11 17:00:52,274 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:52,275 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,275 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 1, 'ppSeqNo': 1}, 'Node4') 2017-10-11 17:00:52,275 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,279 | DEBUG | replica.py ( 863) | processCommit | Node1:0 received COMMIT(0, 1) from Node2:0 2017-10-11 17:00:52,279 | TRACE | replica.py ( 894) | tryOrder | Node1:0 returning request to node 2017-10-11 17:00:52,280 | INFO | replica.py (1395) | doOrder | Node1:0 ordering COMMIT(0, 1) 2017-10-11 17:00:52,280 | DEBUG | replica.py ( 310) | last_ordered_3pc | Node1:0 set last ordered as (0, 1) 2017-10-11 17:00:52,280 | INFO | replica.py (2064) | send | Node1:0 sending Ordered 2017-10-11 17:00:52,281 | TRACE | replica.py (2065) | send | Node1:0 sending ORDERED{'ppTime': 1507741252, 'txnRootHash': '2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3', 'instId': 0, 'viewNo': 0, 'stateRootHash': 'HysNuYRsJBXaczDjpZerFF2eevtjVVJDqVDQU2HJZwsZ', 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258)], 'ledgerId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,281 | DEBUG | replica.py (1426) | order_3pc_key | Node1:0 ordered request (0, 1) 2017-10-11 17:00:52,281 | DEBUG | replica.py (1494) | _newCheckpointState | Node1:0 adding new checkpoint state for (1, 100) 2017-10-11 17:00:52,282 | DEBUG | replica.py ( 874) | processCommit | Node1:0 processed incoming COMMIT(0, 1) 2017-10-11 17:00:52,282 | TRACE | message_processor.py ( 28) | discard | Node1:0 discarding message COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 1} because already ordered 3 phase message 2017-10-11 17:00:52,282 | DEBUG | replica.py ( 834) | processPrepare | Node1:1 received PREPARE(0, 1) from Node3:1 2017-10-11 17:00:52,283 | DEBUG | replica.py ( 886) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppTime': 1507741252, 'ppSeqNo': 1, 'digest': 'cff18401c6bd55c81642da455db0a1d5c557f47fdd2da9646a4f5163b9170a59', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:52,283 | DEBUG | replica.py ( 845) | processPrepare | Node1:1 processed incoming PREPARE (0, 1) 2017-10-11 17:00:52,284 | DEBUG | replica.py ( 863) | processCommit | Node1:1 received COMMIT(0, 1) from Node4:1 2017-10-11 17:00:52,284 | DEBUG | replica.py ( 898) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} commits where f is 1 2017-10-11 17:00:52,284 | DEBUG | replica.py ( 874) | processCommit | Node1:1 processed incoming COMMIT(0, 1) 2017-10-11 17:00:52,285 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:00:52,286 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,286 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'ppSeqNo': 1, 'instId': 1}, 'Node3') 2017-10-11 17:00:52,287 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,290 | DEBUG | node.py (1881) | processOrdered | Node1 executing Ordered batch 0 1 of 1 requests 2017-10-11 17:00:52,302 | DEBUG | ledger.py ( 68) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:00:52,303 | DEBUG | node.py (2374) | sendReplyToClient | Node1 sending reply for ('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258) to client 2017-10-11 17:00:52,303 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'{"op":"REPLY","result":{"auditPath":["GbwFpoGxJMdHfwZJmT2WpE3rtWD2cW2Cevq7bC75nkyb","HYRnUmRJ2hQMtr1smmXw3sqWyQyKTzCeTh5D9EweiD8z"],"type":"1","dest":"542MVr22zcHbVyGzaXmbT1","reqId":1507741252094258,"signature":"4RSVWEnDKexCuWpxLdbhmXbfQprab1QFbNSF2J9eWivyHa1jjuemxp5i6iKToEe7dZagtgAsifzeVZYg4Jf6C1Vh","txnTime":1507741252,"identifier":"V4SGRU86Z58d6TV7PBUe6f","rootHash":"2zbyjmFFrtYpWE6aVggU7hoc4uz2B1K5BgNnExpnRRy3","seqNo":10}}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:00:52,303 | DEBUG | node.py (2295) | executeBatch | Node1 storing 3PC key (0, 1) for ledger 1 range (10, 10) 2017-10-11 17:00:52,304 | DEBUG | replica.py ( 863) | processCommit | Node1:1 received COMMIT(0, 1) from Node3:1 2017-10-11 17:00:52,304 | TRACE | replica.py ( 894) | tryOrder | Node1:1 returning request to node 2017-10-11 17:00:52,304 | INFO | replica.py (1395) | doOrder | Node1:1 ordering COMMIT(0, 1) 2017-10-11 17:00:52,304 | DEBUG | replica.py ( 310) | last_ordered_3pc | Node1:1 set last ordered as (0, 1) 2017-10-11 17:00:52,305 | INFO | replica.py (2064) | send | Node1:1 sending Ordered 2017-10-11 17:00:52,305 | TRACE | replica.py (2065) | send | Node1:1 sending ORDERED{'ppTime': 1507741252, 'txnRootHash': None, 'instId': 1, 'viewNo': 0, 'stateRootHash': None, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741252094258)], 'ledgerId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,305 | DEBUG | replica.py (1426) | order_3pc_key | Node1:1 ordered request (0, 1) 2017-10-11 17:00:52,305 | DEBUG | replica.py (1494) | _newCheckpointState | Node1:1 adding new checkpoint state for (1, 100) 2017-10-11 17:00:52,305 | DEBUG | replica.py ( 874) | processCommit | Node1:1 processed incoming COMMIT(0, 1) 2017-10-11 17:00:52,306 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:00:52,307 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,307 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 1, 'ppSeqNo': 1}, 'Node2') 2017-10-11 17:00:52,307 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} 2017-10-11 17:00:52,309 | TRACE | node.py (1892) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:00:52,310 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:00:52,310 | TRACE | message_processor.py ( 28) | discard | Node1:1 discarding message COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 1} because already ordered 3 phase message 2017-10-11 17:00:52,323 | WARNING | base_events.py (1308) | _run_once | Executing wait_for= cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.109 seconds 2017-10-11 17:00:56,183 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:00:56,184 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,186 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,193 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on request 1507741256159186 2017-10-11 17:00:56,194 | TRACE | node.py (1444) | validateClientMsg | Node1C received CLIENT message: SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:56,195 | DISPLAY | node.py (1490) | processClientInBox | Node1C processing b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' request SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:56,211 | DEBUG | node.py (1751) | processRequest | Node1 received client request: SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:00:56,214 | TRACE | propagator.py ( 159) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:56,215 | INFO | propagator.py ( 145) | propagate | Node1 propagating V4SGRU86Z58d6TV7PBUe6f request 1507741256159186 from client b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:00:56,215 | DEBUG | node.py (2593) | send | Node1 sending message PROPAGATE{'request': {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:56,215 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since not finalised 2017-10-11 17:00:56,215 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'{"op":"REQACK","reqId":1507741256159186,"identifier":"V4SGRU86Z58d6TV7PBUe6f"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:00:56,216 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3","reqId":1507741256159186,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT2"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node3 2017-10-11 17:00:56,228 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3","reqId":1507741256159186,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT2"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node3 2017-10-11 17:00:56,229 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3","reqId":1507741256159186,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT2"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node4 2017-10-11 17:00:56,229 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3","reqId":1507741256159186,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT2"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node4 2017-10-11 17:00:56,229 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3","reqId":1507741256159186,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT2"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node2 2017-10-11 17:00:56,229 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3","reqId":1507741256159186,"operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT2"},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node2 2017-10-11 17:00:56,231 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:00:56,232 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,232 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741256159186 2017-10-11 17:00:56,232 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,232 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'op': 'PROPAGATE', 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node2') 2017-10-11 17:00:56,233 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,233 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,233 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741256159186 2017-10-11 17:00:56,234 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'request': {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,234 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'PROPAGATE', 'request': {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node4') 2017-10-11 17:00:56,234 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,234 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,236 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741256159186 2017-10-11 17:00:56,236 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,248 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'PROPAGATE', 'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node3') 2017-10-11 17:00:56,248 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,248 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,249 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,249 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:56,249 | DEBUG | propagator.py ( 208) | forward | Node1 forwarding request ('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186) to 2 replicas 2017-10-11 17:00:56,249 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,250 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,250 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:56,250 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since already forwarded 2017-10-11 17:00:56,250 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'reqId': 1507741256159186, 'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:00:56,253 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:dest:542MVr22zcHbVyGzaXmbT2|type:1|reqId:1507741256159186 2017-10-11 17:00:56,253 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:00:56,253 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3', 'reqId': 1507741256159186, 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT2'}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since already forwarded 2017-10-11 17:00:56,257 | INFO | replica.py ( 642) | create3PCBatch | Node1:0 creating batch 2 for ledger 1 with state root b'\xfcLzv\x00\x9b#\xb5j\xc5\x1c\xde\x84a\xd6\xf5\xb0[\x9e\x9fB\x11\xc0\xb0\x13\xef\x84\x98@\xaa\x0e\xdc' 2017-10-11 17:00:56,259 | DISPLAY | replica.py ( 673) | create3PCBatch | Node1:0 created a PRE-PREPARE with 1 requests for ledger 1 2017-10-11 17:00:56,259 | INFO | replica.py (2064) | send | Node1:0 sending PrePrepare 2017-10-11 17:00:56,259 | TRACE | replica.py (2065) | send | Node1:0 sending PREPREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186)]} 2017-10-11 17:00:56,259 | DEBUG | replica.py ( 594) | trackBatches | Node1:0 tracking batch for PREPREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186)]} with state root b'\xfcLzv\x00\x9b#\xb5j\xc5\x1c\xde\x84a\xd6\xf5\xb0[\x9e\x9fB\x11\xc0\xb0\x13\xef\x84\x98@\xaa\x0e\xdc' 2017-10-11 17:00:56,270 | DEBUG | node.py (2593) | send | Node1 sending message PREPREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186)]} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:56,274 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741256,"txnRootHash":"CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw","ppSeqNo":2,"stateRootHash":"8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741256159186]],"discarded":1,"instId":0}' to Node3 2017-10-11 17:00:56,275 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741256,"txnRootHash":"CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw","ppSeqNo":2,"stateRootHash":"8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741256159186]],"discarded":1,"instId":0}' to Node3 2017-10-11 17:00:56,275 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741256,"txnRootHash":"CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw","ppSeqNo":2,"stateRootHash":"8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741256159186]],"discarded":1,"instId":0}' to Node4 2017-10-11 17:00:56,275 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741256,"txnRootHash":"CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw","ppSeqNo":2,"stateRootHash":"8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741256159186]],"discarded":1,"instId":0}' to Node4 2017-10-11 17:00:56,275 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741256,"txnRootHash":"CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw","ppSeqNo":2,"stateRootHash":"8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741256159186]],"discarded":1,"instId":0}' to Node2 2017-10-11 17:00:56,276 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPREPARE","ledgerId":1,"ppTime":1507741256,"txnRootHash":"CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw","ppSeqNo":2,"stateRootHash":"8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741256159186]],"discarded":1,"instId":0}' to Node2 2017-10-11 17:00:56,280 | WARNING | base_events.py (1308) | _run_once | Executing wait_for= cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.100 seconds 2017-10-11 17:00:56,311 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:00:56,311 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'stateRootHash': None, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741256159186]]} 2017-10-11 17:00:56,312 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741256, 'op': 'PREPREPARE', 'txnRootHash': None, 'stateRootHash': None, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741256159186]], 'ledgerId': 1, 'ppSeqNo': 2}, 'Node2') 2017-10-11 17:00:56,312 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'stateRootHash': None, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741256159186]]} 2017-10-11 17:00:56,312 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} 2017-10-11 17:00:56,312 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741256, 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0, 'ppSeqNo': 2, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42'}, 'Node3') 2017-10-11 17:00:56,315 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} 2017-10-11 17:00:56,315 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} 2017-10-11 17:00:56,316 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 0, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw'}, 'Node4') 2017-10-11 17:00:56,316 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} 2017-10-11 17:00:56,320 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 2) from Node3:0 2017-10-11 17:00:56,321 | DEBUG | replica.py ( 886) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} 2017-10-11 17:00:56,321 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 2) 2017-10-11 17:00:56,321 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 2) from Node4:0 2017-10-11 17:00:56,321 | DEBUG | replica.py ( 922) | doCommit | Node1:0 Sending COMMIT(0, 2) at 2270341.779062423 2017-10-11 17:00:56,321 | INFO | replica.py (2064) | send | Node1:0 sending Commit 2017-10-11 17:00:56,321 | TRACE | replica.py (2065) | send | Node1:0 sending COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} 2017-10-11 17:00:56,322 | DEBUG | replica.py ( 898) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} commits where f is 1 2017-10-11 17:00:56,322 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 2) 2017-10-11 17:00:56,322 | DEBUG | replica.py ( 790) | processPrePrepare | Node1:1 received PRE-PREPARE(0, 2) from Node2:1 at 2270341.779738102 2017-10-11 17:00:56,322 | DEBUG | replica.py ( 903) | doPrepare | Node1:1 Sending PREPARE(0, 2) at 2270341.780083058 2017-10-11 17:00:56,322 | INFO | replica.py (2064) | send | Node1:1 sending Prepare 2017-10-11 17:00:56,323 | TRACE | replica.py (2065) | send | Node1:1 sending PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:56,323 | DEBUG | replica.py ( 886) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:56,323 | DEBUG | replica.py ( 594) | trackBatches | Node1:1 tracking batch for PREPREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'stateRootHash': None, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186)]} with state root None 2017-10-11 17:00:56,323 | DEBUG | replica.py ( 809) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(0, 2) 2017-10-11 17:00:56,326 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:00:56,328 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"COMMIT","ppSeqNo":2,"viewNo":0,"instId":0}', '{"ppSeqNo":2,"op":"PREPARE","ppTime":1507741256,"stateRootHash":null,"viewNo":0,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","txnRootHash":null,"instId":1}']} 2017-10-11 17:00:56,328 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"op":"COMMIT","ppSeqNo":2,"viewNo":0,"instId":0}', '{"ppSeqNo":2,"op":"PREPARE","ppTime":1507741256,"stateRootHash":null,"viewNo":0,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","txnRootHash":null,"instId":1}']}, 'Node3') 2017-10-11 17:00:56,328 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"op":"COMMIT","ppSeqNo":2,"viewNo":0,"instId":0}', '{"ppSeqNo":2,"op":"PREPARE","ppTime":1507741256,"stateRootHash":null,"viewNo":0,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","txnRootHash":null,"instId":1}']} 2017-10-11 17:00:56,328 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} 2017-10-11 17:00:56,329 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'ppSeqNo': 2, 'instId': 0}, 'Node3') 2017-10-11 17:00:56,329 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} 2017-10-11 17:00:56,329 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:56,329 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741256, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 2, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': None, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42'}, 'Node3') 2017-10-11 17:00:56,329 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:56,330 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","ppSeqNo":2,"viewNo":0}', '{"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"txnRootHash":null,"ppTime":1507741256,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","ppSeqNo":2}']} 2017-10-11 17:00:56,330 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","ppSeqNo":2,"viewNo":0}', '{"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"txnRootHash":null,"ppTime":1507741256,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","ppSeqNo":2}']}, 'Node4') 2017-10-11 17:00:56,330 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","ppSeqNo":2,"viewNo":0}', '{"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"txnRootHash":null,"ppTime":1507741256,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","ppSeqNo":2}']} 2017-10-11 17:00:56,333 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} 2017-10-11 17:00:56,333 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 0, 'ppSeqNo': 2}, 'Node4') 2017-10-11 17:00:56,333 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} 2017-10-11 17:00:56,334 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:56,334 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 1, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': None, 'txnRootHash': None}, 'Node4') 2017-10-11 17:00:56,334 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:56,338 | DEBUG | node.py (2593) | send | Node1 sending message COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:56,339 | DEBUG | node.py (2593) | send | Node1 sending message PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:56,339 | DEBUG | replica.py ( 863) | processCommit | Node1:0 received COMMIT(0, 2) from Node3:0 2017-10-11 17:00:56,339 | DEBUG | replica.py ( 898) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} commits where f is 1 2017-10-11 17:00:56,340 | DEBUG | replica.py ( 874) | processCommit | Node1:0 processed incoming COMMIT(0, 2) 2017-10-11 17:00:56,340 | DEBUG | replica.py ( 863) | processCommit | Node1:0 received COMMIT(0, 2) from Node4:0 2017-10-11 17:00:56,340 | TRACE | replica.py ( 894) | tryOrder | Node1:0 returning request to node 2017-10-11 17:00:56,340 | INFO | replica.py (1395) | doOrder | Node1:0 ordering COMMIT(0, 2) 2017-10-11 17:00:56,340 | DEBUG | replica.py ( 310) | last_ordered_3pc | Node1:0 set last ordered as (0, 2) 2017-10-11 17:00:56,340 | INFO | replica.py (2064) | send | Node1:0 sending Ordered 2017-10-11 17:00:56,341 | TRACE | replica.py (2065) | send | Node1:0 sending ORDERED{'ppTime': 1507741256, 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0, 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186)], 'ledgerId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,341 | DEBUG | replica.py (1426) | order_3pc_key | Node1:0 ordered request (0, 2) 2017-10-11 17:00:56,341 | DEBUG | replica.py ( 874) | processCommit | Node1:0 processed incoming COMMIT(0, 2) 2017-10-11 17:00:56,341 | DEBUG | replica.py ( 834) | processPrepare | Node1:1 received PREPARE(0, 2) from Node3:1 2017-10-11 17:00:56,341 | DEBUG | replica.py ( 922) | doCommit | Node1:1 Sending COMMIT(0, 2) at 2270341.799076781 2017-10-11 17:00:56,341 | INFO | replica.py (2064) | send | Node1:1 sending Commit 2017-10-11 17:00:56,341 | TRACE | replica.py (2065) | send | Node1:1 sending COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,342 | DEBUG | replica.py ( 898) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} commits where f is 1 2017-10-11 17:00:56,342 | DEBUG | replica.py ( 845) | processPrepare | Node1:1 processed incoming PREPARE (0, 2) 2017-10-11 17:00:56,342 | DEBUG | replica.py ( 834) | processPrepare | Node1:1 received PREPARE(0, 2) from Node4:1 2017-10-11 17:00:56,342 | DEBUG | replica.py ( 886) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:00:56,342 | DEBUG | replica.py ( 845) | processPrepare | Node1:1 processed incoming PREPARE (0, 2) 2017-10-11 17:00:56,343 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:00:56,347 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} 2017-10-11 17:00:56,347 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741256, 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'instId': 0, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'ppSeqNo': 2}, 'Node2') 2017-10-11 17:00:56,347 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} 2017-10-11 17:00:56,350 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node3 into one transmission 2017-10-11 17:00:56,350 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":2}', b'{"instId":1,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPARE","ppTime":1507741256,"ppSeqNo":2,"stateRootHash":null,"txnRootHash":null}']) 2017-10-11 17:00:56,351 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node3: b'{"op":"BATCH","signature":null,"messages":["{\\"viewNo\\":0,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"ppSeqNo\\":2}","{\\"instId\\":1,\\"digest\\":\\"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42\\",\\"viewNo\\":0,\\"op\\":\\"PREPARE\\",\\"ppTime\\":1507741256,\\"ppSeqNo\\":2,\\"stateRootHash\\":null,\\"txnRootHash\\":null}"]}' 2017-10-11 17:00:56,351 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"viewNo\\":0,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"ppSeqNo\\":2}","{\\"instId\\":1,\\"digest\\":\\"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42\\",\\"viewNo\\":0,\\"op\\":\\"PREPARE\\",\\"ppTime\\":1507741256,\\"ppSeqNo\\":2,\\"stateRootHash\\":null,\\"txnRootHash\\":null}"]}' to Node3 2017-10-11 17:00:56,351 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node4 into one transmission 2017-10-11 17:00:56,351 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":2}', b'{"instId":1,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPARE","ppTime":1507741256,"ppSeqNo":2,"stateRootHash":null,"txnRootHash":null}']) 2017-10-11 17:00:56,351 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node4: b'{"op":"BATCH","signature":null,"messages":["{\\"viewNo\\":0,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"ppSeqNo\\":2}","{\\"instId\\":1,\\"digest\\":\\"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42\\",\\"viewNo\\":0,\\"op\\":\\"PREPARE\\",\\"ppTime\\":1507741256,\\"ppSeqNo\\":2,\\"stateRootHash\\":null,\\"txnRootHash\\":null}"]}' 2017-10-11 17:00:56,351 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"viewNo\\":0,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"ppSeqNo\\":2}","{\\"instId\\":1,\\"digest\\":\\"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42\\",\\"viewNo\\":0,\\"op\\":\\"PREPARE\\",\\"ppTime\\":1507741256,\\"ppSeqNo\\":2,\\"stateRootHash\\":null,\\"txnRootHash\\":null}"]}' to Node4 2017-10-11 17:00:56,352 | DEBUG | batched.py ( 89) | flushOutBoxes | Node1 batching 2 msgs to Node2 into one transmission 2017-10-11 17:00:56,352 | TRACE | batched.py ( 90) | flushOutBoxes | messages: deque([b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":2}', b'{"instId":1,"digest":"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42","viewNo":0,"op":"PREPARE","ppTime":1507741256,"ppSeqNo":2,"stateRootHash":null,"txnRootHash":null}']) 2017-10-11 17:00:56,352 | TRACE | batched.py ( 98) | flushOutBoxes | Node1 sending payload to Node2: b'{"op":"BATCH","signature":null,"messages":["{\\"viewNo\\":0,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"ppSeqNo\\":2}","{\\"instId\\":1,\\"digest\\":\\"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42\\",\\"viewNo\\":0,\\"op\\":\\"PREPARE\\",\\"ppTime\\":1507741256,\\"ppSeqNo\\":2,\\"stateRootHash\\":null,\\"txnRootHash\\":null}"]}' 2017-10-11 17:00:56,352 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"BATCH","signature":null,"messages":["{\\"viewNo\\":0,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"ppSeqNo\\":2}","{\\"instId\\":1,\\"digest\\":\\"9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42\\",\\"viewNo\\":0,\\"op\\":\\"PREPARE\\",\\"ppTime\\":1507741256,\\"ppSeqNo\\":2,\\"stateRootHash\\":null,\\"txnRootHash\\":null}"]}' to Node2 2017-10-11 17:00:56,355 | DEBUG | node.py (1881) | processOrdered | Node1 executing Ordered batch 0 2 of 1 requests 2017-10-11 17:00:56,360 | DEBUG | ledger.py ( 68) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:00:56,361 | DEBUG | node.py (2374) | sendReplyToClient | Node1 sending reply for ('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186) to client 2017-10-11 17:00:56,361 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'{"op":"REPLY","result":{"auditPath":["6BTmbK7RWRUL8AmtzKmreZKwvp2VDHorHRND22Jqr6pW","HYRnUmRJ2hQMtr1smmXw3sqWyQyKTzCeTh5D9EweiD8z"],"type":"1","dest":"542MVr22zcHbVyGzaXmbT2","reqId":1507741256159186,"signature":"2ACvk8ddWAa5daTzD8prMaa4Eig62fv5WcDpth9QqCZe6PvR9UMnXXU9CTcWQyq9P3HcDV4naXfg7Syy9h22Q3R3","txnTime":1507741256,"identifier":"V4SGRU86Z58d6TV7PBUe6f","rootHash":"CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw","seqNo":11}}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:00:56,362 | DEBUG | node.py (2295) | executeBatch | Node1 storing 3PC key (0, 2) for ledger 1 range (11, 11) 2017-10-11 17:00:56,363 | DEBUG | node.py (2593) | send | Node1 sending message COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:00:56,363 | TRACE | message_processor.py ( 28) | discard | Node1:0 discarding message PREPARE{'ppTime': 1507741256, 'ppSeqNo': 2, 'digest': '9f801966da1529fe8b7a50b3fb23ec25a7e1e927dbb727a6d9580224f7bfad42', 'viewNo': 0, 'stateRootHash': '8F8st3cEZDFKTucjctdXwyAddoLTXatnRtpS63YHxZYX', 'txnRootHash': 'CmvKha4N44FPRqPyp2VfEjjDq6FCSjfXPwBusRQz7sXw', 'instId': 0} because already ordered 3 phase message 2017-10-11 17:00:56,364 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 4 messages through listener 2017-10-11 17:00:56,365 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,365 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 1, 'ppSeqNo': 2}, 'Node4') 2017-10-11 17:00:56,365 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,366 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} 2017-10-11 17:00:56,366 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 0, 'ppSeqNo': 2}, 'Node2') 2017-10-11 17:00:56,366 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} 2017-10-11 17:00:56,366 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,366 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'ppSeqNo': 2, 'instId': 1}, 'Node3') 2017-10-11 17:00:56,366 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,367 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,367 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 1, 'ppSeqNo': 2}, 'Node2') 2017-10-11 17:00:56,367 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,369 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":2}' to Node3 2017-10-11 17:00:56,370 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":2}' to Node3 2017-10-11 17:00:56,370 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":2}' to Node4 2017-10-11 17:00:56,371 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":2}' to Node4 2017-10-11 17:00:56,371 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":2}' to Node2 2017-10-11 17:00:56,371 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":2}' to Node2 2017-10-11 17:00:56,372 | TRACE | message_processor.py ( 28) | discard | Node1:0 discarding message COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 2} because already ordered 3 phase message 2017-10-11 17:00:56,372 | DEBUG | replica.py ( 863) | processCommit | Node1:1 received COMMIT(0, 2) from Node4:1 2017-10-11 17:00:56,373 | DEBUG | replica.py ( 898) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} commits where f is 1 2017-10-11 17:00:56,373 | DEBUG | replica.py ( 874) | processCommit | Node1:1 processed incoming COMMIT(0, 2) 2017-10-11 17:00:56,373 | DEBUG | replica.py ( 863) | processCommit | Node1:1 received COMMIT(0, 2) from Node3:1 2017-10-11 17:00:56,373 | TRACE | replica.py ( 894) | tryOrder | Node1:1 returning request to node 2017-10-11 17:00:56,373 | INFO | replica.py (1395) | doOrder | Node1:1 ordering COMMIT(0, 2) 2017-10-11 17:00:56,373 | DEBUG | replica.py ( 310) | last_ordered_3pc | Node1:1 set last ordered as (0, 2) 2017-10-11 17:00:56,373 | INFO | replica.py (2064) | send | Node1:1 sending Ordered 2017-10-11 17:00:56,374 | TRACE | replica.py (2065) | send | Node1:1 sending ORDERED{'ppTime': 1507741256, 'txnRootHash': None, 'instId': 1, 'viewNo': 0, 'stateRootHash': None, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741256159186)], 'ledgerId': 1, 'ppSeqNo': 2} 2017-10-11 17:00:56,374 | DEBUG | replica.py (1426) | order_3pc_key | Node1:1 ordered request (0, 2) 2017-10-11 17:00:56,374 | DEBUG | replica.py ( 874) | processCommit | Node1:1 processed incoming COMMIT(0, 2) 2017-10-11 17:00:56,374 | TRACE | message_processor.py ( 28) | discard | Node1:1 discarding message COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 2} because already ordered 3 phase message 2017-10-11 17:00:56,378 | TRACE | node.py (1892) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:00:56,379 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:01:01,556 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 12 2017-10-11 17:01:01,556 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:01:01,556 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:01:01,556 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:01:01,557 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:01:01,557 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:01:01,557 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:01:01,557 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 13 to run in 10 seconds 2017-10-11 17:01:06,501 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:01:06,502 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:01:06,502 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:01:06,503 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:01:11,557 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 13 2017-10-11 17:01:11,558 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:01:11,558 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:01:11,558 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:01:11,558 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:01:11,558 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:01:11,559 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:01:11,559 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 14 to run in 10 seconds 2017-10-11 17:01:21,437 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkNodeRequestSpike with id 8 2017-10-11 17:01:21,437 | DEBUG | node.py (2026) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:01:21,438 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:01:21,438 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 15 to run in 60 seconds 2017-10-11 17:01:21,438 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 2 2017-10-11 17:01:21,438 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:01:21,438 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 3 to run in 60 seconds 2017-10-11 17:01:21,514 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:01:21,515 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:01:21,515 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:01:21,515 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:01:21,560 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 14 2017-10-11 17:01:21,560 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:01:21,560 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:01:21,561 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:01:21,561 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:01:21,561 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:01:21,561 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:01:21,561 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 16 to run in 10 seconds 2017-10-11 17:01:31,569 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 16 2017-10-11 17:01:31,573 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:01:31,573 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:01:31,573 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:01:31,574 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:01:31,574 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:01:31,574 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:01:31,574 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 17 to run in 10 seconds 2017-10-11 17:01:36,528 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:01:36,528 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:01:36,528 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:01:36,529 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:01:41,580 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 17 2017-10-11 17:01:41,581 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:01:41,581 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:01:41,581 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:01:41,582 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:01:41,582 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:01:41,583 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:01:41,583 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 18 to run in 10 seconds 2017-10-11 17:01:51,534 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:01:51,535 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:01:51,535 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:01:51,535 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:01:51,593 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 18 2017-10-11 17:01:51,593 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:01:51,593 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:01:51,593 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:01:51,594 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:01:51,594 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:01:51,594 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:01:51,594 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 19 to run in 10 seconds 2017-10-11 17:02:01,606 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 19 2017-10-11 17:02:01,606 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:02:01,606 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:02:01,606 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:02:01,607 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:02:01,607 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:02:01,607 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:02:01,607 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 20 to run in 10 seconds 2017-10-11 17:02:06,534 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:02:06,540 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:02:06,540 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:02:06,541 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:02:11,618 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 20 2017-10-11 17:02:11,619 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:02:11,619 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:02:11,619 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:02:11,620 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:02:11,620 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:02:11,620 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:02:11,620 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 21 to run in 10 seconds 2017-10-11 17:02:21,444 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkNodeRequestSpike with id 15 2017-10-11 17:02:21,445 | DEBUG | node.py (2026) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:02:21,445 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:02:21,445 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 22 to run in 60 seconds 2017-10-11 17:02:21,445 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 3 2017-10-11 17:02:21,445 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:02:21,445 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 4 to run in 60 seconds 2017-10-11 17:02:21,537 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:02:21,537 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:02:21,537 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:02:21,537 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:02:21,635 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 21 2017-10-11 17:02:21,635 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:02:21,636 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:02:21,636 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:02:21,636 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:02:21,636 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:02:21,636 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:02:21,636 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 23 to run in 10 seconds 2017-10-11 17:02:31,637 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 23 2017-10-11 17:02:31,637 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:02:31,637 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:02:31,638 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:02:31,638 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:02:31,638 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:02:31,638 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:02:31,639 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 24 to run in 10 seconds 2017-10-11 17:02:36,543 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:02:36,544 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:02:36,544 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:02:36,544 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:02:41,648 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 24 2017-10-11 17:02:41,648 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:02:41,649 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:02:41,649 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:02:41,649 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:02:41,649 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:02:41,649 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:02:41,649 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 25 to run in 10 seconds 2017-10-11 17:02:51,546 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:02:51,546 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:02:51,547 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:02:51,547 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:02:51,657 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 25 2017-10-11 17:02:51,658 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:02:51,658 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:02:51,658 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:02:51,658 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:02:51,659 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:02:51,659 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:02:51,659 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 26 to run in 10 seconds 2017-10-11 17:03:01,671 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 26 2017-10-11 17:03:01,671 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:03:01,671 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:03:01,672 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:03:01,672 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:03:01,672 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:03:01,673 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:03:01,673 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 27 to run in 10 seconds 2017-10-11 17:03:06,566 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:03:06,566 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:03:06,566 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:03:06,567 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:03:11,679 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 27 2017-10-11 17:03:11,679 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:03:11,679 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:03:11,680 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:03:11,680 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:03:11,680 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:03:11,681 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:03:11,681 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 28 to run in 10 seconds 2017-10-11 17:03:21,453 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkNodeRequestSpike with id 22 2017-10-11 17:03:21,454 | DEBUG | node.py (2026) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:03:21,454 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:03:21,454 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 29 to run in 60 seconds 2017-10-11 17:03:21,455 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 4 2017-10-11 17:03:21,456 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:03:21,456 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 5 to run in 60 seconds 2017-10-11 17:03:21,574 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:03:21,575 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:03:21,575 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:03:21,575 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:03:21,691 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 28 2017-10-11 17:03:21,692 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:03:21,692 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:03:21,692 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:03:21,693 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:03:21,693 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:03:21,694 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:03:21,695 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 30 to run in 10 seconds 2017-10-11 17:03:31,707 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 30 2017-10-11 17:03:31,708 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:03:31,708 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:03:31,708 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0847652940951469 is acceptable. 2017-10-11 17:03:31,709 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:03:31,709 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:03:31,709 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:03:31,710 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 31 to run in 10 seconds 2017-10-11 17:03:33,792 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:03:33,793 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,794 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741413733229 2017-10-11 17:03:33,794 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'request': {'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,799 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'PROPAGATE', 'request': {'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node3') 2017-10-11 17:03:33,800 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,800 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,801 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741413733229 2017-10-11 17:03:33,801 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'request': {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,801 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'PROPAGATE', 'request': {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node4') 2017-10-11 17:03:33,801 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,802 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,803 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,803 | TRACE | propagator.py ( 159) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:03:33,803 | INFO | propagator.py ( 145) | propagate | Node1 propagating V4SGRU86Z58d6TV7PBUe6f request 1507741413733229 from client fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S% 2017-10-11 17:03:33,804 | DEBUG | node.py (2593) | send | Node1 sending message PROPAGATE{'request': {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'name': 'upgrade37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:03:33,804 | DEBUG | propagator.py ( 208) | forward | Node1 forwarding request ('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229) to 2 replicas 2017-10-11 17:03:33,804 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,805 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,805 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:03:33,805 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'name': 'upgrade37', 'action': 'start', 'version': '1.1.37', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since already forwarded 2017-10-11 17:03:33,806 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:03:33,835 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,837 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,851 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on request 1507741413733229 2017-10-11 17:03:33,851 | TRACE | node.py (1444) | validateClientMsg | Node1C received CLIENT message: SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:03:33,852 | DISPLAY | node.py (1490) | processClientInBox | Node1C processing b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' request SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:03:33,853 | INFO | upgrader.py ( 308) | _scheduleUpgrade | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv's upgrader processing upgrade for version 1.1.37 2017-10-11 17:03:33,854 | INFO | upgrader.py ( 369) | _callUpgradeAgent | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv's upgrader calling agent for upgrade 2017-10-11 17:03:33,855 | DEBUG | node.py (1751) | processRequest | Node1 received client request: SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:03:33,855 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:03:33,855 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'type': '109', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since already forwarded 2017-10-11 17:03:33,856 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'{"op":"REQACK","reqId":1507741413733229,"identifier":"V4SGRU86Z58d6TV7PBUe6f"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:03:33,868 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz","reqId":1507741413733229,"operation":{"schedule":{"DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T14:25:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T14:30:00.258870+00:00","Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T14:15:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T14:20:00.258870+00:00"},"type":"109","name":"upgrade37","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","action":"start","version":"1.1.37","force":true,"timeout":10,"justification":null},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node3 2017-10-11 17:03:33,871 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz","reqId":1507741413733229,"operation":{"schedule":{"DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T14:25:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T14:30:00.258870+00:00","Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T14:15:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T14:20:00.258870+00:00"},"type":"109","name":"upgrade37","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","action":"start","version":"1.1.37","force":true,"timeout":10,"justification":null},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node3 2017-10-11 17:03:33,872 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz","reqId":1507741413733229,"operation":{"schedule":{"DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T14:25:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T14:30:00.258870+00:00","Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T14:15:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T14:20:00.258870+00:00"},"type":"109","name":"upgrade37","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","action":"start","version":"1.1.37","force":true,"timeout":10,"justification":null},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node4 2017-10-11 17:03:33,872 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz","reqId":1507741413733229,"operation":{"schedule":{"DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T14:25:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T14:30:00.258870+00:00","Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T14:15:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T14:20:00.258870+00:00"},"type":"109","name":"upgrade37","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","action":"start","version":"1.1.37","force":true,"timeout":10,"justification":null},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node4 2017-10-11 17:03:33,872 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"op":"PROPAGATE","request":{"signature":"4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz","reqId":1507741413733229,"operation":{"schedule":{"DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T14:25:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T14:30:00.258870+00:00","Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T14:15:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T14:20:00.258870+00:00"},"type":"109","name":"upgrade37","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","action":"start","version":"1.1.37","force":true,"timeout":10,"justification":null},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node2 2017-10-11 17:03:33,873 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"op":"PROPAGATE","request":{"signature":"4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz","reqId":1507741413733229,"operation":{"schedule":{"DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T14:25:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T14:30:00.258870+00:00","Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T14:15:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T14:20:00.258870+00:00"},"type":"109","name":"upgrade37","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","action":"start","version":"1.1.37","force":true,"timeout":10,"justification":null},"identifier":"V4SGRU86Z58d6TV7PBUe6f"},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%"}' to Node2 2017-10-11 17:03:33,874 | INFO | replica.py ( 642) | create3PCBatch | Node1:0 creating batch 3 for ledger 2 with state root b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f' 2017-10-11 17:03:33,875 | DISPLAY | replica.py ( 673) | create3PCBatch | Node1:0 created a PRE-PREPARE with 1 requests for ledger 2 2017-10-11 17:03:33,875 | INFO | replica.py (2064) | send | Node1:0 sending PrePrepare 2017-10-11 17:03:33,876 | TRACE | replica.py (2065) | send | Node1:0 sending PREPREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'ledgerId': 2, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229)]} 2017-10-11 17:03:33,876 | DEBUG | replica.py ( 594) | trackBatches | Node1:0 tracking batch for PREPREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'ledgerId': 2, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229)]} with state root b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f' 2017-10-11 17:03:33,877 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:03:33,878 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,879 | INFO | node.py (2250) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741413733229 2017-10-11 17:03:33,879 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'request': {'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,879 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'request': {'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'op': 'PROPAGATE', 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'}, 'Node2') 2017-10-11 17:03:33,879 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,880 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'stateRootHash': None, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 2, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741413733229]]} 2017-10-11 17:03:33,880 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741413, 'op': 'PREPREPARE', 'txnRootHash': None, 'stateRootHash': None, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741413733229]], 'ledgerId': 2, 'ppSeqNo': 3}, 'Node2') 2017-10-11 17:03:33,880 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'stateRootHash': None, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 2, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741413733229]]} 2017-10-11 17:03:33,880 | DEBUG | node.py (1799) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'reqId': 1507741413733229, 'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:03:33,881 | TRACE | signing.py ( 98) | serializeMsg | serialized msg {'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} into identifier:V4SGRU86Z58d6TV7PBUe6f|operation:action:start|force:True|justification:|name:upgrade37|schedule:4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA:2017-10-11T14:30:00.258870+00:00|8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb:2017-10-11T14:20:00.258870+00:00|DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya:2017-10-11T14:25:00.258870+00:00|Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv:2017-10-11T14:15:00.258870+00:00|sha256:ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f|timeout:10|type:109|version:1.1.37|reqId:1507741413733229 2017-10-11 17:03:33,881 | TRACE | propagator.py ( 138) | propagate | Node1 already propagated SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} 2017-10-11 17:03:33,881 | DEBUG | propagator.py ( 241) | tryForwarding | Node1 not forwarding request SafeRequest: {'signature': '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz', 'reqId': 1507741413733229, 'operation': {'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T14:30:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T14:25:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T14:15:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T14:20:00.258870+00:00'}, 'type': '109', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'action': 'start', 'version': '1.1.37', 'name': 'upgrade37', 'force': True, 'timeout': 10, 'justification': None}, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f'} to its replicas since already forwarded 2017-10-11 17:03:33,884 | DEBUG | node.py (2593) | send | Node1 sending message PREPREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 0, 'viewNo': 0, 'discarded': 1, 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'ledgerId': 2, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229)]} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:03:33,884 | DEBUG | replica.py ( 790) | processPrePrepare | Node1:1 received PRE-PREPARE(0, 3) from Node2:1 at 2270499.341836818 2017-10-11 17:03:33,884 | DEBUG | replica.py ( 903) | doPrepare | Node1:1 Sending PREPARE(0, 3) at 2270499.342262592 2017-10-11 17:03:33,885 | INFO | replica.py (2064) | send | Node1:1 sending Prepare 2017-10-11 17:03:33,885 | TRACE | replica.py (2065) | send | Node1:1 sending PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:03:33,885 | DEBUG | replica.py ( 886) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:03:33,885 | DEBUG | replica.py ( 594) | trackBatches | Node1:1 tracking batch for PREPREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'stateRootHash': None, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 1, 'viewNo': 0, 'discarded': 1, 'txnRootHash': None, 'ledgerId': 2, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229)]} with state root None 2017-10-11 17:03:33,885 | DEBUG | replica.py ( 809) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(0, 3) 2017-10-11 17:03:33,889 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPREPARE","ledgerId":2,"ppTime":1507741413,"txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppSeqNo":3,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741413733229]],"discarded":1,"instId":0}' to Node3 2017-10-11 17:03:33,890 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPREPARE","ledgerId":2,"ppTime":1507741413,"txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppSeqNo":3,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741413733229]],"discarded":1,"instId":0}' to Node3 2017-10-11 17:03:33,890 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPREPARE","ledgerId":2,"ppTime":1507741413,"txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppSeqNo":3,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741413733229]],"discarded":1,"instId":0}' to Node4 2017-10-11 17:03:33,892 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPREPARE","ledgerId":2,"ppTime":1507741413,"txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppSeqNo":3,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741413733229]],"discarded":1,"instId":0}' to Node4 2017-10-11 17:03:33,892 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPREPARE","ledgerId":2,"ppTime":1507741413,"txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppSeqNo":3,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741413733229]],"discarded":1,"instId":0}' to Node2 2017-10-11 17:03:33,892 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPREPARE","ledgerId":2,"ppTime":1507741413,"txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppSeqNo":3,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741413733229]],"discarded":1,"instId":0}' to Node2 2017-10-11 17:03:33,893 | DEBUG | node.py (2593) | send | Node1 sending message PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:03:33,896 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"instId":1,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPARE","ppTime":1507741413,"ppSeqNo":3,"stateRootHash":null,"txnRootHash":null}' to Node3 2017-10-11 17:03:33,896 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"instId":1,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPARE","ppTime":1507741413,"ppSeqNo":3,"stateRootHash":null,"txnRootHash":null}' to Node3 2017-10-11 17:03:33,897 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"instId":1,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPARE","ppTime":1507741413,"ppSeqNo":3,"stateRootHash":null,"txnRootHash":null}' to Node4 2017-10-11 17:03:33,897 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"instId":1,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPARE","ppTime":1507741413,"ppSeqNo":3,"stateRootHash":null,"txnRootHash":null}' to Node4 2017-10-11 17:03:33,897 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"instId":1,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPARE","ppTime":1507741413,"ppSeqNo":3,"stateRootHash":null,"txnRootHash":null}' to Node2 2017-10-11 17:03:33,897 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"instId":1,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","viewNo":0,"op":"PREPARE","ppTime":1507741413,"ppSeqNo":3,"stateRootHash":null,"txnRootHash":null}' to Node2 2017-10-11 17:03:33,901 | WARNING | base_events.py (1308) | _run_once | Executing wait_for= cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.138 seconds 2017-10-11 17:03:33,912 | DEBUG | base_events.py ( 681) | create_connection | connect to ('127.0.0.1', 30003) 2017-10-11 17:03:33,917 | DEBUG | base_events.py (1270) | _run_once | poll 9.821 ms took 0.041 ms: 1 events 2017-10-11 17:03:33,921 | DEBUG | base_events.py ( 719) | create_connection | connected to 127.0.0.1:'30003': (<_SelectorSocketTransport fd=75 read=polling write=>, ) 2017-10-11 17:03:33,922 | TRACE | has_action_queue.py ( 34) | _schedule | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv scheduling action partial(_declareTimeoutExceeded) with id 1 to run in 600 seconds 2017-10-11 17:03:33,935 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:03:33,936 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:33,936 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741413, 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 0, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'ppSeqNo': 3}, 'Node2') 2017-10-11 17:03:33,936 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:33,939 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 3) from Node2:0 2017-10-11 17:03:33,942 | DEBUG | replica.py ( 886) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:33,948 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 3) 2017-10-11 17:03:33,949 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:03:33,949 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:03:33,950 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 1, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': None, 'txnRootHash': None}, 'Node4') 2017-10-11 17:03:33,950 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:03:33,960 | DEBUG | replica.py ( 834) | processPrepare | Node1:1 received PREPARE(0, 3) from Node4:1 2017-10-11 17:03:33,960 | DEBUG | replica.py ( 922) | doCommit | Node1:1 Sending COMMIT(0, 3) at 2270499.41805486 2017-10-11 17:03:33,961 | INFO | replica.py (2064) | send | Node1:1 sending Commit 2017-10-11 17:03:33,961 | TRACE | replica.py (2065) | send | Node1:1 sending COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} 2017-10-11 17:03:33,961 | DEBUG | replica.py ( 898) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} commits where f is 1 2017-10-11 17:03:33,962 | DEBUG | replica.py ( 845) | processPrepare | Node1:1 processed incoming PREPARE (0, 3) 2017-10-11 17:03:33,971 | DEBUG | node.py (2593) | send | Node1 sending message COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:03:33,974 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":3}' to Node3 2017-10-11 17:03:33,974 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":3}' to Node3 2017-10-11 17:03:33,978 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":3}' to Node4 2017-10-11 17:03:33,983 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":3}' to Node4 2017-10-11 17:03:33,984 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":3}' to Node2 2017-10-11 17:03:33,985 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":1,"ppSeqNo":3}' to Node2 2017-10-11 17:03:34,019 | TRACE | zstack.py ( 472) | _receiveFromListener | Node1 got 7 messages through listener 2017-10-11 17:03:34,021 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:03:34,027 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741413, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 3, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': None, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff'}, 'Node3') 2017-10-11 17:03:34,036 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:03:34,037 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} 2017-10-11 17:03:34,037 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 1, 'ppSeqNo': 3}, 'Node2') 2017-10-11 17:03:34,037 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} 2017-10-11 17:03:34,038 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppTime":1507741413,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","ppSeqNo":3}', '{"instId":1,"op":"COMMIT","ppSeqNo":3,"viewNo":0}']} 2017-10-11 17:03:34,038 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppTime":1507741413,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","ppSeqNo":3}', '{"instId":1,"op":"COMMIT","ppSeqNo":3,"viewNo":0}']}, 'Node4') 2017-10-11 17:03:34,038 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","ppTime":1507741413,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","ppSeqNo":3}', '{"instId":1,"op":"COMMIT","ppSeqNo":3,"viewNo":0}']} 2017-10-11 17:03:34,038 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:34,040 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'instId': 0, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami'}, 'Node4') 2017-10-11 17:03:34,041 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:34,041 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} 2017-10-11 17:03:34,042 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 1, 'ppSeqNo': 3}, 'Node4') 2017-10-11 17:03:34,043 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} 2017-10-11 17:03:34,049 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"ppSeqNo":3,"op":"PREPARE","ppTime":1507741413,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","instId":0}', '{"op":"COMMIT","ppSeqNo":3,"viewNo":0,"instId":1}']} 2017-10-11 17:03:34,050 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"ppSeqNo":3,"op":"PREPARE","ppTime":1507741413,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","instId":0}', '{"op":"COMMIT","ppSeqNo":3,"viewNo":0,"instId":1}']}, 'Node3') 2017-10-11 17:03:34,051 | DEBUG | node.py (1328) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"ppSeqNo":3,"op":"PREPARE","ppTime":1507741413,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"digest":"6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff","txnRootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","instId":0}', '{"op":"COMMIT","ppSeqNo":3,"viewNo":0,"instId":1}']} 2017-10-11 17:03:34,051 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:34,052 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'ppTime': 1507741413, 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0, 'ppSeqNo': 3, 'viewNo': 0, 'op': 'PREPARE', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff'}, 'Node3') 2017-10-11 17:03:34,052 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:34,054 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} 2017-10-11 17:03:34,054 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'ppSeqNo': 3, 'instId': 1}, 'Node3') 2017-10-11 17:03:34,054 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} 2017-10-11 17:03:34,054 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} 2017-10-11 17:03:34,054 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 0, 'ppSeqNo': 3}, 'Node2') 2017-10-11 17:03:34,054 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} 2017-10-11 17:03:34,055 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} 2017-10-11 17:03:34,055 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'instId': 0, 'ppSeqNo': 3}, 'Node4') 2017-10-11 17:03:34,055 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} 2017-10-11 17:03:34,056 | DEBUG | node.py (1313) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} 2017-10-11 17:03:34,056 | INFO | node.py (1275) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'op': 'COMMIT', 'ppSeqNo': 3, 'instId': 0}, 'Node3') 2017-10-11 17:03:34,056 | DEBUG | node.py (1342) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} 2017-10-11 17:03:34,072 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 3) from Node4:0 2017-10-11 17:03:34,072 | DEBUG | replica.py ( 922) | doCommit | Node1:0 Sending COMMIT(0, 3) at 2270499.52987742 2017-10-11 17:03:34,072 | INFO | replica.py (2064) | send | Node1:0 sending Commit 2017-10-11 17:03:34,072 | TRACE | replica.py (2065) | send | Node1:0 sending COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} 2017-10-11 17:03:34,073 | DEBUG | replica.py ( 898) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} commits where f is 1 2017-10-11 17:03:34,073 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 3) 2017-10-11 17:03:34,073 | DEBUG | replica.py ( 834) | processPrepare | Node1:0 received PREPARE(0, 3) from Node3:0 2017-10-11 17:03:34,073 | DEBUG | replica.py ( 886) | tryCommit | Node1:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0} 2017-10-11 17:03:34,073 | DEBUG | replica.py ( 845) | processPrepare | Node1:0 processed incoming PREPARE (0, 3) 2017-10-11 17:03:34,074 | DEBUG | replica.py ( 863) | processCommit | Node1:0 received COMMIT(0, 3) from Node2:0 2017-10-11 17:03:34,075 | DEBUG | replica.py ( 898) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} commits where f is 1 2017-10-11 17:03:34,075 | DEBUG | replica.py ( 874) | processCommit | Node1:0 processed incoming COMMIT(0, 3) 2017-10-11 17:03:34,075 | DEBUG | replica.py ( 863) | processCommit | Node1:0 received COMMIT(0, 3) from Node4:0 2017-10-11 17:03:34,075 | TRACE | replica.py ( 894) | tryOrder | Node1:0 returning request to node 2017-10-11 17:03:34,075 | INFO | replica.py (1395) | doOrder | Node1:0 ordering COMMIT(0, 3) 2017-10-11 17:03:34,075 | DEBUG | replica.py ( 310) | last_ordered_3pc | Node1:0 set last ordered as (0, 3) 2017-10-11 17:03:34,076 | INFO | replica.py (2064) | send | Node1:0 sending Ordered 2017-10-11 17:03:34,076 | TRACE | replica.py (2065) | send | Node1:0 sending ORDERED{'ppTime': 1507741413, 'txnRootHash': '4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami', 'instId': 0, 'viewNo': 0, 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229)], 'ledgerId': 2, 'ppSeqNo': 3} 2017-10-11 17:03:34,076 | DEBUG | replica.py (1426) | order_3pc_key | Node1:0 ordered request (0, 3) 2017-10-11 17:03:34,076 | DEBUG | replica.py ( 874) | processCommit | Node1:0 processed incoming COMMIT(0, 3) 2017-10-11 17:03:34,076 | TRACE | message_processor.py ( 28) | discard | Node1:0 discarding message COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} because already ordered 3 phase message 2017-10-11 17:03:34,077 | DEBUG | replica.py ( 834) | processPrepare | Node1:1 received PREPARE(0, 3) from Node3:1 2017-10-11 17:03:34,077 | DEBUG | replica.py ( 886) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppTime': 1507741413, 'ppSeqNo': 3, 'digest': '6d95b08f4d95a879e27b0cf519926b989fad4dbf0b42a44d2a496925fa45faff', 'viewNo': 0, 'stateRootHash': None, 'txnRootHash': None, 'instId': 1} 2017-10-11 17:03:34,077 | DEBUG | replica.py ( 845) | processPrepare | Node1:1 processed incoming PREPARE (0, 3) 2017-10-11 17:03:34,077 | DEBUG | replica.py ( 863) | processCommit | Node1:1 received COMMIT(0, 3) from Node2:1 2017-10-11 17:03:34,077 | DEBUG | replica.py ( 898) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} commits where f is 1 2017-10-11 17:03:34,077 | DEBUG | replica.py ( 874) | processCommit | Node1:1 processed incoming COMMIT(0, 3) 2017-10-11 17:03:34,078 | DEBUG | replica.py ( 863) | processCommit | Node1:1 received COMMIT(0, 3) from Node4:1 2017-10-11 17:03:34,078 | TRACE | replica.py ( 894) | tryOrder | Node1:1 returning request to node 2017-10-11 17:03:34,078 | INFO | replica.py (1395) | doOrder | Node1:1 ordering COMMIT(0, 3) 2017-10-11 17:03:34,078 | DEBUG | replica.py ( 310) | last_ordered_3pc | Node1:1 set last ordered as (0, 3) 2017-10-11 17:03:34,078 | INFO | replica.py (2064) | send | Node1:1 sending Ordered 2017-10-11 17:03:34,079 | TRACE | replica.py (2065) | send | Node1:1 sending ORDERED{'ppTime': 1507741413, 'txnRootHash': None, 'instId': 1, 'viewNo': 0, 'stateRootHash': None, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229)], 'ledgerId': 2, 'ppSeqNo': 3} 2017-10-11 17:03:34,079 | DEBUG | replica.py (1426) | order_3pc_key | Node1:1 ordered request (0, 3) 2017-10-11 17:03:34,079 | DEBUG | replica.py ( 874) | processCommit | Node1:1 processed incoming COMMIT(0, 3) 2017-10-11 17:03:34,079 | TRACE | message_processor.py ( 28) | discard | Node1:1 discarding message COMMIT{'viewNo': 0, 'instId': 1, 'ppSeqNo': 3} because already ordered 3 phase message 2017-10-11 17:03:34,086 | DEBUG | node.py (2593) | send | Node1 sending message COMMIT{'viewNo': 0, 'instId': 0, 'ppSeqNo': 3} to all recipients: ['Node3', 'Node4', 'Node2'] 2017-10-11 17:03:34,086 | DEBUG | node.py (1881) | processOrdered | Node1 executing Ordered batch 0 3 of 1 requests 2017-10-11 17:03:34,090 | DEBUG | ledger.py ( 68) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:03:34,091 | INFO | upgrader.py ( 308) | _scheduleUpgrade | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv's upgrader processing upgrade for version 1.1.37 2017-10-11 17:03:34,091 | INFO | upgrader.py ( 369) | _callUpgradeAgent | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv's upgrader calling agent for upgrade 2017-10-11 17:03:34,092 | DEBUG | node.py (2374) | sendReplyToClient | Node1 sending reply for ('V4SGRU86Z58d6TV7PBUe6f', 1507741413733229) to client 2017-10-11 17:03:34,093 | TRACE | zstack.py ( 726) | transmitThroughListener | Node1C transmitting b'{"op":"REPLY","result":{"schedule":{"4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T14:30:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T14:25:00.258870+00:00","Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T14:15:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T14:20:00.258870+00:00"},"reqId":1507741413733229,"action":"start","version":"1.1.37","name":"upgrade37","signature":"4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz","rootHash":"4KLT8Jvfc9dEy91Tk8nsiWbFrvmwhBNu6avtVG3jcami","identifier":"V4SGRU86Z58d6TV7PBUe6f","auditPath":[],"type":"109","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","txnTime":1507741413,"force":true,"seqNo":1,"timeout":10,"justification":null}}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:03:34,093 | DEBUG | node.py (2295) | executeBatch | Node1 storing 3PC key (0, 3) for ledger 2 range (1, 1) 2017-10-11 17:03:34,094 | TRACE | node.py (1892) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:03:34,094 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:03:34,097 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":3}' to Node3 2017-10-11 17:03:34,097 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":3}' to Node3 2017-10-11 17:03:34,098 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":3}' to Node4 2017-10-11 17:03:34,098 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":3}' to Node4 2017-10-11 17:03:34,099 | DEBUG | zstack.py ( 699) | transmit | Node1 transmitting message b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":3}' to Node2 2017-10-11 17:03:34,099 | TRACE | batched.py ( 85) | flushOutBoxes | Node1 sending msg b'{"viewNo":0,"op":"COMMIT","instId":0,"ppSeqNo":3}' to Node2 2017-10-11 17:03:34,114 | WARNING | base_events.py (1308) | _run_once | Executing wait_for= cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.180 seconds 2017-10-11 17:03:34,115 | DEBUG | base_events.py ( 681) | create_connection | connect to ('127.0.0.1', 30003) 2017-10-11 17:03:34,115 | DEBUG | base_events.py (1270) | _run_once | poll 7.784 ms took 0.012 ms: 1 events 2017-10-11 17:03:34,117 | DEBUG | base_events.py ( 719) | create_connection | connected to 127.0.0.1:'30003': (<_SelectorSocketTransport fd=75 read=polling write=>, ) 2017-10-11 17:03:34,121 | TRACE | has_action_queue.py ( 34) | _schedule | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv scheduling action partial(_declareTimeoutExceeded) with id 2 to run in 600 seconds 2017-10-11 17:03:36,576 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:03:36,576 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:03:36,576 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:03:36,577 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:03:41,726 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 31 2017-10-11 17:03:41,726 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:03:41,733 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:03:41,733 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0403626389033194 is acceptable. 2017-10-11 17:03:41,733 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:03:41,734 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:03:41,734 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:03:41,735 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 32 to run in 10 seconds 2017-10-11 17:03:51,599 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:03:51,599 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:03:51,599 | DEBUG | kit_zstack.py ( 65) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:03:51,600 | DEBUG | kit_zstack.py ( 47) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:03:51,736 | TRACE | has_action_queue.py ( 64) | _serviceActions | Node1 running action checkPerformance with id 32 2017-10-11 17:03:51,737 | TRACE | node.py (2005) | checkPerformance | Node1 checking its performance 2017-10-11 17:03:51,737 | DEBUG | notifier_plugin_manager.py ( 65) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:03:51,737 | TRACE | monitor.py ( 299) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0403626389033194 is acceptable. 2017-10-11 17:03:51,737 | TRACE | monitor.py ( 315) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:03:51,737 | TRACE | monitor.py ( 345) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:03:51,737 | DEBUG | node.py (2022) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:03:51,737 | TRACE | has_action_queue.py ( 34) | _schedule | Node1 scheduling action checkPerformance with id 33 to run in 10 seconds 2017-10-11 17:03:56,837 | TRACE | remote.py ( 117) | hasLostConnection | Remote Node2:HA(host='10.0.0.3', port=9703) has monitor events: [512, 4] 2017-10-11 17:03:56,838 | DEBUG | remote.py ( 121) | hasLostConnection | Node2:HA(host='10.0.0.3', port=9703) found disconnected event on monitor 2017-10-11 17:03:56,839 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node3', 'Node4', 'Node2'} to {'Node3', 'Node4'} 2017-10-11 17:03:56,840 | INFO | keep_in_touch.py ( 92) | _connsChanged | Node1 disconnected from Node2 2017-10-11 17:03:56,840 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from started to started_hungry 2017-10-11 17:03:56,840 | DEBUG | node.py ( 918) | checkInstances | Node1 choosing to start election on the basis of count 3 and nodes {'Node3', 'Node4'} 2017-10-11 17:03:56,941 | TRACE | remote.py ( 117) | hasLostConnection | Remote Node3:HA(host='10.0.0.4', port=9705) has monitor events: [512, 4] 2017-10-11 17:03:56,942 | DEBUG | remote.py ( 121) | hasLostConnection | Node3:HA(host='10.0.0.4', port=9705) found disconnected event on monitor 2017-10-11 17:03:56,942 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node3', 'Node4'} to {'Node4'} 2017-10-11 17:03:56,943 | INFO | keep_in_touch.py ( 92) | _connsChanged | Node1 disconnected from Node3 2017-10-11 17:03:56,943 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from started_hungry to starting 2017-10-11 17:03:58,311 | TRACE | remote.py ( 117) | hasLostConnection | Remote Node4:HA(host='10.0.0.5', port=9707) has monitor events: [512, 4] 2017-10-11 17:03:58,311 | DEBUG | remote.py ( 121) | hasLostConnection | Node4:HA(host='10.0.0.5', port=9707) found disconnected event on monitor 2017-10-11 17:03:58,311 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node4'} to set() 2017-10-11 17:03:58,312 | INFO | keep_in_touch.py ( 92) | _connsChanged | Node1 disconnected from Node4 2017-10-11 17:04:06,208 | DEBUG | node_runner.py ( 18) | run_node | You can find logs in /home/sovrin/.sovrin/Node1.log 2017-10-11 17:04:06,209 | DEBUG | node_runner.py ( 21) | run_node | Sovrin related env vars: [] 2017-10-11 17:04:07,708 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: git 2017-10-11 17:04:07,731 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: hg 2017-10-11 17:04:07,778 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: svn 2017-10-11 17:04:07,779 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: bzr 2017-10-11 17:04:08,604 | DEBUG | selector_events.py ( 53) | __init__ | Using selector: EpollSelector 2017-10-11 17:04:08,605 | DEBUG | looper.py ( 125) | __init__ | Setting handler for SIGINT 2017-10-11 17:04:08,623 | DEBUG | ledger.py ( 200) | start | Starting ledger... 2017-10-11 17:04:08,627 | DEBUG | ledger.py ( 72) | recoverTree | Recovering tree from hash store of size 11 2017-10-11 17:04:08,628 | DEBUG | ledger.py ( 82) | recoverTree | Recovered tree in 0.0005327621474862099 seconds 2017-10-11 17:04:08,641 | DEBUG | idr_cache.py ( 25) | __init__ | Initializing identity cache Node1 2017-10-11 17:04:08,645 | INFO | node.py (2420) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 17:04:08,689 | DEBUG | ledger.py ( 200) | start | Starting ledger... 2017-10-11 17:04:08,693 | DEBUG | ledger.py ( 72) | recoverTree | Recovering tree from hash store of size 4 2017-10-11 17:04:08,694 | DEBUG | ledger.py ( 82) | recoverTree | Recovered tree in 0.00040306011214852333 seconds 2017-10-11 17:04:08,694 | INFO | node.py (2420) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 17:04:08,738 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:04:08,738 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: typing.Dict<~KT, ~VT>] because it does not have a 'pluginType' attribute 2017-10-11 17:04:08,738 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:04:08,738 | INFO | plugin_loader.py ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-10-11 17:04:08,738 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:04:08,738 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:04:08,739 | DEBUG | has_action_queue.py ( 79) | startRepeating | checkPerformance will be repeating every 60 seconds 2017-10-11 17:04:08,739 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 1 to run in 60 seconds 2017-10-11 17:04:08,739 | DEBUG | replica.py ( 313) | h | Node1:0 set watermarks as 0 300 2017-10-11 17:04:08,739 | DISPLAY | replicas.py ( 36) | grow | Node1 added replica Node1:0 to instance 0 (master) 2017-10-11 17:04:08,739 | DEBUG | replica.py ( 313) | h | Node1:1 set watermarks as 0 300 2017-10-11 17:04:08,739 | DISPLAY | replicas.py ( 36) | grow | Node1 added replica Node1:1 to instance 1 (backup) 2017-10-11 17:04:08,740 | DEBUG | has_action_queue.py ( 79) | startRepeating | checkPerformance will be repeating every 10 seconds 2017-10-11 17:04:08,740 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 1 to run in 10 seconds 2017-10-11 17:04:08,740 | DEBUG | has_action_queue.py ( 79) | startRepeating | checkNodeRequestSpike will be repeating every 60 seconds 2017-10-11 17:04:08,740 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 2 to run in 60 seconds 2017-10-11 17:04:08,740 | DEBUG | plugin_helper.py ( 24) | loadPlugins | Plugin loading started to load plugins from basedir: /home/sovrin/.sovrin 2017-10-11 17:04:08,740 | DEBUG | plugin_helper.py ( 68) | loadPlugins | Total plugins loaded from basedir /home/sovrin/.sovrin are : 0 2017-10-11 17:04:08,740 | DEBUG | node.py ( 340) | __init__ | total plugins loaded in node: 0 2017-10-11 17:04:08,759 | DEBUG | ledger.py ( 200) | start | Starting ledger... 2017-10-11 17:04:08,763 | DEBUG | ledger.py ( 72) | recoverTree | Recovering tree from hash store of size 1 2017-10-11 17:04:08,764 | DEBUG | ledger.py ( 82) | recoverTree | Recovered tree in 0.0003346716985106468 seconds 2017-10-11 17:04:08,771 | WARNING | upgrader.py ( 126) | check_upgrade_succeeded | Upgrade for node Node1 was not scheduled. Last event is scheduled:2017-10-11 14:15:00.258870+00:00:1.1.37:None 2017-10-11 17:04:08,771 | INFO | node.py (2420) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 17:04:08,771 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from stopped to starting 2017-10-11 17:04:08,772 | DEBUG | ledger.py ( 198) | start | Ledger already started. 2017-10-11 17:04:08,772 | DEBUG | ledger.py ( 198) | start | Ledger already started. 2017-10-11 17:04:08,772 | DEBUG | ledger.py ( 198) | start | Ledger already started. 2017-10-11 17:04:08,772 | DEBUG | zstack.py ( 319) | start | Node1 starting with restricted as True and reSetupAuth as True 2017-10-11 17:04:08,772 | DEBUG | authenticator.py ( 31) | start | Starting ZAP at inproc://zeromq.zap.1 2017-10-11 17:04:08,772 | DEBUG | base.py ( 72) | allow | Allowing 0.0.0.0 2017-10-11 17:04:08,772 | DEBUG | base.py ( 112) | configure_curve | Configure curve: *[/home/sovrin/.sovrin/Node1/public_keys] 2017-10-11 17:04:08,773 | DEBUG | zstack.py ( 347) | open | Node1 will bind its listener at 9701 2017-10-11 17:04:08,773 | INFO | stacks.py ( 84) | start | CONNECTION: Node1 listening for other nodes at 0.0.0.0:9701 2017-10-11 17:04:08,773 | DEBUG | zstack.py ( 319) | start | Node1C starting with restricted as False and reSetupAuth as True 2017-10-11 17:04:08,773 | DEBUG | authenticator.py ( 31) | start | Starting ZAP at inproc://zeromq.zap.2 2017-10-11 17:04:08,773 | DEBUG | base.py ( 72) | allow | Allowing 0.0.0.0 2017-10-11 17:04:08,774 | DEBUG | base.py ( 112) | configure_curve | Configure curve: *[*] 2017-10-11 17:04:08,774 | DEBUG | zstack.py ( 347) | open | Node1C will bind its listener at 9702 2017-10-11 17:04:08,774 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action propose_view_change with id 3 to run in 60 seconds 2017-10-11 17:04:08,774 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 4 to run in 3 seconds 2017-10-11 17:04:08,774 | DEBUG | has_action_queue.py ( 79) | startRepeating | dump_json_file will be repeating every 60 seconds 2017-10-11 17:04:08,774 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 5 to run in 60 seconds 2017-10-11 17:04:08,774 | INFO | node.py ( 621) | start | Node1 first time running... 2017-10-11 17:04:08,775 | DEBUG | kit_zstack.py ( 97) | connectToMissing | CONNECTION: Node1 found the following missing connections: Node4, Node2, Node3 2017-10-11 17:04:08,776 | TRACE | remote.py ( 86) | connect | connecting socket 78 41672656 to remote Node4:HA(host='10.0.0.5', port=9707) 2017-10-11 17:04:08,776 | INFO | zstack.py ( 590) | connect | CONNECTION: Node1 looking for Node4 at 10.0.0.5:9707 2017-10-11 17:04:08,778 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 pinged Node4 2017-10-11 17:04:08,778 | TRACE | remote.py ( 86) | connect | connecting socket 81 41776320 to remote Node2:HA(host='10.0.0.3', port=9703) 2017-10-11 17:04:08,778 | INFO | zstack.py ( 590) | connect | CONNECTION: Node1 looking for Node2 at 10.0.0.3:9703 2017-10-11 17:04:08,778 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 pinged Node2 2017-10-11 17:04:08,779 | TRACE | remote.py ( 86) | connect | connecting socket 82 41813840 to remote Node3:HA(host='10.0.0.4', port=9705) 2017-10-11 17:04:08,779 | INFO | zstack.py ( 590) | connect | CONNECTION: Node1 looking for Node3 at 10.0.0.4:9705 2017-10-11 17:04:08,779 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 pinged Node3 2017-10-11 17:04:08,781 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:04:08,794 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'pi' to Node4 2017-10-11 17:04:08,794 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'pi' to Node4 2017-10-11 17:04:08,794 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'pi' to Node2 2017-10-11 17:04:08,797 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'pi' to Node2 2017-10-11 17:04:08,798 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'pi' to Node3 2017-10-11 17:04:08,798 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'pi' to Node3 2017-10-11 17:04:08,833 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 4 messages through listener 2017-10-11 17:04:08,834 | DEBUG | zstack.py ( 663) | handlePingPong | Node1 got ping from Node4 2017-10-11 17:04:08,834 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 ponged Node4 2017-10-11 17:04:08,835 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}'], 'signature': None} 2017-10-11 17:04:08,835 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:04:08,835 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}'], 'signature': None} 2017-10-11 17:04:08,836 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,836 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:08,836 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,836 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,837 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:08,837 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,837 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"name":"Node1","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"viewNo":0,"op":"VIEW_CHANGE_DONE"}', '{"name":"Node1","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"viewNo":0,"op":"VIEW_CHANGE_DONE"}'], 'signature': None} 2017-10-11 17:04:08,837 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"name":"Node1","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"viewNo":0,"op":"VIEW_CHANGE_DONE"}', '{"name":"Node1","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"viewNo":0,"op":"VIEW_CHANGE_DONE"}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:04:08,837 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"name":"Node1","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"viewNo":0,"op":"VIEW_CHANGE_DONE"}', '{"name":"Node1","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"viewNo":0,"op":"VIEW_CHANGE_DONE"}'], 'signature': None} 2017-10-11 17:04:08,838 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,838 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:08,838 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,838 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,838 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:08,838 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,839 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,839 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'op': 'VIEW_CHANGE_DONE', 'viewNo': 0}, 'Node4') 2017-10-11 17:04:08,840 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,840 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,840 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'op': 'VIEW_CHANGE_DONE', 'viewNo': 0}, 'Node4') 2017-10-11 17:04:08,840 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,841 | DEBUG | zstack.py ( 669) | handlePingPong | Node1 got pong from Node4 2017-10-11 17:04:08,841 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 17:04:08,842 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 17:04:08,842 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 17:04:08,843 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 17:04:08,843 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}, 'Node4') 2017-10-11 17:04:08,844 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}, 'Node4') 2017-10-11 17:04:08,845 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 5 msgs to Node4 into one transmission 2017-10-11 17:04:08,845 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'po', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"op":"MESSAGE_RESPONSE"}']) 2017-10-11 17:04:08,845 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node4: b'{"messages":["po","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:04:08,845 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["po","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' to Node4 2017-10-11 17:04:08,846 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from set() to {'Node4'} 2017-10-11 17:04:08,846 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node1 now connected to Node4 2017-10-11 17:04:08,847 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} to 1 recipients: ['Node4'] 2017-10-11 17:04:08,848 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"txnSeqNo":4,"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}' to Node4 2017-10-11 17:04:08,848 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"txnSeqNo":4,"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}' to Node4 2017-10-11 17:04:08,870 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:04:08,871 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"viewNo":0,"op":"CURRENT_STATE"}', '{"op":"LEDGER_STATUS","merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"ledgerId":0,"txnSeqNo":4}', '{"op":"LEDGER_STATUS","merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","viewNo":null,"ppSeqNo":null,"ledgerId":1,"txnSeqNo":11}', '{"op":"LEDGER_STATUS","merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","viewNo":null,"ppSeqNo":null,"ledgerId":2,"txnSeqNo":1}'], 'signature': None} 2017-10-11 17:04:08,871 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"viewNo":0,"op":"CURRENT_STATE"}', '{"op":"LEDGER_STATUS","merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"ledgerId":0,"txnSeqNo":4}', '{"op":"LEDGER_STATUS","merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","viewNo":null,"ppSeqNo":null,"ledgerId":1,"txnSeqNo":11}', '{"op":"LEDGER_STATUS","merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","viewNo":null,"ppSeqNo":null,"ledgerId":2,"txnSeqNo":1}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:04:08,871 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"viewNo":0,"op":"CURRENT_STATE"}', '{"op":"LEDGER_STATUS","merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"ledgerId":0,"txnSeqNo":4}', '{"op":"LEDGER_STATUS","merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","viewNo":null,"ppSeqNo":null,"ledgerId":1,"txnSeqNo":11}', '{"op":"LEDGER_STATUS","merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","viewNo":null,"ppSeqNo":null,"ledgerId":2,"txnSeqNo":1}'], 'signature': None} 2017-10-11 17:04:08,871 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} 2017-10-11 17:04:08,872 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'CURRENT_STATE', 'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0}, 'Node4') 2017-10-11 17:04:08,872 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} 2017-10-11 17:04:08,872 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:04:08,872 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 4, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None, 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:08,872 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:04:08,872 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} 2017-10-11 17:04:08,872 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 11, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:08,872 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} 2017-10-11 17:04:08,873 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} 2017-10-11 17:04:08,873 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 1, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:08,873 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} 2017-10-11 17:04:08,873 | DEBUG | node.py ( 907) | process_current_state_message | Node1 processing current state CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} from Node4 2017-10-11 17:04:08,873 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}, 'Node4') 2017-10-11 17:04:08,873 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} from Node4 2017-10-11 17:04:08,874 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,874 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,874 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node4 2017-10-11 17:04:08,874 | DEBUG | ledger_manager.py (1016) | stashLedgerStatus | Node1 stashing ledger status LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node4 2017-10-11 17:04:08,874 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node4 2017-10-11 17:04:08,874 | DEBUG | ledger_manager.py (1016) | stashLedgerStatus | Node1 stashing ledger status LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node4 2017-10-11 17:04:08,875 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:04:08,875 | DEBUG | zstack.py ( 663) | handlePingPong | Node1C got ping from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:04:08,875 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'po' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:04:08,875 | DEBUG | zstack.py ( 645) | sendPingPong | Node1C ponged b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:04:08,887 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:04:08,888 | TRACE | node.py (1388) | validateClientMsg | Node1C received CLIENT message: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'viewNo': None} 2017-10-11 17:04:08,888 | DEBUG | node.py (1434) | processClientInBox | Node1C processing b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' request LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'viewNo': None} 2017-10-11 17:04:08,888 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '6mQmSGzvyAeSpp5E7rBcYyAwgim9pTdggDXwL4quw8HA', 'viewNo': None} from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:04:08,888 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,888 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"txnSeqNo":4,"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:04:08,888 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,889 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,923 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 5 messages through listener 2017-10-11 17:04:08,924 | DEBUG | zstack.py ( 663) | handlePingPong | Node1 got ping from Node3 2017-10-11 17:04:08,924 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 ponged Node3 2017-10-11 17:04:08,924 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}'], 'signature': None} 2017-10-11 17:04:08,924 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}'], 'signature': None, 'op': 'BATCH'}, 'Node3') 2017-10-11 17:04:08,924 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}'], 'signature': None} 2017-10-11 17:04:08,924 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,925 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:08,925 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,925 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,925 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:08,925 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,925 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}'], 'signature': None} 2017-10-11 17:04:08,925 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}'], 'signature': None, 'op': 'BATCH'}, 'Node3') 2017-10-11 17:04:08,925 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}'], 'signature': None} 2017-10-11 17:04:08,926 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,926 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:08,926 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,926 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,926 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:08,926 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,927 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,927 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'op': 'VIEW_CHANGE_DONE', 'viewNo': 0}, 'Node3') 2017-10-11 17:04:08,927 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,927 | DEBUG | zstack.py ( 669) | handlePingPong | Node1 got pong from Node3 2017-10-11 17:04:08,928 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,928 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,929 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,929 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,930 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}, 'Node3') 2017-10-11 17:04:08,930 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 5 msgs to Node3 into one transmission 2017-10-11 17:04:08,930 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'po', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"op":"MESSAGE_RESPONSE"}']) 2017-10-11 17:04:08,930 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node3: b'{"messages":["po","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:04:08,931 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["po","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' to Node3 2017-10-11 17:04:08,931 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node4'} to {'Node4', 'Node3'} 2017-10-11 17:04:08,932 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node1 now connected to Node3 2017-10-11 17:04:08,932 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from starting to started_hungry 2017-10-11 17:04:08,932 | DEBUG | node.py ( 941) | checkInstances | Node1 choosing to start election on the basis of count 3 and nodes {'Node4', 'Node3'} 2017-10-11 17:04:08,932 | DEBUG | primary_selector.py ( 77) | get_msgs_for_lagged_nodes | Node1 has no ViewChangeDone message to send for view 0 2017-10-11 17:04:08,932 | DEBUG | node.py ( 902) | send_current_state_to_lagging_node | Node1 sending current state CURRENT_STATE{'primary': [], 'viewNo': 0} to lagged node Node3 2017-10-11 17:04:08,932 | DEBUG | node.py (2608) | send | Node1 sending message CURRENT_STATE{'primary': [], 'viewNo': 0} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,933 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,933 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node4 : VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 190) | _hasViewChangeQuorum | Node1 needs 1 ViewChangeDone messages 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 266) | _startSelection | Node1 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node4 : VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 190) | _hasViewChangeQuorum | Node1 needs 1 ViewChangeDone messages 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 266) | _startSelection | Node1 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node4 : VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 190) | _hasViewChangeQuorum | Node1 needs 1 ViewChangeDone messages 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 266) | _startSelection | Node1 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages 2017-10-11 17:04:08,934 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node3 : VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,935 | DEBUG | primary_selector.py ( 196) | _hasViewChangeQuorum | Node1 got view change quorum (2 >= 2) 2017-10-11 17:04:08,935 | DEBUG | primary_selector.py ( 209) | has_view_change_from_primary | Node1 has not received ViewChangeDone from the next primary Node1 (viewNo: 0, totalNodes: 4) 2017-10-11 17:04:08,935 | DEBUG | primary_selector.py ( 266) | _startSelection | Node1 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages 2017-10-11 17:04:08,935 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 2 msgs to Node3 into one transmission 2017-10-11 17:04:08,935 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"viewNo":0,"primary":[],"op":"CURRENT_STATE"}', b'{"txnSeqNo":4,"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}']) 2017-10-11 17:04:08,935 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node3: b'{"messages":["{\\"viewNo\\":0,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"txnSeqNo\\":4,\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:04:08,936 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"viewNo\\":0,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"txnSeqNo\\":4,\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' to Node3 2017-10-11 17:04:08,962 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 6 messages through listener 2017-10-11 17:04:08,963 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'messages': ['{"viewNo":0,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"op":"CURRENT_STATE"}', '{"txnSeqNo":4,"viewNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","op":"LEDGER_STATUS","ledgerId":0,"ppSeqNo":null}', '{"txnSeqNo":11,"viewNo":null,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","op":"LEDGER_STATUS","ledgerId":1,"ppSeqNo":null}', '{"txnSeqNo":1,"viewNo":null,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","op":"LEDGER_STATUS","ledgerId":2,"ppSeqNo":null}'], 'signature': None} 2017-10-11 17:04:08,963 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"viewNo":0,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"op":"CURRENT_STATE"}', '{"txnSeqNo":4,"viewNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","op":"LEDGER_STATUS","ledgerId":0,"ppSeqNo":null}', '{"txnSeqNo":11,"viewNo":null,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","op":"LEDGER_STATUS","ledgerId":1,"ppSeqNo":null}', '{"txnSeqNo":1,"viewNo":null,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","op":"LEDGER_STATUS","ledgerId":2,"ppSeqNo":null}'], 'signature': None, 'op': 'BATCH'}, 'Node3') 2017-10-11 17:04:08,963 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"viewNo":0,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"op":"CURRENT_STATE"}', '{"txnSeqNo":4,"viewNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","op":"LEDGER_STATUS","ledgerId":0,"ppSeqNo":null}', '{"txnSeqNo":11,"viewNo":null,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","op":"LEDGER_STATUS","ledgerId":1,"ppSeqNo":null}', '{"txnSeqNo":1,"viewNo":null,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","op":"LEDGER_STATUS","ledgerId":2,"ppSeqNo":null}'], 'signature': None} 2017-10-11 17:04:08,963 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} 2017-10-11 17:04:08,963 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'CURRENT_STATE', 'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0}, 'Node3') 2017-10-11 17:04:08,963 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} 2017-10-11 17:04:08,964 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:04:08,964 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None, 'ppSeqNo': None, 'op': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:08,964 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:04:08,964 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} 2017-10-11 17:04:08,964 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None, 'op': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:08,965 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} 2017-10-11 17:04:08,965 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} 2017-10-11 17:04:08,965 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None, 'op': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:08,965 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} 2017-10-11 17:04:08,965 | DEBUG | zstack.py ( 663) | handlePingPong | Node1 got ping from Node2 2017-10-11 17:04:08,965 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 ponged Node2 2017-10-11 17:04:08,966 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}'], 'signature': None} 2017-10-11 17:04:08,966 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}'], 'signature': None, 'op': 'BATCH'}, 'Node2') 2017-10-11 17:04:08,966 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}'], 'signature': None} 2017-10-11 17:04:08,966 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,967 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:08,967 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,968 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,968 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:08,968 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,968 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}'], 'signature': None} 2017-10-11 17:04:08,968 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}'], 'signature': None, 'op': 'BATCH'}, 'Node2') 2017-10-11 17:04:08,968 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}'], 'signature': None} 2017-10-11 17:04:08,968 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,968 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:08,968 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,969 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,969 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST', 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:08,969 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:08,969 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,969 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0, 'op': 'VIEW_CHANGE_DONE'}, 'Node2') 2017-10-11 17:04:08,969 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,969 | DEBUG | zstack.py ( 669) | handlePingPong | Node1 got pong from Node2 2017-10-11 17:04:08,970 | DEBUG | node.py ( 907) | process_current_state_message | Node1 processing current state CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} from Node3 2017-10-11 17:04:08,970 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}, 'Node3') 2017-10-11 17:04:08,970 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} from Node3 2017-10-11 17:04:08,970 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,971 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,971 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node3'} that its ledger of type 0 is latest 2017-10-11 17:04:08,971 | DEBUG | ledger_manager.py ( 315) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} that it does not need catchup 2017-10-11 17:04:08,971 | DEBUG | node.py (1479) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 17:04:08,971 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 17:04:08,971 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 17:04:08,971 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 17:04:08,971 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 17:04:08,971 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:04:08,971 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:04:08,972 | INFO | node.py (1489) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 0 2017-10-11 17:04:08,972 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:08,972 | DEBUG | node.py ( 941) | checkInstances | Node1 choosing to start election on the basis of count 3 and nodes {'Node4', 'Node3'} 2017-10-11 17:04:08,972 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:04:08,972 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node1 for ledger status of ledger 2 2017-10-11 17:04:08,972 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,972 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node2 for ledger status of ledger 2 2017-10-11 17:04:08,972 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,972 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node3 for ledger status of ledger 2 2017-10-11 17:04:08,973 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 17:04:08,973 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node4 for ledger status of ledger 2 2017-10-11 17:04:08,973 | DEBUG | ledger_manager.py (1027) | processStashedLedgerStatuses | Node1 going to process 1 stashed ledger statuses for ledger 2 2017-10-11 17:04:08,973 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node4 2017-10-11 17:04:08,973 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:08,973 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:08,973 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node1 completed catching up ledger 0, caught up 0 in total 2017-10-11 17:04:08,973 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node3 2017-10-11 17:04:08,973 | DEBUG | ledger_manager.py (1016) | stashLedgerStatus | Node1 stashing ledger status LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node3 2017-10-11 17:04:08,974 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node3 2017-10-11 17:04:08,974 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:08,974 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:08,974 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node3'} that its ledger of type 2 is latest 2017-10-11 17:04:08,974 | DEBUG | ledger_manager.py ( 315) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} that it does not need catchup 2017-10-11 17:04:08,974 | DEBUG | node.py (1479) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 17:04:08,974 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 17:04:08,974 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 17:04:08,974 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 17:04:08,974 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 17:04:08,974 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:04:08,975 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:04:08,975 | INFO | node.py (1489) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 2 2017-10-11 17:04:08,976 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:08,976 | DEBUG | pool_config.py ( 35) | processLedger | processing config ledger for any POOL_CONFIGs 2017-10-11 17:04:08,976 | DEBUG | upgrader.py ( 208) | processLedger | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv processing config ledger for any upgrades 2017-10-11 17:04:08,976 | INFO | upgrader.py ( 214) | processLedger | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv found upgrade START txn OrderedDict([('action', 'start'), ('force', True), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade37'), ('reqId', 1507741413733229), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-10-11T14:30:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-10-11T14:20:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-10-11T14:25:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-10-11T14:15:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz'), ('timeout', 10), ('txnTime', 1507741413), ('type', '109'), ('version', '1.1.37'), ('seqNo', 1)]) 2017-10-11 17:04:08,976 | INFO | upgrader.py ( 292) | handleUpgradeTxn | Node 'Node1' handles upgrade txn OrderedDict([('action', 'start'), ('force', True), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade37'), ('reqId', 1507741413733229), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-10-11T14:30:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-10-11T14:20:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-10-11T14:25:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-10-11T14:15:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '4WCgZKWQtWJxH8rXhpJxKzyuWEuCDKLZqR8VbAHeb4Vfk9RjkQ5oSDz7FnD5tNYbXL97suiRXwySXdq2jvd18tXz'), ('timeout', 10), ('txnTime', 1507741413), ('type', '109'), ('version', '1.1.37'), ('seqNo', 1)]) 2017-10-11 17:04:08,976 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:04:08,976 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node1 for ledger status of ledger 1 2017-10-11 17:04:08,977 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,977 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node2 for ledger status of ledger 1 2017-10-11 17:04:08,977 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node3'] 2017-10-11 17:04:08,977 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node3 for ledger status of ledger 1 2017-10-11 17:04:08,977 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node4'] 2017-10-11 17:04:08,977 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node4 for ledger status of ledger 1 2017-10-11 17:04:08,977 | DEBUG | ledger_manager.py (1027) | processStashedLedgerStatuses | Node1 going to process 2 stashed ledger statuses for ledger 1 2017-10-11 17:04:08,977 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node4 2017-10-11 17:04:08,977 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:08,977 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:08,978 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node3 2017-10-11 17:04:08,978 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:08,978 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:08,978 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node3'} that its ledger of type 1 is latest 2017-10-11 17:04:08,978 | DEBUG | ledger_manager.py ( 315) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} that it does not need catchup 2017-10-11 17:04:08,978 | DEBUG | node.py (1479) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 17:04:08,978 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 17:04:08,978 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 17:04:08,978 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 17:04:08,978 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 17:04:08,978 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:04:08,979 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:04:08,979 | INFO | node.py (1489) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 1 2017-10-11 17:04:08,979 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:08,979 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node1 completed catching up ledger 1, caught up 0 in total 2017-10-11 17:04:08,979 | DEBUG | node.py (1609) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 17:04:08,979 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:04:08,979 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:04:08,979 | DEBUG | primary_selector.py ( 196) | _hasViewChangeQuorum | Node1 got view change quorum (2 >= 2) 2017-10-11 17:04:08,980 | DEBUG | primary_selector.py ( 238) | has_sufficient_same_view_change_done_messages | Node1 found acceptable primary Node1 and ledger info ((0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'), (2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ')) 2017-10-11 17:04:08,980 | DEBUG | node.py (1549) | is_catchup_needed | Node1 is caught up for the current view 0 2017-10-11 17:04:08,980 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node1 does not need any more catchups 2017-10-11 17:04:08,981 | DEBUG | primary_decider.py ( 134) | send | Node1's elector sending VIEW_CHANGE_DONE{'ledgerInfo': [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'), (2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ')], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,981 | DEBUG | primary_selector.py ( 196) | _hasViewChangeQuorum | Node1 got view change quorum (3 >= 2) 2017-10-11 17:04:08,981 | DEBUG | primary_selector.py ( 215) | has_view_change_from_primary | Node1 received ViewChangeDone from primary Node1 2017-10-11 17:04:08,981 | DEBUG | primary_selector.py ( 281) | _startSelection | Node1 starting selection 2017-10-11 17:04:08,982 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node1:0 selected primary Node1:0 for instance 0 (view 0) 2017-10-11 17:04:08,982 | INFO | node.py ( 510) | start_participating | Node1 started participating 2017-10-11 17:04:08,982 | DEBUG | replica.py ( 408) | primaryName | Node1:0 setting primaryName for view no 0 to: Node1:0 2017-10-11 17:04:08,982 | DEBUG | replica.py (1694) | _gc | Node1:0 cleaning up till (0, 0) 2017-10-11 17:04:08,982 | DEBUG | replica.py (1709) | _gc | Node1:0 found 0 3-phase keys to clean 2017-10-11 17:04:08,982 | DEBUG | replica.py (1711) | _gc | Node1:0 found 0 request keys to clean 2017-10-11 17:04:08,982 | DEBUG | replica.py ( 313) | h | Node1:0 set watermarks as 0 300 2017-10-11 17:04:08,982 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node1:0 declares view change 0 as completed for instance 0, new primary is Node1:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'), (2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ')] 2017-10-11 17:04:08,983 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node1:1 selected primary Node2:1 for instance 1 (view 0) 2017-10-11 17:04:08,983 | DEBUG | replica.py ( 408) | primaryName | Node1:1 setting primaryName for view no 0 to: Node2:1 2017-10-11 17:04:08,983 | DEBUG | replica.py (1694) | _gc | Node1:1 cleaning up till (0, 0) 2017-10-11 17:04:08,984 | DEBUG | replica.py (1709) | _gc | Node1:1 found 0 3-phase keys to clean 2017-10-11 17:04:08,984 | DEBUG | replica.py (1711) | _gc | Node1:1 found 0 request keys to clean 2017-10-11 17:04:08,984 | DEBUG | replica.py ( 313) | h | Node1:1 set watermarks as 0 300 2017-10-11 17:04:08,984 | DEBUG | replica.py ( 514) | _setup_for_non_master | Node1:1 Setting last ordered for non-master as (0, 0) 2017-10-11 17:04:08,984 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:1 set last ordered as (0, 0) 2017-10-11 17:04:08,984 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node1:1 declares view change 0 as completed for instance 1, new primary is Node2:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'), (2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ')] 2017-10-11 17:04:08,984 | INFO | upgrader.py ( 150) | should_notify_about_upgrade_result | Node's 'Node1' last upgrade txn is None 2017-10-11 17:04:08,985 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node1 completed catching up ledger 2, caught up 0 in total 2017-10-11 17:04:08,985 | DEBUG | node.py (1609) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 17:04:08,985 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:04:08,985 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:04:08,985 | DEBUG | primary_selector.py ( 196) | _hasViewChangeQuorum | Node1 got view change quorum (3 >= 2) 2017-10-11 17:04:08,985 | DEBUG | node.py (1549) | is_catchup_needed | Node1 is caught up for the current view 0 2017-10-11 17:04:08,985 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node1 does not need any more catchups 2017-10-11 17:04:08,986 | DEBUG | primary_selector.py ( 281) | _startSelection | Node1 starting selection 2017-10-11 17:04:08,986 | DEBUG | primary_selector.py ( 284) | _startSelection | Node1:0 already has a primary 2017-10-11 17:04:08,986 | DEBUG | primary_selector.py ( 284) | _startSelection | Node1:1 already has a primary 2017-10-11 17:04:08,986 | INFO | node.py (1624) | no_more_catchups_needed | Node1 starting to participate since catchup is done, primaries are selected but mode was not set to participating 2017-10-11 17:04:08,986 | INFO | node.py ( 510) | start_participating | Node1 started participating 2017-10-11 17:04:08,986 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,987 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,987 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,987 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None}, 'msg_type': 'LEDGER_STATUS'} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,987 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}, 'Node2') 2017-10-11 17:04:08,988 | DEBUG | node.py (2608) | send | Node1 sending message VIEW_CHANGE_DONE{'ledgerInfo': [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'), (2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ')], 'name': 'Node1', 'viewNo': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:04:08,988 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node3 : VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,988 | DEBUG | message_processor.py ( 29) | discard | Node1 discarding message VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} because it already decided primary which is Node1:0 2017-10-11 17:04:08,988 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node2 : VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:08,988 | DEBUG | message_processor.py ( 29) | discard | Node1 discarding message VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} because it already decided primary which is Node1:0 2017-10-11 17:04:08,989 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 5 msgs to Node4 into one transmission 2017-10-11 17:04:08,989 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0,"op":"VIEW_CHANGE_DONE"}']) 2017-10-11 17:04:08,989 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node4: b'{"messages":["{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0,\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:04:08,989 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0,\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' to Node4 2017-10-11 17:04:08,989 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 10 msgs to Node2 into one transmission 2017-10-11 17:04:08,989 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'po', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0,"op":"VIEW_CHANGE_DONE"}']) 2017-10-11 17:04:08,990 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node2: b'{"messages":["po","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0,\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:04:08,990 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["po","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":1,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":11,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0,\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' to Node2 2017-10-11 17:04:08,990 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 5 msgs to Node3 into one transmission 2017-10-11 17:04:08,991 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0,"op":"VIEW_CHANGE_DONE"}']) 2017-10-11 17:04:08,991 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node3: b'{"messages":["{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0,\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:04:08,991 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0,\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' to Node3 2017-10-11 17:04:08,991 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node4', 'Node3'} to {'Node4', 'Node2', 'Node3'} 2017-10-11 17:04:08,991 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node1 now connected to Node2 2017-10-11 17:04:08,992 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from started_hungry to started 2017-10-11 17:04:08,992 | DEBUG | node.py ( 941) | checkInstances | Node1 choosing to start election on the basis of count 4 and nodes {'Node4', 'Node2', 'Node3'} 2017-10-11 17:04:08,992 | DEBUG | node.py ( 902) | send_current_state_to_lagging_node | Node1 sending current state CURRENT_STATE{'primary': [VIEW_CHANGE_DONE{'ledgerInfo': ((0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'), (2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ')), 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} to lagged node Node2 2017-10-11 17:04:08,992 | DEBUG | node.py (2608) | send | Node1 sending message CURRENT_STATE{'primary': [VIEW_CHANGE_DONE{'ledgerInfo': ((0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'), (2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ')), 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,992 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,993 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,993 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} to 1 recipients: ['Node2'] 2017-10-11 17:04:08,993 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 4 msgs to Node2 into one transmission 2017-10-11 17:04:08,994 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"viewNo":0,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"op":"CURRENT_STATE"}', b'{"txnSeqNo":4,"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}', b'{"txnSeqNo":11,"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}', b'{"txnSeqNo":1,"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}']) 2017-10-11 17:04:08,994 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node2: b'{"messages":["{\\"viewNo\\":0,\\"primary\\":[{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0}],\\"op\\":\\"CURRENT_STATE\\"}","{\\"txnSeqNo\\":4,\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"txnSeqNo\\":11,\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"txnSeqNo\\":1,\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:04:08,994 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"viewNo\\":0,\\"primary\\":[{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,11,\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\"],[2,1,\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\"]],\\"name\\":\\"Node1\\",\\"viewNo\\":0}],\\"op\\":\\"CURRENT_STATE\\"}","{\\"txnSeqNo\\":4,\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"txnSeqNo\\":11,\\"ledgerId\\":1,\\"merkleRoot\\":\\"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"txnSeqNo\\":1,\\"ledgerId\\":2,\\"merkleRoot\\":\\"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ\\",\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' to Node2 2017-10-11 17:04:09,018 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:04:09,018 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}'], 'signature': None} 2017-10-11 17:04:09,018 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:04:09,018 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}'], 'signature': None} 2017-10-11 17:04:09,019 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,019 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:09,019 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,019 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,019 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:09,020 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,020 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,020 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:09,020 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,020 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,020 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node4') 2017-10-11 17:04:09,020 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,020 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}'], 'signature': None} 2017-10-11 17:04:09,021 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}'], 'signature': None, 'op': 'BATCH'}, 'Node2') 2017-10-11 17:04:09,021 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}'], 'signature': None} 2017-10-11 17:04:09,021 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,021 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:09,021 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,021 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,021 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:09,021 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,021 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,022 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:09,022 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,022 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,022 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:09,022 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,022 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null}}'], 'signature': None} 2017-10-11 17:04:09,023 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null}}'], 'signature': None, 'op': 'BATCH'}, 'Node3') 2017-10-11 17:04:09,023 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ppSeqNo":null,"txnSeqNo":1,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ppSeqNo":null,"txnSeqNo":11,"viewNo":null}}'], 'signature': None} 2017-10-11 17:04:09,023 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,023 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:09,023 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,023 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,023 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:09,023 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 2}, 'msg': {'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,024 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,024 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:09,024 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,024 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,024 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:04:09,024 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'params': {'ledgerId': 1}, 'msg': {'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None, 'ppSeqNo': None}, 'msg_type': 'LEDGER_STATUS'} 2017-10-11 17:04:09,024 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node4 2017-10-11 17:04:09,025 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,025 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,025 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node4 2017-10-11 17:04:09,025 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,025 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,025 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node4 2017-10-11 17:04:09,026 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,026 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,026 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node4 2017-10-11 17:04:09,026 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,026 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,026 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node2 2017-10-11 17:04:09,027 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,027 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,027 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 2 is latest 2017-10-11 17:04:09,027 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node2 2017-10-11 17:04:09,027 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,027 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,027 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 2 is latest 2017-10-11 17:04:09,028 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node2 2017-10-11 17:04:09,028 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,028 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,028 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 1 is latest 2017-10-11 17:04:09,029 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node2 2017-10-11 17:04:09,029 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,029 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,029 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2'} that its ledger of type 1 is latest 2017-10-11 17:04:09,029 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node3 2017-10-11 17:04:09,030 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,030 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,030 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2', 'Node3'} that its ledger of type 2 is latest 2017-10-11 17:04:09,030 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node3 2017-10-11 17:04:09,030 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,030 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,031 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2', 'Node3'} that its ledger of type 2 is latest 2017-10-11 17:04:09,031 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node3 2017-10-11 17:04:09,031 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,031 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,031 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2', 'Node3'} that its ledger of type 1 is latest 2017-10-11 17:04:09,031 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node3 2017-10-11 17:04:09,032 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,032 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,032 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2', 'Node3'} that its ledger of type 1 is latest 2017-10-11 17:04:09,034 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:04:09,035 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"op":"CURRENT_STATE","viewNo":0}', '{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"viewNo":null}', '{"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ledgerId":1,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":11,"viewNo":null}', '{"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ledgerId":2,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":1,"viewNo":null}'], 'signature': None} 2017-10-11 17:04:09,035 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"op":"CURRENT_STATE","viewNo":0}', '{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"viewNo":null}', '{"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ledgerId":1,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":11,"viewNo":null}', '{"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ledgerId":2,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":1,"viewNo":null}'], 'signature': None, 'op': 'BATCH'}, 'Node2') 2017-10-11 17:04:09,035 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,11,"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq"],[2,1,"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"]],"name":"Node1","viewNo":0}],"op":"CURRENT_STATE","viewNo":0}', '{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"viewNo":null}', '{"merkleRoot":"6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq","ledgerId":1,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":11,"viewNo":null}', '{"merkleRoot":"AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ","ledgerId":2,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":1,"viewNo":null}'], 'signature': None} 2017-10-11 17:04:09,035 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} 2017-10-11 17:04:09,035 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'op': 'CURRENT_STATE'}, 'Node2') 2017-10-11 17:04:09,035 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} 2017-10-11 17:04:09,036 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:04:09,036 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ppSeqNo': None, 'viewNo': None, 'op': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:09,036 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:04:09,036 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} 2017-10-11 17:04:09,036 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 11, 'ledgerId': 1, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'ppSeqNo': None, 'viewNo': None, 'op': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:09,036 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} 2017-10-11 17:04:09,036 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} 2017-10-11 17:04:09,036 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'txnSeqNo': 1, 'ledgerId': 2, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'ppSeqNo': None, 'viewNo': None, 'op': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:04:09,037 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} 2017-10-11 17:04:09,037 | DEBUG | node.py ( 907) | process_current_state_message | Node1 processing current state CURRENT_STATE{'primary': [{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}], 'viewNo': 0} from Node2 2017-10-11 17:04:09,037 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0}, 'Node2') 2017-10-11 17:04:09,037 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} from Node2 2017-10-11 17:04:09,037 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:09,037 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:04:09,038 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 11, 'ledgerId': 1, 'ppSeqNo': None, 'merkleRoot': '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq', 'viewNo': None} from Node2 2017-10-11 17:04:09,038 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,038 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 11 with 11 2017-10-11 17:04:09,038 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2', 'Node3'} that its ledger of type 1 is latest 2017-10-11 17:04:09,038 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 1, 'ledgerId': 2, 'ppSeqNo': None, 'merkleRoot': 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ', 'viewNo': None} from Node2 2017-10-11 17:04:09,038 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,039 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 1 with 1 2017-10-11 17:04:09,039 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node4', 'Node2', 'Node3'} that its ledger of type 2 is latest 2017-10-11 17:04:09,039 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node2 : VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} 2017-10-11 17:04:09,039 | DEBUG | message_processor.py ( 29) | discard | Node1 discarding message VIEW_CHANGE_DONE{'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 11, '6L8BXEMTzhyfkb3A6rx5qEZyraHJQeJ8d3RWWLwK8Hvq'], [2, 1, 'AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ']], 'name': 'Node1', 'viewNo': 0} because it already decided primary which is Node1:0 2017-10-11 17:04:11,780 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 4 2017-10-11 17:04:18,747 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 1 2017-10-11 17:04:18,747 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:04:18,747 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:04:18,747 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:04:18,748 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:04:18,748 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:04:18,748 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:04:18,748 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 6 to run in 10 seconds 2017-10-11 17:04:23,785 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:04:23,785 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:04:23,785 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:04:23,786 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:04:28,750 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 6 2017-10-11 17:04:28,750 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:04:28,751 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:04:28,751 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:04:28,751 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:04:28,751 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:04:28,751 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:04:28,751 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 7 to run in 10 seconds 2017-10-11 17:04:38,760 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 7 2017-10-11 17:04:38,760 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:04:38,760 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:04:38,760 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:04:38,760 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:04:38,760 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:04:38,760 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:04:38,760 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 8 to run in 10 seconds 2017-10-11 17:04:38,786 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:04:38,787 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:04:38,787 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:04:38,787 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:04:48,763 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 8 2017-10-11 17:04:48,763 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:04:48,763 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:04:48,763 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:04:48,763 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:04:48,763 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:04:48,763 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:04:48,763 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 9 to run in 10 seconds 2017-10-11 17:04:53,797 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:04:53,797 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:04:53,797 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:04:53,797 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:04:58,765 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 9 2017-10-11 17:04:58,765 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:04:58,765 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:04:58,765 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:04:58,765 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:04:58,766 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:04:58,766 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:04:58,766 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 10 to run in 10 seconds 2017-10-11 17:05:00,992 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:05:00,997 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on request 1507741500981867 2017-10-11 17:05:00,998 | TRACE | node.py (1388) | validateClientMsg | Node1C received CLIENT message: SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} 2017-10-11 17:05:00,998 | DEBUG | node.py (1434) | processClientInBox | Node1C processing b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} 2017-10-11 17:05:00,998 | DEBUG | node.py (1708) | processRequest | Node1 received client request: SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:05:00,998 | TRACE | propagator.py ( 165) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} 2017-10-11 17:05:00,999 | INFO | propagator.py ( 148) | propagate | Node1 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867) from client b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:05:00,999 | DEBUG | node.py (2608) | send | Node1 sending message PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:00,999 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} to its replicas since not finalised 2017-10-11 17:05:01,000 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"op":"REQACK"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:05:01,000 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"signature":"MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT3"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node4 2017-10-11 17:05:01,000 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"signature":"MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT3"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node4 2017-10-11 17:05:01,000 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"signature":"MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT3"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node2 2017-10-11 17:05:01,000 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"signature":"MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT3"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node2 2017-10-11 17:05:01,001 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"signature":"MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT3"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node3 2017-10-11 17:05:01,001 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"signature":"MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT3"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node3 2017-10-11 17:05:01,018 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:05:01,019 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741500981867 2017-10-11 17:05:01,020 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'request': {'reqId': 1507741500981867, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,020 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'reqId': 1507741500981867, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%', 'op': 'PROPAGATE'}, 'Node3') 2017-10-11 17:05:01,020 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'reqId': 1507741500981867, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,020 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741500981867 2017-10-11 17:05:01,021 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,021 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%', 'op': 'PROPAGATE'}, 'Node4') 2017-10-11 17:05:01,021 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,022 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741500981867 2017-10-11 17:05:01,028 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,029 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%', 'op': 'PROPAGATE'}, 'Node2') 2017-10-11 17:05:01,030 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,031 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'reqId': 1507741500981867, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,032 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} 2017-10-11 17:05:01,032 | DEBUG | propagator.py ( 214) | forward | Node1 forwarding request ('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867) to 0 replicas 2017-10-11 17:05:01,032 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,032 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} 2017-10-11 17:05:01,032 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} to its replicas since already forwarded 2017-10-11 17:05:01,032 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:01,033 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} 2017-10-11 17:05:01,034 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741500981867, 'signature': 'MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT3'}} to its replicas since already forwarded 2017-10-11 17:05:01,040 | DEBUG | replica.py ( 667) | create3PCBatch | Node1:0 creating batch 1 for ledger 1 with state root b'\x93\xdf\xc1\x87\x14\xea\xbfa}i\x80\x90\xd6\xf2\\6K~\x81x=4\xe4\xf0o\x81G.^Y\xb5\x07' 2017-10-11 17:05:01,044 | DEBUG | replica.py ( 700) | create3PCBatch | Node1:0 created a PRE-PREPARE with 1 requests for ledger 1 2017-10-11 17:05:01,044 | DEBUG | replica.py (2193) | send | Node1:0 sending PrePrepare 2017-10-11 17:05:01,044 | TRACE | replica.py (2194) | send | Node1:0 sending PREPREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], 'instId': 0} 2017-10-11 17:05:01,044 | DEBUG | replica.py ( 614) | trackBatches | Node1:0 tracking batch for PREPREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], 'instId': 0} with state root b'\x93\xdf\xc1\x87\x14\xea\xbfa}i\x80\x90\xd6\xf2\\6K~\x81x=4\xe4\xf0o\x81G.^Y\xb5\x07' 2017-10-11 17:05:01,045 | DEBUG | node.py (2608) | send | Node1 sending message PREPREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:01,045 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:01,046 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741500981867]], 'instId': 1} 2017-10-11 17:05:01,046 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerId': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'ppSeqNo': 1, 'viewNo': 0, 'discarded': 1, 'instId': 1, 'ppTime': 1507741501, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741500981867]], 'op': 'PREPREPARE'}, 'Node2') 2017-10-11 17:05:01,047 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741500981867]], 'instId': 1} 2017-10-11 17:05:01,048 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","viewNo":0,"ppTime":1507741501,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741500981867]]}' to Node4 2017-10-11 17:05:01,048 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","viewNo":0,"ppTime":1507741501,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741500981867]]}' to Node4 2017-10-11 17:05:01,048 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","viewNo":0,"ppTime":1507741501,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741500981867]]}' to Node2 2017-10-11 17:05:01,048 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","viewNo":0,"ppTime":1507741501,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741500981867]]}' to Node2 2017-10-11 17:05:01,048 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","viewNo":0,"ppTime":1507741501,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741500981867]]}' to Node3 2017-10-11 17:05:01,049 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","viewNo":0,"ppTime":1507741501,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741500981867]]}' to Node3 2017-10-11 17:05:01,049 | DEBUG | replica.py ( 823) | processPrePrepare | Node1:1 received PRE-PREPARE(0, 1) from Node2:1 at 2270586.50687226 2017-10-11 17:05:01,049 | DEBUG | replica.py ( 934) | doPrepare | Node1:1 Sending PREPARE(0, 1) at 2270586.507301203 2017-10-11 17:05:01,050 | DEBUG | replica.py (2193) | send | Node1:1 sending Prepare 2017-10-11 17:05:01,050 | TRACE | replica.py (2194) | send | Node1:1 sending PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} 2017-10-11 17:05:01,050 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} 2017-10-11 17:05:01,050 | DEBUG | replica.py ( 614) | trackBatches | Node1:1 tracking batch for PREPREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], 'instId': 1} with state root None 2017-10-11 17:05:01,050 | DEBUG | replica.py ( 842) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(0, 1) 2017-10-11 17:05:01,050 | DEBUG | node.py (2608) | send | Node1 sending message PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:01,051 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":null,"viewNo":0,"ppTime":1507741501,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:05:01,051 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":null,"viewNo":0,"ppTime":1507741501,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:05:01,051 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":null,"viewNo":0,"ppTime":1507741501,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:05:01,052 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":null,"viewNo":0,"ppTime":1507741501,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:05:01,054 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":null,"viewNo":0,"ppTime":1507741501,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:05:01,054 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","stateRootHash":null,"viewNo":0,"ppTime":1507741501,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:05:01,070 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:05:01,070 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} 2017-10-11 17:05:01,070 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'op': 'PREPARE', 'instId': 1}, 'Node3') 2017-10-11 17:05:01,071 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} 2017-10-11 17:05:01,071 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,071 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0, 'op': 'PREPARE'}, 'Node2') 2017-10-11 17:05:01,071 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,071 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} 2017-10-11 17:05:01,071 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'stateRootHash': None, 'txnRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:05:01,071 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} 2017-10-11 17:05:01,072 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,072 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 1) 2017-10-11 17:05:01,073 | DEBUG | replica.py ( 953) | doCommit | Node1:1 Sending COMMIT(0, 1) at 2270586.530483938 2017-10-11 17:05:01,073 | DEBUG | replica.py (2193) | send | Node1:1 sending Commit 2017-10-11 17:05:01,073 | TRACE | replica.py (2194) | send | Node1:1 sending COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:01,073 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:05:01,073 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 1) 2017-10-11 17:05:01,073 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741501, 'instId': 1} 2017-10-11 17:05:01,073 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 1) 2017-10-11 17:05:01,073 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:01,080 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"op":"COMMIT","instId":1,"viewNo":0}' to Node4 2017-10-11 17:05:01,080 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"op":"COMMIT","instId":1,"viewNo":0}' to Node4 2017-10-11 17:05:01,080 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"op":"COMMIT","instId":1,"viewNo":0}' to Node2 2017-10-11 17:05:01,081 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"op":"COMMIT","instId":1,"viewNo":0}' to Node2 2017-10-11 17:05:01,081 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"op":"COMMIT","instId":1,"viewNo":0}' to Node3 2017-10-11 17:05:01,081 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"op":"COMMIT","instId":1,"viewNo":0}' to Node3 2017-10-11 17:05:01,081 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:05:01,082 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","op":"PREPARE","viewNo":0,"ppTime":1507741501,"stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","instId":0}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":1}'], 'signature': None} 2017-10-11 17:05:01,082 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","op":"PREPARE","viewNo":0,"ppTime":1507741501,"stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","instId":0}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":1}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:05:01,082 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","ppSeqNo":1,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","op":"PREPARE","viewNo":0,"ppTime":1507741501,"stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","instId":0}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":1}'], 'signature': None} 2017-10-11 17:05:01,082 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,082 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:05:01,083 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,083 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:01,083 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 1, 'instId': 1, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:05:01,083 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:01,083 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:01,083 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 1, 'instId': 0, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:05:01,083 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:01,084 | DEBUG | replica.py ( 953) | doCommit | Node1:0 Sending COMMIT(0, 1) at 2270586.542093462 2017-10-11 17:05:01,085 | DEBUG | replica.py (2193) | send | Node1:0 sending Commit 2017-10-11 17:05:01,085 | TRACE | replica.py (2194) | send | Node1:0 sending COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:01,085 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:05:01,085 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 1) 2017-10-11 17:05:01,085 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 1) from Node4:0 2017-10-11 17:05:01,085 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:05:01,085 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 1) 2017-10-11 17:05:01,086 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 1) from Node4:1 2017-10-11 17:05:01,086 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:05:01,086 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 1) 2017-10-11 17:05:01,086 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:01,086 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:01,095 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'messages': ['{"ppSeqNo":1,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","ppTime":1507741501,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2"}', '{"viewNo":0,"instId":1,"op":"COMMIT","ppSeqNo":1}'], 'signature': None} 2017-10-11 17:05:01,096 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"ppSeqNo":1,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","ppTime":1507741501,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2"}', '{"viewNo":0,"instId":1,"op":"COMMIT","ppSeqNo":1}'], 'signature': None, 'op': 'BATCH'}, 'Node3') 2017-10-11 17:05:01,096 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"ppSeqNo":1,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE","ppTime":1507741501,"digest":"6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897","txnRootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2"}', '{"viewNo":0,"instId":1,"op":"COMMIT","ppSeqNo":1}'], 'signature': None} 2017-10-11 17:05:01,096 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,097 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'op': 'PREPARE', 'instId': 0}, 'Node3') 2017-10-11 17:05:01,097 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,097 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:01,097 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'instId': 1, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:05:01,098 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:01,098 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"op":"COMMIT","instId":0,"viewNo":0}' to Node4 2017-10-11 17:05:01,098 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"op":"COMMIT","instId":0,"viewNo":0}' to Node4 2017-10-11 17:05:01,098 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"op":"COMMIT","instId":0,"viewNo":0}' to Node2 2017-10-11 17:05:01,099 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"op":"COMMIT","instId":0,"viewNo":0}' to Node2 2017-10-11 17:05:01,099 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":1,"op":"COMMIT","instId":0,"viewNo":0}' to Node3 2017-10-11 17:05:01,099 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":1,"op":"COMMIT","instId":0,"viewNo":0}' to Node3 2017-10-11 17:05:01,100 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 1, 'digest': '6ea68488f1f1b7bd5690b63f741538fc871a0fd01740d645a253b285d4d72897', 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ppTime': 1507741501, 'instId': 0} 2017-10-11 17:05:01,101 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 1) 2017-10-11 17:05:01,101 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 1) from Node3:1 2017-10-11 17:05:01,101 | TRACE | replica.py ( 925) | tryOrder | Node1:1 returning request to node 2017-10-11 17:05:01,101 | DEBUG | replica.py (1444) | doOrder | Node1:1 ordering COMMIT (0, 1) 2017-10-11 17:05:01,101 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:1 set last ordered as (0, 1) 2017-10-11 17:05:01,101 | DEBUG | replica.py (2193) | send | Node1:1 sending Ordered 2017-10-11 17:05:01,102 | TRACE | replica.py (2194) | send | Node1:1 sending ORDERED{'ppSeqNo': 1, 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], 'instId': 1} 2017-10-11 17:05:01,102 | INFO | replica.py (1483) | order_3pc_key | Node1:1 ordered batch request, view no 0, ppSeqNo 1, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], discarded [] 2017-10-11 17:05:01,102 | DEBUG | replica.py (1556) | _newCheckpointState | Node1:1 adding new checkpoint state for (1, 100) 2017-10-11 17:05:01,102 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 1) 2017-10-11 17:05:01,103 | TRACE | node.py (1858) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:05:01,103 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:05:01,104 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:01,104 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'instId': 0, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:05:01,104 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:01,104 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:01,104 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'op': 'COMMIT', 'viewNo': 0, 'instId': 1}, 'Node2') 2017-10-11 17:05:01,104 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:01,105 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:01,105 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'op': 'COMMIT', 'viewNo': 0, 'instId': 0}, 'Node2') 2017-10-11 17:05:01,105 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:01,106 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 1) from Node3:0 2017-10-11 17:05:01,107 | TRACE | replica.py ( 925) | tryOrder | Node1:0 returning request to node 2017-10-11 17:05:01,107 | DEBUG | replica.py (1444) | doOrder | Node1:0 ordering COMMIT (0, 1) 2017-10-11 17:05:01,107 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:0 set last ordered as (0, 1) 2017-10-11 17:05:01,107 | DEBUG | replica.py (2193) | send | Node1:0 sending Ordered 2017-10-11 17:05:01,107 | TRACE | replica.py (2194) | send | Node1:0 sending ORDERED{'ppSeqNo': 1, 'txnRootHash': 'wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2', 'stateRootHash': '24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741501, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], 'instId': 0} 2017-10-11 17:05:01,107 | INFO | replica.py (1483) | order_3pc_key | Node1:0 ordered batch request, view no 0, ppSeqNo 1, ledger 1, state root 24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE, txn root wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)], discarded [] 2017-10-11 17:05:01,107 | DEBUG | replica.py (1556) | _newCheckpointState | Node1:0 adding new checkpoint state for (1, 100) 2017-10-11 17:05:01,107 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 1) 2017-10-11 17:05:01,108 | TRACE | message_processor.py ( 29) | discard | Node1:0 discarding message COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 0} because already ordered 3 phase message 2017-10-11 17:05:01,108 | TRACE | message_processor.py ( 29) | discard | Node1:1 discarding message COMMIT{'ppSeqNo': 1, 'viewNo': 0, 'instId': 1} because already ordered 3 phase message 2017-10-11 17:05:01,108 | DEBUG | node.py (1847) | processOrdered | Node1 executing Ordered batch 0 1 of 1 requests 2017-10-11 17:05:01,108 | DEBUG | ledger.py ( 52) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:05:01,109 | DEBUG | node.py (2386) | sendReplyToClient | Node1 sending reply for ('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867) to client 2017-10-11 17:05:01,109 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"result":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741500981867,"type":"1","dest":"542MVr22zcHbVyGzaXmbT3","auditPath":["D9h1kJ9Egbq3d9BNRpAgZQmRvDTACpiyucoes3URbZWF","7LVA4w9Yeaprrpy8NTv4yitZ7hrpsXCeduCMbuh7B936","Gi2HwXVBFyDVgs7rwus6Mvr4PTuRgvyN3tR5izEkMLcE"],"seqNo":12,"rootHash":"wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2","signature":"MfpjZjjhLbGuZwmjAMjFbKrZeitkMsLdmrXNK6yP62J3tDcbcvrtXp7dqqYhdZcvCw2DnwTt4FG5932Mg2Z1fnU","txnTime":1507741501},"op":"REPLY"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:05:01,109 | INFO | node.py (2290) | executeBatch | Node1 committed batch request, view no 0, ppSeqNo 1, ledger 1, state root 24sLDAYWjNXuoxUhsmcVPxLg8NK9vaWZAW75AQGrCtfE, txn root wLJkMsXcnqqii7GSsqKxSB4SC79yzrLi5Rx6dBv2ch2, requests: [('V4SGRU86Z58d6TV7PBUe6f', 1507741500981867)] 2017-10-11 17:05:01,109 | DEBUG | node.py (2305) | executeBatch | Node1 storing 3PC key (0, 1) for ledger 1 range (12, 12) 2017-10-11 17:05:01,110 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:04,010 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:05:04,015 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on request 1507741503995435 2017-10-11 17:05:04,015 | TRACE | node.py (1388) | validateClientMsg | Node1C received CLIENT message: SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} 2017-10-11 17:05:04,015 | DEBUG | node.py (1434) | processClientInBox | Node1C processing b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} 2017-10-11 17:05:04,016 | DEBUG | node.py (1708) | processRequest | Node1 received client request: SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} from b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:05:04,016 | TRACE | propagator.py ( 165) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} 2017-10-11 17:05:04,016 | INFO | propagator.py ( 148) | propagate | Node1 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435) from client b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' 2017-10-11 17:05:04,016 | DEBUG | node.py (2608) | send | Node1 sending message PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:04,016 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} to its replicas since not finalised 2017-10-11 17:05:04,016 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"op":"REQACK"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:05:04,016 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"signature":"4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT4"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node4 2017-10-11 17:05:04,017 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"signature":"4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT4"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node4 2017-10-11 17:05:04,017 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"signature":"4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT4"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node2 2017-10-11 17:05:04,017 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"signature":"4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT4"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node2 2017-10-11 17:05:04,017 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"signature":"4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT4"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node3 2017-10-11 17:05:04,017 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"signature":"4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ","operation":{"type":"1","dest":"542MVr22zcHbVyGzaXmbT4"}},"senderClient":"fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%","op":"PROPAGATE"}' to Node3 2017-10-11 17:05:04,033 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:05:04,034 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741503995435 2017-10-11 17:05:04,034 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'request': {'reqId': 1507741503995435, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,035 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'reqId': 1507741503995435, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%', 'op': 'PROPAGATE'}, 'Node3') 2017-10-11 17:05:04,041 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'reqId': 1507741503995435, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,041 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741503995435 2017-10-11 17:05:04,042 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,042 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%', 'op': 'PROPAGATE'}, 'Node4') 2017-10-11 17:05:04,042 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,042 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741503995435 2017-10-11 17:05:04,043 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,043 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%', 'op': 'PROPAGATE'}, 'Node2') 2017-10-11 17:05:04,043 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,043 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'reqId': 1507741503995435, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,043 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} 2017-10-11 17:05:04,044 | DEBUG | propagator.py ( 214) | forward | Node1 forwarding request ('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435) to 0 replicas 2017-10-11 17:05:04,044 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,044 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} 2017-10-11 17:05:04,044 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} to its replicas since already forwarded 2017-10-11 17:05:04,044 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}}, 'senderClient': 'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%'} 2017-10-11 17:05:04,046 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} 2017-10-11 17:05:04,047 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741503995435, 'signature': '4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ', 'operation': {'type': '1', 'dest': '542MVr22zcHbVyGzaXmbT4'}} to its replicas since already forwarded 2017-10-11 17:05:04,050 | DEBUG | replica.py ( 667) | create3PCBatch | Node1:0 creating batch 2 for ledger 1 with state root b'\x0f\xd9V\xca_4\x81I\x98\r\xf0\xc7F%\x1a\xb2PM\xd92*O\xe6\xa5o\xc6!h\r\xb7U=' 2017-10-11 17:05:04,052 | DEBUG | replica.py ( 700) | create3PCBatch | Node1:0 created a PRE-PREPARE with 1 requests for ledger 1 2017-10-11 17:05:04,057 | DEBUG | replica.py (2193) | send | Node1:0 sending PrePrepare 2017-10-11 17:05:04,057 | TRACE | replica.py (2194) | send | Node1:0 sending PREPREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], 'instId': 0} 2017-10-11 17:05:04,057 | DEBUG | replica.py ( 614) | trackBatches | Node1:0 tracking batch for PREPREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], 'instId': 0} with state root b'\x0f\xd9V\xca_4\x81I\x98\r\xf0\xc7F%\x1a\xb2PM\xd92*O\xe6\xa5o\xc6!h\r\xb7U=' 2017-10-11 17:05:04,057 | DEBUG | node.py (2608) | send | Node1 sending message PREPREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:04,058 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:04,060 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741503995435]], 'instId': 1} 2017-10-11 17:05:04,060 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerId': 1, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'ppSeqNo': 2, 'viewNo': 0, 'discarded': 1, 'instId': 1, 'ppTime': 1507741504, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741503995435]], 'op': 'PREPREPARE'}, 'Node2') 2017-10-11 17:05:04,060 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741503995435]], 'instId': 1} 2017-10-11 17:05:04,061 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"ppTime":1507741504,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741503995435]]}' to Node4 2017-10-11 17:05:04,061 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"ppTime":1507741504,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741503995435]]}' to Node4 2017-10-11 17:05:04,061 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"ppTime":1507741504,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741503995435]]}' to Node2 2017-10-11 17:05:04,062 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"ppTime":1507741504,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741503995435]]}' to Node2 2017-10-11 17:05:04,062 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"ppTime":1507741504,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741503995435]]}' to Node3 2017-10-11 17:05:04,062 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"ppTime":1507741504,"ledgerId":1,"op":"PREPREPARE","txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741503995435]]}' to Node3 2017-10-11 17:05:04,063 | DEBUG | replica.py ( 823) | processPrePrepare | Node1:1 received PRE-PREPARE(0, 2) from Node2:1 at 2270589.52062704 2017-10-11 17:05:04,063 | DEBUG | replica.py ( 934) | doPrepare | Node1:1 Sending PREPARE(0, 2) at 2270589.521130833 2017-10-11 17:05:04,064 | DEBUG | replica.py (2193) | send | Node1:1 sending Prepare 2017-10-11 17:05:04,064 | TRACE | replica.py (2194) | send | Node1:1 sending PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} 2017-10-11 17:05:04,064 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} 2017-10-11 17:05:04,064 | DEBUG | replica.py ( 614) | trackBatches | Node1:1 tracking batch for PREPREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], 'instId': 1} with state root None 2017-10-11 17:05:04,064 | DEBUG | replica.py ( 842) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(0, 2) 2017-10-11 17:05:04,064 | DEBUG | node.py (2608) | send | Node1 sending message PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:04,065 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:04,065 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} 2017-10-11 17:05:04,065 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'op': 'PREPARE', 'instId': 1}, 'Node3') 2017-10-11 17:05:04,065 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} 2017-10-11 17:05:04,066 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":null,"viewNo":0,"ppTime":1507741504,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:05:04,066 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":null,"viewNo":0,"ppTime":1507741504,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:05:04,066 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":null,"viewNo":0,"ppTime":1507741504,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:05:04,066 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":null,"viewNo":0,"ppTime":1507741504,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:05:04,067 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":null,"viewNo":0,"ppTime":1507741504,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:05:04,067 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","stateRootHash":null,"viewNo":0,"ppTime":1507741504,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:05:04,068 | DEBUG | replica.py ( 953) | doCommit | Node1:1 Sending COMMIT(0, 2) at 2270589.525384478 2017-10-11 17:05:04,068 | DEBUG | replica.py (2193) | send | Node1:1 sending Commit 2017-10-11 17:05:04,068 | TRACE | replica.py (2194) | send | Node1:1 sending COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:04,068 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:05:04,068 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 2) 2017-10-11 17:05:04,068 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:04,069 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:04,071 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} 2017-10-11 17:05:04,071 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'stateRootHash': None, 'txnRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:05:04,071 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} 2017-10-11 17:05:04,071 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"op":"COMMIT","instId":1,"viewNo":0}' to Node4 2017-10-11 17:05:04,072 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"op":"COMMIT","instId":1,"viewNo":0}' to Node4 2017-10-11 17:05:04,072 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"op":"COMMIT","instId":1,"viewNo":0}' to Node2 2017-10-11 17:05:04,072 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"op":"COMMIT","instId":1,"viewNo":0}' to Node2 2017-10-11 17:05:04,072 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"op":"COMMIT","instId":1,"viewNo":0}' to Node3 2017-10-11 17:05:04,072 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"op":"COMMIT","instId":1,"viewNo":0}' to Node3 2017-10-11 17:05:04,074 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741504, 'instId': 1} 2017-10-11 17:05:04,074 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 2) 2017-10-11 17:05:04,075 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:04,076 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,084 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'op': 'PREPARE', 'instId': 0}, 'Node3') 2017-10-11 17:05:04,085 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,086 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,086 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 2) 2017-10-11 17:05:04,086 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:04,091 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:04,091 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'instId': 1, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:05:04,091 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:04,093 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 2) from Node3:1 2017-10-11 17:05:04,093 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:05:04,093 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 2) 2017-10-11 17:05:04,109 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 4 messages through listener 2017-10-11 17:05:04,111 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","op":"PREPARE","viewNo":0,"ppTime":1507741504,"stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","instId":0}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":2}'], 'signature': None} 2017-10-11 17:05:04,112 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","op":"PREPARE","viewNo":0,"ppTime":1507741504,"stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","instId":0}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":2}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:05:04,112 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","op":"PREPARE","viewNo":0,"ppTime":1507741504,"stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","instId":0}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":2}'], 'signature': None} 2017-10-11 17:05:04,113 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,113 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:05:04,114 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,115 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:04,115 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 2, 'instId': 1, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:05:04,116 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:04,116 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'messages': ['{"stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","ppSeqNo":2,"op":"PREPARE","instId":0,"ppTime":1507741504,"txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"}', '{"instId":1,"ppSeqNo":2,"op":"COMMIT","viewNo":0}'], 'signature': None} 2017-10-11 17:05:04,116 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","ppSeqNo":2,"op":"PREPARE","instId":0,"ppTime":1507741504,"txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"}', '{"instId":1,"ppSeqNo":2,"op":"COMMIT","viewNo":0}'], 'signature': None, 'op': 'BATCH'}, 'Node2') 2017-10-11 17:05:04,116 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"stateRootHash":"CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY","viewNo":0,"digest":"79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23","ppSeqNo":2,"op":"PREPARE","instId":0,"ppTime":1507741504,"txnRootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"}', '{"instId":1,"ppSeqNo":2,"op":"COMMIT","viewNo":0}'], 'signature': None} 2017-10-11 17:05:04,117 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,117 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0, 'op': 'PREPARE'}, 'Node2') 2017-10-11 17:05:04,117 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,117 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:04,117 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'op': 'COMMIT', 'viewNo': 0, 'instId': 1}, 'Node2') 2017-10-11 17:05:04,118 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} 2017-10-11 17:05:04,118 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:04,118 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'instId': 0, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:05:04,118 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:04,118 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:04,118 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 2, 'instId': 0, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:05:04,118 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:04,119 | DEBUG | replica.py ( 953) | doCommit | Node1:0 Sending COMMIT(0, 2) at 2270589.577141946 2017-10-11 17:05:04,119 | DEBUG | replica.py (2193) | send | Node1:0 sending Commit 2017-10-11 17:05:04,120 | TRACE | replica.py (2194) | send | Node1:0 sending COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:04,120 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:05:04,120 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 2) 2017-10-11 17:05:04,120 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 2, 'digest': '79dcf7fb12e16a9769b789d9cd302b8143ea89ba185e26c1cac9cc0be5558b23', 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ppTime': 1507741504, 'instId': 0} 2017-10-11 17:05:04,120 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 2) 2017-10-11 17:05:04,120 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 2) from Node3:0 2017-10-11 17:05:04,120 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:05:04,120 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 2) 2017-10-11 17:05:04,121 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 2) from Node4:0 2017-10-11 17:05:04,121 | TRACE | replica.py ( 925) | tryOrder | Node1:0 returning request to node 2017-10-11 17:05:04,121 | DEBUG | replica.py (1444) | doOrder | Node1:0 ordering COMMIT (0, 2) 2017-10-11 17:05:04,121 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:0 set last ordered as (0, 2) 2017-10-11 17:05:04,121 | DEBUG | replica.py (2193) | send | Node1:0 sending Ordered 2017-10-11 17:05:04,121 | TRACE | replica.py (2194) | send | Node1:0 sending ORDERED{'ppSeqNo': 2, 'txnRootHash': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'stateRootHash': 'CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY', 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], 'instId': 0} 2017-10-11 17:05:04,122 | INFO | replica.py (1483) | order_3pc_key | Node1:0 ordered batch request, view no 0, ppSeqNo 2, ledger 1, state root CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY, txn root DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], discarded [] 2017-10-11 17:05:04,122 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 2) 2017-10-11 17:05:04,122 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 2) from Node4:1 2017-10-11 17:05:04,122 | TRACE | replica.py ( 925) | tryOrder | Node1:1 returning request to node 2017-10-11 17:05:04,123 | DEBUG | replica.py (1444) | doOrder | Node1:1 ordering COMMIT (0, 2) 2017-10-11 17:05:04,123 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:1 set last ordered as (0, 2) 2017-10-11 17:05:04,123 | DEBUG | replica.py (2193) | send | Node1:1 sending Ordered 2017-10-11 17:05:04,123 | TRACE | replica.py (2194) | send | Node1:1 sending ORDERED{'ppSeqNo': 2, 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 1, 'ppTime': 1507741504, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], 'instId': 1} 2017-10-11 17:05:04,123 | INFO | replica.py (1483) | order_3pc_key | Node1:1 ordered batch request, view no 0, ppSeqNo 2, ledger 1, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)], discarded [] 2017-10-11 17:05:04,123 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 2) 2017-10-11 17:05:04,123 | TRACE | message_processor.py ( 29) | discard | Node1:1 discarding message COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 1} because already ordered 3 phase message 2017-10-11 17:05:04,123 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:04,124 | DEBUG | node.py (1847) | processOrdered | Node1 executing Ordered batch 0 2 of 1 requests 2017-10-11 17:05:04,124 | DEBUG | ledger.py ( 52) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:05:04,124 | DEBUG | node.py (2386) | sendReplyToClient | Node1 sending reply for ('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435) to client 2017-10-11 17:05:04,125 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"result":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741503995435,"type":"1","dest":"542MVr22zcHbVyGzaXmbT4","auditPath":["CFFMfapJTbk3rWENJa7Brjm5cVQTXaJE3P6UAGdyyZwC","Gi2HwXVBFyDVgs7rwus6Mvr4PTuRgvyN3tR5izEkMLcE"],"seqNo":13,"rootHash":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","signature":"4qkeA5A8cXU3cTWFSuYeqTLsK2D6Lqnjt7HxJAkkHLrV8jxsZbybuZBgTf6BMwSL7KCnjwduyXgz6ugSj3LZDqmQ","txnTime":1507741504},"op":"REPLY"}' to b'fUW0HqcUbaEUwY&4ITjkWTgST8Ypps&uh46af]S%' through listener socket 2017-10-11 17:05:04,125 | INFO | node.py (2290) | executeBatch | Node1 committed batch request, view no 0, ppSeqNo 2, ledger 1, state root CZrJjTQqN9QFaEpGiGRpkxiZG3FA43MbDKuVxiqvhDeY, txn root DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn, requests: [('V4SGRU86Z58d6TV7PBUe6f', 1507741503995435)] 2017-10-11 17:05:04,125 | DEBUG | node.py (2305) | executeBatch | Node1 storing 3PC key (0, 2) for ledger 1 range (13, 13) 2017-10-11 17:05:04,126 | TRACE | node.py (1858) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:05:04,127 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:04,127 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:05:04,128 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:04,128 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 2, 'op': 'COMMIT', 'viewNo': 0, 'instId': 0}, 'Node2') 2017-10-11 17:05:04,128 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} 2017-10-11 17:05:04,128 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"op":"COMMIT","instId":0,"viewNo":0}' to Node4 2017-10-11 17:05:04,129 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"op":"COMMIT","instId":0,"viewNo":0}' to Node4 2017-10-11 17:05:04,129 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"op":"COMMIT","instId":0,"viewNo":0}' to Node2 2017-10-11 17:05:04,129 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"op":"COMMIT","instId":0,"viewNo":0}' to Node2 2017-10-11 17:05:04,129 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":2,"op":"COMMIT","instId":0,"viewNo":0}' to Node3 2017-10-11 17:05:04,130 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":2,"op":"COMMIT","instId":0,"viewNo":0}' to Node3 2017-10-11 17:05:04,130 | TRACE | message_processor.py ( 29) | discard | Node1:0 discarding message COMMIT{'ppSeqNo': 2, 'viewNo': 0, 'instId': 0} because already ordered 3 phase message 2017-10-11 17:05:08,740 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 2 2017-10-11 17:05:08,740 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:05:08,740 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:05:08,741 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 11 to run in 60 seconds 2017-10-11 17:05:08,741 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 1 2017-10-11 17:05:08,741 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:05:08,741 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 2 to run in 60 seconds 2017-10-11 17:05:08,776 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 10 2017-10-11 17:05:08,776 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:05:08,776 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:05:08,776 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:05:08,776 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:08,777 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:05:08,777 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:05:08,777 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 12 to run in 10 seconds 2017-10-11 17:05:08,777 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 5 2017-10-11 17:05:08,778 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 13 to run in 60 seconds 2017-10-11 17:05:08,778 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action propose_view_change with id 3 2017-10-11 17:05:08,778 | DEBUG | throttler.py ( 31) | acquire | now: 2270594.235649649, len(actionsLog): 0 2017-10-11 17:05:08,778 | DEBUG | throttler.py ( 34) | acquire | after trim, len(actionsLog): 0 2017-10-11 17:05:08,778 | DEBUG | throttler.py ( 39) | acquire | len(actionsLog) was 0, after append, len(actionsLog): 1 2017-10-11 17:05:08,778 | INFO | node.py (2048) | sendInstanceChange | VIEW CHANGE: Node1 sending an instance change with view_no 1 since Primary of master protocol instance disconnected 2017-10-11 17:05:08,778 | INFO | node.py (2051) | sendInstanceChange | MONITORING: Node1 metrics for monitor: Node1 Monitor metrics:: None Delta: 0.4 Lambda: 60 Omega: 5 instances started: [2270534.197065212, 2270534.197352942] ordered request counts: {0: 2, 1: 2} ordered request durations: {0: 0.1591902836225927, 1: 0.15395257156342268} master request latencies: {} client avg request latencies: [{'V4SGRU86Z58d6TV7PBUe6f': (2, 0.07959514181129634)}, {'V4SGRU86Z58d6TV7PBUe6f': (2, 0.07697628578171134)}] throughput: {0: 12.563580857369331, 1: 12.991013918699469} master throughput: 12.563580857369331 total requests: 2 avg backup throughput: 12.991013918699469 master throughput ratio: 0.9670977905184995 2017-10-11 17:05:08,778 | DEBUG | node.py (2608) | send | Node1 sending message INSTANCE_CHANGE{'reason': 26, 'viewNo': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:05:08,778 | DEBUG | node.py (1958) | do_view_change_if_possible | Node1 has no quorum for view 1 2017-10-11 17:05:08,778 | INFO | node.py (2108) | propose_view_change | Node1 sent view change since was disconnected from primary for too long 2017-10-11 17:05:08,779 | DEBUG | node.py (1958) | do_view_change_if_possible | Node1 has no quorum for view 1 2017-10-11 17:05:08,779 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node4 2017-10-11 17:05:08,779 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node4 2017-10-11 17:05:08,779 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node2 2017-10-11 17:05:08,779 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node2 2017-10-11 17:05:08,780 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node3 2017-10-11 17:05:08,780 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node3 2017-10-11 17:05:08,804 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:05:08,804 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:05:08,804 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:05:08,805 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:05:18,789 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 12 2017-10-11 17:05:18,789 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:05:18,789 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:05:18,789 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:05:18,789 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:18,789 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:05:18,790 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:05:18,790 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 14 to run in 10 seconds 2017-10-11 17:05:23,811 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:05:23,811 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:05:23,811 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:05:23,811 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:05:28,795 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 14 2017-10-11 17:05:28,795 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:05:28,795 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:05:28,795 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:05:28,795 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:28,796 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:05:28,796 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:05:28,796 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 15 to run in 10 seconds 2017-10-11 17:05:38,805 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 15 2017-10-11 17:05:38,805 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:05:38,805 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:05:38,805 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:05:38,805 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:38,806 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:05:38,806 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:05:38,806 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 16 to run in 10 seconds 2017-10-11 17:05:38,819 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:05:38,819 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:05:38,819 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:05:38,819 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:05:48,814 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 16 2017-10-11 17:05:48,814 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:05:48,814 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:05:48,814 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:05:48,814 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:48,815 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:05:48,815 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:05:48,815 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 17 to run in 10 seconds 2017-10-11 17:05:53,829 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:05:53,829 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:05:53,829 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:05:53,829 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:05:58,826 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 17 2017-10-11 17:05:58,826 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:05:58,827 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:05:58,827 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:05:58,827 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:05:58,830 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:05:58,830 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:05:58,831 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 18 to run in 10 seconds 2017-10-11 17:06:08,743 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 11 2017-10-11 17:06:08,743 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:06:08,743 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:06:08,744 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 19 to run in 60 seconds 2017-10-11 17:06:08,744 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 2 2017-10-11 17:06:08,744 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:06:08,744 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 3 to run in 60 seconds 2017-10-11 17:06:08,780 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 13 2017-10-11 17:06:08,780 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 20 to run in 60 seconds 2017-10-11 17:06:08,830 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:06:08,830 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:06:08,830 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:06:08,831 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:06:08,842 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 18 2017-10-11 17:06:08,843 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:06:08,843 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:06:08,843 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:06:08,843 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:06:08,843 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:06:08,843 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:06:08,843 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 21 to run in 10 seconds 2017-10-11 17:06:18,846 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 21 2017-10-11 17:06:18,846 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:06:18,846 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:06:18,846 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:06:18,846 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:06:18,846 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:06:18,846 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:06:18,846 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 22 to run in 10 seconds 2017-10-11 17:06:23,835 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:06:23,835 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:06:23,836 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:06:23,836 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:06:28,851 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 22 2017-10-11 17:06:28,852 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:06:28,852 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:06:28,852 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:06:28,853 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:06:28,853 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:06:28,853 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:06:28,853 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 23 to run in 10 seconds 2017-10-11 17:06:32,824 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:06:32,824 | DEBUG | zstack.py ( 663) | handlePingPong | Node1C got ping from b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:06:32,824 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'po' to b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' through listener socket 2017-10-11 17:06:32,825 | DEBUG | zstack.py ( 645) | sendPingPong | Node1C ponged b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:06:32,851 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:06:32,852 | TRACE | node.py (1388) | validateClientMsg | Node1C received CLIENT message: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:06:32,852 | DEBUG | node.py (1434) | processClientInBox | Node1C processing b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' request LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} 2017-10-11 17:06:32,852 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'txnSeqNo': 4, 'ledgerId': 0, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'viewNo': None} from b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:06:32,852 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:06:32,853 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"txnSeqNo":4,"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"op":"LEDGER_STATUS"}' to b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' through listener socket 2017-10-11 17:06:32,853 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:06:32,853 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:06:38,844 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:06:38,844 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:06:38,844 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:06:38,845 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:06:38,856 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 23 2017-10-11 17:06:38,856 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:06:38,856 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:06:38,857 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9670977905184995 is acceptable. 2017-10-11 17:06:38,857 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:06:38,857 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:06:38,857 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:06:38,857 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 24 to run in 10 seconds 2017-10-11 17:06:47,575 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:06:47,578 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on request 1507741607555952 2017-10-11 17:06:47,578 | TRACE | node.py (1388) | validateClientMsg | Node1C received CLIENT message: SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} 2017-10-11 17:06:47,578 | DEBUG | node.py (1434) | processClientInBox | Node1C processing b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} 2017-10-11 17:06:47,578 | DEBUG | node.py (1708) | processRequest | Node1 received client request: SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} from b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:06:47,591 | TRACE | propagator.py ( 165) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} 2017-10-11 17:06:47,592 | INFO | propagator.py ( 148) | propagate | Node1 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952) from client b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:06:47,592 | DEBUG | node.py (2608) | send | Node1 sending message PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:06:47,592 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} to its replicas since not finalised 2017-10-11 17:06:47,592 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741607555952,"op":"REQACK"}' to b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' through listener socket 2017-10-11 17:06:47,592 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741607555952,"signature":"3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i","operation":{"version":"1.1.40","force":false,"name":"upgrade40","timeout":10,"type":"109","justification":null,"reinstall":false,"schedule":{"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T17:10:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T17:20:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T17:15:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T17:25:00.258870+00:00"},"action":"start","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f"}},"senderClient":"+>]}>OKfYQDWB?@taDN+?+{nc\\/ZoKRv9iH*\\/^F$a","op":"PROPAGATE"}' to Node4 2017-10-11 17:06:47,593 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741607555952,"signature":"3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i","operation":{"version":"1.1.40","force":false,"name":"upgrade40","timeout":10,"type":"109","justification":null,"reinstall":false,"schedule":{"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T17:10:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T17:20:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T17:15:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T17:25:00.258870+00:00"},"action":"start","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f"}},"senderClient":"+>]}>OKfYQDWB?@taDN+?+{nc\\/ZoKRv9iH*\\/^F$a","op":"PROPAGATE"}' to Node4 2017-10-11 17:06:47,593 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741607555952,"signature":"3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i","operation":{"version":"1.1.40","force":false,"name":"upgrade40","timeout":10,"type":"109","justification":null,"reinstall":false,"schedule":{"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T17:10:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T17:20:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T17:15:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T17:25:00.258870+00:00"},"action":"start","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f"}},"senderClient":"+>]}>OKfYQDWB?@taDN+?+{nc\\/ZoKRv9iH*\\/^F$a","op":"PROPAGATE"}' to Node2 2017-10-11 17:06:47,593 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741607555952,"signature":"3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i","operation":{"version":"1.1.40","force":false,"name":"upgrade40","timeout":10,"type":"109","justification":null,"reinstall":false,"schedule":{"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T17:10:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T17:20:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T17:15:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T17:25:00.258870+00:00"},"action":"start","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f"}},"senderClient":"+>]}>OKfYQDWB?@taDN+?+{nc\\/ZoKRv9iH*\\/^F$a","op":"PROPAGATE"}' to Node2 2017-10-11 17:06:47,593 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741607555952,"signature":"3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i","operation":{"version":"1.1.40","force":false,"name":"upgrade40","timeout":10,"type":"109","justification":null,"reinstall":false,"schedule":{"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T17:10:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T17:20:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T17:15:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T17:25:00.258870+00:00"},"action":"start","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f"}},"senderClient":"+>]}>OKfYQDWB?@taDN+?+{nc\\/ZoKRv9iH*\\/^F$a","op":"PROPAGATE"}' to Node3 2017-10-11 17:06:47,593 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"V4SGRU86Z58d6TV7PBUe6f","reqId":1507741607555952,"signature":"3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i","operation":{"version":"1.1.40","force":false,"name":"upgrade40","timeout":10,"type":"109","justification":null,"reinstall":false,"schedule":{"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T17:10:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T17:20:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T17:15:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T17:25:00.258870+00:00"},"action":"start","sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f"}},"senderClient":"+>]}>OKfYQDWB?@taDN+?+{nc\\/ZoKRv9iH*\\/^F$a","op":"PROPAGATE"}' to Node3 2017-10-11 17:06:47,594 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:06:47,598 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741607555952 2017-10-11 17:06:47,598 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,599 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a', 'op': 'PROPAGATE'}, 'Node4') 2017-10-11 17:06:47,600 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,604 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741607555952 2017-10-11 17:06:47,604 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'request': {'reqId': 1507741607555952, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,605 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'reqId': 1507741607555952, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a', 'op': 'PROPAGATE'}, 'Node3') 2017-10-11 17:06:47,605 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'reqId': 1507741607555952, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,605 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,605 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} 2017-10-11 17:06:47,605 | DEBUG | propagator.py ( 214) | forward | Node1 forwarding request ('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952) to 0 replicas 2017-10-11 17:06:47,606 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'reqId': 1507741607555952, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,606 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} 2017-10-11 17:06:47,606 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'timeout': 10, 'type': '109', 'justification': None, 'reinstall': False, 'schedule': {'4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00'}, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} to its replicas since already forwarded 2017-10-11 17:06:47,607 | DEBUG | replica.py ( 667) | create3PCBatch | Node1:0 creating batch 3 for ledger 2 with state root b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f' 2017-10-11 17:06:47,608 | DEBUG | replica.py ( 700) | create3PCBatch | Node1:0 created a PRE-PREPARE with 1 requests for ledger 2 2017-10-11 17:06:47,608 | DEBUG | replica.py (2193) | send | Node1:0 sending PrePrepare 2017-10-11 17:06:47,609 | TRACE | replica.py (2194) | send | Node1:0 sending PREPREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], 'instId': 0} 2017-10-11 17:06:47,609 | DEBUG | replica.py ( 614) | trackBatches | Node1:0 tracking batch for PREPREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], 'instId': 0} with state root b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f' 2017-10-11 17:06:47,609 | DEBUG | node.py (2608) | send | Node1 sending message PREPREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:06:47,609 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:06:47,610 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated V4SGRU86Z58d6TV7PBUe6f signature on propagate request 1507741607555952 2017-10-11 17:06:47,610 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,611 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a', 'op': 'PROPAGATE'}, 'Node2') 2017-10-11 17:06:47,611 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,611 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}}, 'senderClient': '+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a'} 2017-10-11 17:06:47,611 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} 2017-10-11 17:06:47,612 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'reqId': 1507741607555952, 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'operation': {'version': '1.1.40', 'force': False, 'name': 'upgrade40', 'type': '109', 'justification': None, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'reinstall': False, 'timeout': 10, 'action': 'start', 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'}} to its replicas since already forwarded 2017-10-11 17:06:47,612 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741607,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741607555952]]}' to Node4 2017-10-11 17:06:47,612 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741607,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741607555952]]}' to Node4 2017-10-11 17:06:47,613 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741607,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741607555952]]}' to Node2 2017-10-11 17:06:47,613 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741607,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741607555952]]}' to Node2 2017-10-11 17:06:47,613 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741607,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741607555952]]}' to Node3 2017-10-11 17:06:47,613 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741607,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","instId":0,"discarded":1,"reqIdr":[["V4SGRU86Z58d6TV7PBUe6f",1507741607555952]]}' to Node3 2017-10-11 17:06:47,628 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:06:47,628 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741607555952]], 'instId': 1} 2017-10-11 17:06:47,628 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerId': 2, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'ppSeqNo': 3, 'viewNo': 0, 'discarded': 1, 'instId': 1, 'ppTime': 1507741607, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741607555952]], 'op': 'PREPREPARE'}, 'Node2') 2017-10-11 17:06:47,629 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'discarded': 1, 'reqIdr': [['V4SGRU86Z58d6TV7PBUe6f', 1507741607555952]], 'instId': 1} 2017-10-11 17:06:47,629 | DEBUG | replica.py ( 823) | processPrePrepare | Node1:1 received PRE-PREPARE(0, 3) from Node2:1 at 2270693.087047846 2017-10-11 17:06:47,630 | DEBUG | replica.py ( 934) | doPrepare | Node1:1 Sending PREPARE(0, 3) at 2270693.087462651 2017-10-11 17:06:47,630 | DEBUG | replica.py (2193) | send | Node1:1 sending Prepare 2017-10-11 17:06:47,630 | TRACE | replica.py (2194) | send | Node1:1 sending PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} 2017-10-11 17:06:47,630 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} 2017-10-11 17:06:47,630 | DEBUG | replica.py ( 614) | trackBatches | Node1:1 tracking batch for PREPREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'discarded': 1, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], 'instId': 1} with state root None 2017-10-11 17:06:47,630 | DEBUG | replica.py ( 842) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(0, 3) 2017-10-11 17:06:47,630 | DEBUG | node.py (2608) | send | Node1 sending message PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:06:47,642 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:06:47,643 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","op":"PREPARE","viewNo":0,"ppTime":1507741607,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"txnRootHash":null,"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","op":"PREPARE","viewNo":0,"ppTime":1507741607,"stateRootHash":null,"instId":1}'], 'signature': None} 2017-10-11 17:06:47,643 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","op":"PREPARE","viewNo":0,"ppTime":1507741607,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"txnRootHash":null,"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","op":"PREPARE","viewNo":0,"ppTime":1507741607,"stateRootHash":null,"instId":1}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:06:47,643 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7","ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","op":"PREPARE","viewNo":0,"ppTime":1507741607,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"txnRootHash":null,"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","op":"PREPARE","viewNo":0,"ppTime":1507741607,"stateRootHash":null,"instId":1}'], 'signature': None} 2017-10-11 17:06:47,644 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,644 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:06:47,644 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,644 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} 2017-10-11 17:06:47,644 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'stateRootHash': None, 'txnRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:06:47,644 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} 2017-10-11 17:06:47,645 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":null,"viewNo":0,"ppTime":1507741607,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:06:47,645 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":null,"viewNo":0,"ppTime":1507741607,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:06:47,645 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":null,"viewNo":0,"ppTime":1507741607,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:06:47,645 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":null,"viewNo":0,"ppTime":1507741607,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:06:47,645 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":null,"viewNo":0,"ppTime":1507741607,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:06:47,645 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","stateRootHash":null,"viewNo":0,"ppTime":1507741607,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:06:47,646 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,646 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 3) 2017-10-11 17:06:47,646 | DEBUG | replica.py ( 953) | doCommit | Node1:1 Sending COMMIT(0, 3) at 2270693.10416308 2017-10-11 17:06:47,652 | DEBUG | replica.py (2193) | send | Node1:1 sending Commit 2017-10-11 17:06:47,652 | TRACE | replica.py (2194) | send | Node1:1 sending COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} 2017-10-11 17:06:47,652 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:06:47,652 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 3) 2017-10-11 17:06:47,652 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:06:47,653 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:06:47,653 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,653 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0, 'op': 'PREPARE'}, 'Node2') 2017-10-11 17:06:47,654 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,654 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} 2017-10-11 17:06:47,654 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'op': 'COMMIT', 'viewNo': 0, 'instId': 0}, 'Node2') 2017-10-11 17:06:47,654 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} 2017-10-11 17:06:47,654 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"op":"COMMIT","instId":1,"viewNo":0}' to Node4 2017-10-11 17:06:47,655 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"op":"COMMIT","instId":1,"viewNo":0}' to Node4 2017-10-11 17:06:47,655 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"op":"COMMIT","instId":1,"viewNo":0}' to Node2 2017-10-11 17:06:47,655 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"op":"COMMIT","instId":1,"viewNo":0}' to Node2 2017-10-11 17:06:47,655 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"op":"COMMIT","instId":1,"viewNo":0}' to Node3 2017-10-11 17:06:47,655 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"op":"COMMIT","instId":1,"viewNo":0}' to Node3 2017-10-11 17:06:47,656 | DEBUG | replica.py ( 953) | doCommit | Node1:0 Sending COMMIT(0, 3) at 2270693.113716047 2017-10-11 17:06:47,656 | DEBUG | replica.py (2193) | send | Node1:0 sending Commit 2017-10-11 17:06:47,656 | TRACE | replica.py (2194) | send | Node1:0 sending COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} 2017-10-11 17:06:47,656 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:06:47,656 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 3) 2017-10-11 17:06:47,656 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 3) from Node2:0 2017-10-11 17:06:47,657 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:06:47,657 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 3) 2017-10-11 17:06:47,657 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:06:47,664 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"op":"COMMIT","instId":0,"viewNo":0}' to Node4 2017-10-11 17:06:47,664 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"op":"COMMIT","instId":0,"viewNo":0}' to Node4 2017-10-11 17:06:47,664 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"op":"COMMIT","instId":0,"viewNo":0}' to Node2 2017-10-11 17:06:47,666 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"op":"COMMIT","instId":0,"viewNo":0}' to Node2 2017-10-11 17:06:47,666 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":3,"op":"COMMIT","instId":0,"viewNo":0}' to Node3 2017-10-11 17:06:47,668 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":3,"op":"COMMIT","instId":0,"viewNo":0}' to Node3 2017-10-11 17:06:47,671 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:06:47,672 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'messages': ['{"ppSeqNo":3,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","ppTime":1507741607,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7"}', '{"viewNo":0,"instId":0,"op":"COMMIT","ppSeqNo":3}', '{"ppSeqNo":3,"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"ppTime":1507741607,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","txnRootHash":null}', '{"viewNo":0,"instId":1,"op":"COMMIT","ppSeqNo":3}'], 'signature': None} 2017-10-11 17:06:47,673 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"ppSeqNo":3,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","ppTime":1507741607,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7"}', '{"viewNo":0,"instId":0,"op":"COMMIT","ppSeqNo":3}', '{"ppSeqNo":3,"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"ppTime":1507741607,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","txnRootHash":null}', '{"viewNo":0,"instId":1,"op":"COMMIT","ppSeqNo":3}'], 'signature': None, 'op': 'BATCH'}, 'Node3') 2017-10-11 17:06:47,673 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"ppSeqNo":3,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","ppTime":1507741607,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","txnRootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7"}', '{"viewNo":0,"instId":0,"op":"COMMIT","ppSeqNo":3}', '{"ppSeqNo":3,"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"ppTime":1507741607,"digest":"e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c","txnRootHash":null}', '{"viewNo":0,"instId":1,"op":"COMMIT","ppSeqNo":3}'], 'signature': None} 2017-10-11 17:06:47,674 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,674 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'op': 'PREPARE', 'instId': 0}, 'Node3') 2017-10-11 17:06:47,674 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,674 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} 2017-10-11 17:06:47,674 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'instId': 0, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:06:47,674 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} 2017-10-11 17:06:47,674 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} 2017-10-11 17:06:47,675 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'op': 'PREPARE', 'instId': 1}, 'Node3') 2017-10-11 17:06:47,675 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} 2017-10-11 17:06:47,675 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} 2017-10-11 17:06:47,675 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'instId': 1, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:06:47,675 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} 2017-10-11 17:06:47,675 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"op":"COMMIT","instId":0,"viewNo":0,"ppSeqNo":3}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":3}'], 'signature': None} 2017-10-11 17:06:47,675 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"op":"COMMIT","instId":0,"viewNo":0,"ppSeqNo":3}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":3}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:06:47,675 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"op":"COMMIT","instId":0,"viewNo":0,"ppSeqNo":3}', '{"op":"COMMIT","instId":1,"viewNo":0,"ppSeqNo":3}'], 'signature': None} 2017-10-11 17:06:47,676 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} 2017-10-11 17:06:47,676 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 3, 'instId': 0, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:06:47,676 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} 2017-10-11 17:06:47,676 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} 2017-10-11 17:06:47,676 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 3, 'instId': 1, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:06:47,676 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} 2017-10-11 17:06:47,677 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741607, 'instId': 0} 2017-10-11 17:06:47,677 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 3) 2017-10-11 17:06:47,678 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 3) from Node3:0 2017-10-11 17:06:47,678 | TRACE | replica.py ( 925) | tryOrder | Node1:0 returning request to node 2017-10-11 17:06:47,678 | DEBUG | replica.py (1444) | doOrder | Node1:0 ordering COMMIT (0, 3) 2017-10-11 17:06:47,678 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:0 set last ordered as (0, 3) 2017-10-11 17:06:47,679 | DEBUG | replica.py (2193) | send | Node1:0 sending Ordered 2017-10-11 17:06:47,679 | TRACE | replica.py (2194) | send | Node1:0 sending ORDERED{'ppSeqNo': 3, 'txnRootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], 'instId': 0} 2017-10-11 17:06:47,679 | INFO | replica.py (1483) | order_3pc_key | Node1:0 ordered batch request, view no 0, ppSeqNo 3, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], discarded [] 2017-10-11 17:06:47,679 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 3) 2017-10-11 17:06:47,679 | TRACE | message_processor.py ( 29) | discard | Node1:0 discarding message COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 0} because already ordered 3 phase message 2017-10-11 17:06:47,680 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 3, 'digest': 'e304c8fbb37d600e25530ff017ff312ad4eba587158b5c57a238d2a92ca8bb6c', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741607, 'instId': 1} 2017-10-11 17:06:47,680 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 3) 2017-10-11 17:06:47,680 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 3) from Node3:1 2017-10-11 17:06:47,680 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:06:47,680 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 3) 2017-10-11 17:06:47,681 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 3) from Node4:1 2017-10-11 17:06:47,681 | TRACE | replica.py ( 925) | tryOrder | Node1:1 returning request to node 2017-10-11 17:06:47,681 | DEBUG | replica.py (1444) | doOrder | Node1:1 ordering COMMIT (0, 3) 2017-10-11 17:06:47,681 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:1 set last ordered as (0, 3) 2017-10-11 17:06:47,681 | DEBUG | replica.py (2193) | send | Node1:1 sending Ordered 2017-10-11 17:06:47,681 | TRACE | replica.py (2194) | send | Node1:1 sending ORDERED{'ppSeqNo': 3, 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741607, 'reqIdr': [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], 'instId': 1} 2017-10-11 17:06:47,681 | INFO | replica.py (1483) | order_3pc_key | Node1:1 ordered batch request, view no 0, ppSeqNo 3, ledger 2, state root None, txn root None, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)], discarded [] 2017-10-11 17:06:47,682 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 3) 2017-10-11 17:06:47,682 | DEBUG | node.py (1847) | processOrdered | Node1 executing Ordered batch 0 3 of 1 requests 2017-10-11 17:06:47,682 | DEBUG | ledger.py ( 52) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:06:47,682 | INFO | upgrader.py ( 292) | handleUpgradeTxn | Node 'Node1' handles upgrade txn {'reqId': 1507741607555952, 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'justification': None, 'reinstall': False, 'action': 'start', 'signature': '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i', 'timeout': 10, 'sha256': 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f', 'force': False, 'txnTime': 1507741607, 'version': '1.1.40', 'auditPath': ['AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ'], 'name': 'upgrade40', 'type': '109', 'seqNo': 2, 'schedule': {'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv': '2017-10-11T17:10:00.258870+00:00', 'DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya': '2017-10-11T17:20:00.258870+00:00', '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb': '2017-10-11T17:15:00.258870+00:00', '4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA': '2017-10-11T17:25:00.258870+00:00'}, 'rootHash': '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7'} 2017-10-11 17:06:47,683 | INFO | upgrader.py ( 321) | handleUpgradeTxn | Node 'Node1' schedules upgrade to 1.1.40 2017-10-11 17:06:47,683 | INFO | upgrader.py ( 359) | _scheduleUpgrade | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv's upgrader processing upgrade for version 1.1.40 2017-10-11 17:06:47,684 | TRACE | has_action_queue.py ( 36) | _schedule | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv scheduling action partial(_callUpgradeAgent) with id 1 to run in 192 seconds 2017-10-11 17:06:47,684 | DEBUG | node.py (2386) | sendReplyToClient | Node1 sending reply for ('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952) to client 2017-10-11 17:06:47,684 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"result":{"reqId":1507741607555952,"identifier":"V4SGRU86Z58d6TV7PBUe6f","justification":null,"reinstall":false,"action":"start","signature":"3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i","timeout":10,"sha256":"ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f","force":false,"txnTime":1507741607,"version":"1.1.40","auditPath":["AbMRpp8BeBz1f2CzFcoode4pTaGV2d4rv32XCRVMY4pZ"],"name":"upgrade40","type":"109","seqNo":2,"schedule":{"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv":"2017-10-11T17:10:00.258870+00:00","DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya":"2017-10-11T17:20:00.258870+00:00","8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb":"2017-10-11T17:15:00.258870+00:00","4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA":"2017-10-11T17:25:00.258870+00:00"},"rootHash":"4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7"},"op":"REPLY"}' to b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' through listener socket 2017-10-11 17:06:47,684 | INFO | node.py (2290) | executeBatch | Node1 committed batch request, view no 0, ppSeqNo 3, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7, requests: [('V4SGRU86Z58d6TV7PBUe6f', 1507741607555952)] 2017-10-11 17:06:47,684 | DEBUG | node.py (2305) | executeBatch | Node1 storing 3PC key (0, 3) for ledger 2 range (2, 2) 2017-10-11 17:06:47,685 | TRACE | node.py (1858) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:06:47,685 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:06:47,685 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:06:47,686 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} 2017-10-11 17:06:47,686 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 3, 'op': 'COMMIT', 'viewNo': 0, 'instId': 1}, 'Node2') 2017-10-11 17:06:47,686 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} 2017-10-11 17:06:47,687 | TRACE | message_processor.py ( 29) | discard | Node1:1 discarding message COMMIT{'ppSeqNo': 3, 'viewNo': 0, 'instId': 1} because already ordered 3 phase message 2017-10-11 17:06:48,861 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 24 2017-10-11 17:06:48,861 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:06:48,862 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:06:48,862 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:06:48,862 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:06:48,862 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:06:48,862 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:06:48,862 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 25 to run in 10 seconds 2017-10-11 17:06:53,851 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:06:53,852 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:06:53,852 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:06:53,852 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:06:58,873 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 25 2017-10-11 17:06:58,873 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:06:58,873 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:06:58,873 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:06:58,873 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:06:58,873 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:06:58,873 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:06:58,874 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 26 to run in 10 seconds 2017-10-11 17:07:08,750 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 19 2017-10-11 17:07:08,750 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:07:08,750 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:07:08,750 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 27 to run in 60 seconds 2017-10-11 17:07:08,750 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 3 2017-10-11 17:07:08,750 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:07:08,750 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 4 to run in 60 seconds 2017-10-11 17:07:08,786 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 20 2017-10-11 17:07:08,787 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 28 to run in 60 seconds 2017-10-11 17:07:08,859 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:07:08,859 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:07:08,859 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:07:08,860 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:07:08,882 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 26 2017-10-11 17:07:08,883 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:07:08,883 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:07:08,883 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:07:08,883 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:07:08,883 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:07:08,883 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:07:08,883 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 29 to run in 10 seconds 2017-10-11 17:07:18,887 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 29 2017-10-11 17:07:18,887 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:07:18,887 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:07:18,887 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:07:18,887 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:07:18,888 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:07:18,888 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:07:18,888 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 30 to run in 10 seconds 2017-10-11 17:07:23,859 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:07:23,860 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:07:23,860 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:07:23,860 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:07:28,896 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 30 2017-10-11 17:07:28,896 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:07:28,896 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:07:28,896 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:07:28,896 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:07:28,897 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:07:28,897 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:07:28,897 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 31 to run in 10 seconds 2017-10-11 17:07:38,867 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:07:38,867 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:07:38,867 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:07:38,867 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:07:38,902 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 31 2017-10-11 17:07:38,903 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:07:38,903 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:07:38,903 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:07:38,903 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:07:38,903 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:07:38,903 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:07:38,903 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 32 to run in 10 seconds 2017-10-11 17:07:48,911 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 32 2017-10-11 17:07:48,911 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:07:48,911 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:07:48,911 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:07:48,912 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:07:48,912 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:07:48,912 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:07:48,912 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 33 to run in 10 seconds 2017-10-11 17:07:53,874 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:07:53,875 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:07:53,875 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:07:53,875 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:07:58,915 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 33 2017-10-11 17:07:58,915 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:07:58,915 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:07:58,915 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:07:58,915 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:07:58,916 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:07:58,916 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:07:58,916 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 34 to run in 10 seconds 2017-10-11 17:08:08,760 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 27 2017-10-11 17:08:08,761 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:08:08,761 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:08:08,761 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 35 to run in 60 seconds 2017-10-11 17:08:08,761 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 4 2017-10-11 17:08:08,761 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:08:08,761 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 5 to run in 60 seconds 2017-10-11 17:08:08,796 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 28 2017-10-11 17:08:08,797 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 36 to run in 60 seconds 2017-10-11 17:08:08,879 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:08:08,879 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:08:08,879 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:08:08,880 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:08:08,927 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 34 2017-10-11 17:08:08,927 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:08:08,927 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:08:08,927 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:08:08,927 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:08:08,927 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:08:08,927 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:08:08,927 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 37 to run in 10 seconds 2017-10-11 17:08:18,934 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 37 2017-10-11 17:08:18,935 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:08:18,935 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:08:18,935 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:08:18,935 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:08:18,935 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:08:18,935 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:08:18,935 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 38 to run in 10 seconds 2017-10-11 17:08:23,889 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:08:23,889 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:08:23,889 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:08:23,890 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:08:28,943 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 38 2017-10-11 17:08:28,943 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:08:28,943 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:08:28,943 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:08:28,943 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:08:28,943 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:08:28,943 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:08:28,943 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 39 to run in 10 seconds 2017-10-11 17:08:38,899 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:08:38,899 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:08:38,900 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:08:38,900 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:08:38,946 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 39 2017-10-11 17:08:38,946 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:08:38,946 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:08:38,946 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:08:38,946 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:08:38,946 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:08:38,946 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:08:38,946 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 40 to run in 10 seconds 2017-10-11 17:08:48,953 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 40 2017-10-11 17:08:48,953 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:08:48,953 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:08:48,953 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:08:48,953 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:08:48,953 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:08:48,953 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:08:48,953 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 41 to run in 10 seconds 2017-10-11 17:08:53,910 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:08:53,910 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:08:53,910 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:08:53,911 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:08:58,959 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 41 2017-10-11 17:08:58,960 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:08:58,960 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:08:58,960 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:08:58,960 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:08:58,960 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:08:58,960 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:08:58,960 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 42 to run in 10 seconds 2017-10-11 17:09:08,771 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 35 2017-10-11 17:09:08,771 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:09:08,771 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:09:08,771 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 43 to run in 60 seconds 2017-10-11 17:09:08,771 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 5 2017-10-11 17:09:08,772 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:09:08,772 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 6 to run in 60 seconds 2017-10-11 17:09:08,807 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 36 2017-10-11 17:09:08,808 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 44 to run in 60 seconds 2017-10-11 17:09:08,915 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:09:08,915 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:09:08,915 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:09:08,916 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:09:08,962 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 42 2017-10-11 17:09:08,962 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:09:08,962 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:09:08,962 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:09:08,963 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:09:08,963 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:09:08,963 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:09:08,963 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 45 to run in 10 seconds 2017-10-11 17:09:18,974 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 45 2017-10-11 17:09:18,974 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:09:18,974 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:09:18,975 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:09:18,975 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:09:18,975 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:09:18,975 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:09:18,975 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 46 to run in 10 seconds 2017-10-11 17:09:23,917 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:09:23,917 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:09:23,917 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:09:23,917 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:09:28,982 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 46 2017-10-11 17:09:28,982 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:09:28,983 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:09:28,983 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:09:28,983 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:09:28,983 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:09:28,983 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:09:28,983 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 47 to run in 10 seconds 2017-10-11 17:09:38,923 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:09:38,924 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:09:38,924 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:09:38,924 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:09:38,994 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 47 2017-10-11 17:09:38,994 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:09:38,994 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:09:38,994 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:09:38,994 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:09:38,994 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:09:38,994 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:09:38,995 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 48 to run in 10 seconds 2017-10-11 17:09:48,997 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 48 2017-10-11 17:09:48,997 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:09:48,998 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:09:48,998 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:09:48,998 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:09:48,998 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:09:48,998 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:09:48,998 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 49 to run in 10 seconds 2017-10-11 17:09:53,928 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:09:53,929 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:09:53,929 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:09:53,929 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:09:59,003 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 49 2017-10-11 17:09:59,003 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:09:59,003 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:09:59,003 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 0.9786776771586602 is acceptable. 2017-10-11 17:09:59,003 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:09:59,004 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:09:59,004 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:09:59,004 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 50 to run in 10 seconds 2017-10-11 17:09:59,688 | TRACE | has_action_queue.py ( 66) | _serviceActions | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv running action partial(_callUpgradeAgent) with id 1 2017-10-11 17:09:59,688 | INFO | upgrader.py ( 421) | _callUpgradeAgent | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv's upgrader calling agent for upgrade 2017-10-11 17:09:59,688 | INFO | node.py ( 254) | notify_upgrade_start | Node1 is about to be upgraded, sending NODE_UPGRADE 2017-10-11 17:09:59,689 | DEBUG | node.py (2608) | send | Node1 sending message Request: {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:09:59,689 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}}' to Node4 2017-10-11 17:09:59,690 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}}' to Node4 2017-10-11 17:09:59,690 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}}' to Node2 2017-10-11 17:09:59,690 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}}' to Node2 2017-10-11 17:09:59,690 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}}' to Node3 2017-10-11 17:09:59,691 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}}' to Node3 2017-10-11 17:09:59,691 | INFO | upgrader.py ( 433) | _sendUpdateRequest | Sending message to control tool: {"version": "1.1.40"} 2017-10-11 17:09:59,692 | INFO | upgrader.py ( 452) | _sendUpdateRequest | Waiting 10 minutes for upgrade to be performed 2017-10-11 17:09:59,692 | TRACE | has_action_queue.py ( 36) | _schedule | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv scheduling action partial(_declareTimeoutExceeded) with id 2 to run in 600 seconds 2017-10-11 17:09:59,703 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:09:59,704 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv signature on propagate request 1507741799688892 2017-10-11 17:09:59,704 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,704 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1', 'op': 'PROPAGATE'}, 'Node2') 2017-10-11 17:09:59,704 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,704 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,705 | TRACE | propagator.py ( 165) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}} 2017-10-11 17:09:59,705 | INFO | propagator.py ( 148) | propagate | Node1 propagating request ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892) from client Node1 2017-10-11 17:09:59,705 | DEBUG | node.py (2608) | send | Node1 sending message PROPAGATE{'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1'} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:09:59,705 | DEBUG | propagator.py ( 214) | forward | Node1 forwarding request ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892) to 0 replicas 2017-10-11 17:09:59,706 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}},"senderClient":"Node1","op":"PROPAGATE"}' to Node4 2017-10-11 17:09:59,706 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}},"senderClient":"Node1","op":"PROPAGATE"}' to Node4 2017-10-11 17:09:59,706 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}},"senderClient":"Node1","op":"PROPAGATE"}' to Node2 2017-10-11 17:09:59,706 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}},"senderClient":"Node1","op":"PROPAGATE"}' to Node2 2017-10-11 17:09:59,707 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}},"senderClient":"Node1","op":"PROPAGATE"}' to Node3 2017-10-11 17:09:59,707 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","reqId":1507741799688892,"signature":"3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL","operation":{"data":{"version":"1.1.40","action":"in_progress"},"signature":"JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw","type":"110"}},"senderClient":"Node1","op":"PROPAGATE"}' to Node3 2017-10-11 17:09:59,707 | DEBUG | replica.py ( 667) | create3PCBatch | Node1:0 creating batch 4 for ledger 2 with state root b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f' 2017-10-11 17:09:59,708 | DEBUG | replica.py ( 700) | create3PCBatch | Node1:0 created a PRE-PREPARE with 1 requests for ledger 2 2017-10-11 17:09:59,708 | DEBUG | replica.py (2193) | send | Node1:0 sending PrePrepare 2017-10-11 17:09:59,708 | TRACE | replica.py (2194) | send | Node1:0 sending PREPREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'discarded': 1, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], 'instId': 0} 2017-10-11 17:09:59,708 | DEBUG | replica.py ( 614) | trackBatches | Node1:0 tracking batch for PREPREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'discarded': 1, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], 'instId': 0} with state root b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f' 2017-10-11 17:09:59,708 | DEBUG | node.py (2608) | send | Node1 sending message PREPREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'discarded': 1, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:09:59,709 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:09:59,710 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv signature on propagate request 1507741799688892 2017-10-11 17:09:59,710 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'request': {'reqId': 1507741799688892, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,711 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'reqId': 1507741799688892, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1', 'op': 'PROPAGATE'}, 'Node3') 2017-10-11 17:09:59,711 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'reqId': 1507741799688892, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,711 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'reqId': 1507741799688892, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,712 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}} 2017-10-11 17:09:59,712 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'type': '110'}} to its replicas since already forwarded 2017-10-11 17:09:59,712 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741799,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","instId":0,"discarded":1,"reqIdr":[["Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv",1507741799688892]]}' to Node4 2017-10-11 17:09:59,712 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741799,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","instId":0,"discarded":1,"reqIdr":[["Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv",1507741799688892]]}' to Node4 2017-10-11 17:09:59,713 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741799,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","instId":0,"discarded":1,"reqIdr":[["Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv",1507741799688892]]}' to Node2 2017-10-11 17:09:59,713 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741799,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","instId":0,"discarded":1,"reqIdr":[["Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv",1507741799688892]]}' to Node2 2017-10-11 17:09:59,713 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741799,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","instId":0,"discarded":1,"reqIdr":[["Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv",1507741799688892]]}' to Node3 2017-10-11 17:09:59,713 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","viewNo":0,"ppTime":1507741799,"ledgerId":2,"op":"PREPREPARE","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","instId":0,"discarded":1,"reqIdr":[["Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv",1507741799688892]]}' to Node3 2017-10-11 17:09:59,726 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:09:59,727 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'discarded': 1, 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892]], 'instId': 1} 2017-10-11 17:09:59,727 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ledgerId': 2, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'ppSeqNo': 4, 'viewNo': 0, 'discarded': 1, 'instId': 1, 'ppTime': 1507741799, 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892]], 'op': 'PREPREPARE'}, 'Node2') 2017-10-11 17:09:59,727 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'discarded': 1, 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892]], 'instId': 1} 2017-10-11 17:09:59,728 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv signature on propagate request 1507741799688892 2017-10-11 17:09:59,728 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'type': '110', 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'data': {'version': '1.1.40', 'action': 'in_progress'}}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,728 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'type': '110', 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'data': {'version': '1.1.40', 'action': 'in_progress'}}}, 'senderClient': 'Node1', 'op': 'PROPAGATE'}, 'Node4') 2017-10-11 17:09:59,728 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'type': '110', 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'data': {'version': '1.1.40', 'action': 'in_progress'}}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,729 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,729 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0, 'op': 'PREPARE'}, 'Node2') 2017-10-11 17:09:59,729 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,729 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'request': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'type': '110', 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'data': {'version': '1.1.40', 'action': 'in_progress'}}}, 'senderClient': 'Node1'} 2017-10-11 17:09:59,729 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'type': '110', 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'data': {'version': '1.1.40', 'action': 'in_progress'}}} 2017-10-11 17:09:59,730 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'signature': '3vKjs8t62JJURrrFFCfqaWwRQBsj563DHh7Yuy87BSbCdZWfzYUfPQVvSjpww2fMLwabneqkKQkAtKyh4Quvb9xL', 'operation': {'type': '110', 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'data': {'version': '1.1.40', 'action': 'in_progress'}}} to its replicas since already forwarded 2017-10-11 17:09:59,731 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,731 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 4) 2017-10-11 17:09:59,732 | DEBUG | replica.py ( 823) | processPrePrepare | Node1:1 received PRE-PREPARE(0, 4) from Node2:1 at 2270885.189531307 2017-10-11 17:09:59,732 | DEBUG | replica.py ( 934) | doPrepare | Node1:1 Sending PREPARE(0, 4) at 2270885.189932923 2017-10-11 17:09:59,732 | DEBUG | replica.py (2193) | send | Node1:1 sending Prepare 2017-10-11 17:09:59,732 | TRACE | replica.py (2194) | send | Node1:1 sending PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} 2017-10-11 17:09:59,732 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} 2017-10-11 17:09:59,733 | DEBUG | replica.py ( 614) | trackBatches | Node1:1 tracking batch for PREPREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'discarded': 1, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], 'instId': 1} with state root None 2017-10-11 17:09:59,733 | DEBUG | replica.py ( 842) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(0, 4) 2017-10-11 17:09:59,733 | DEBUG | node.py (2608) | send | Node1 sending message PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:09:59,733 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:09:59,736 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'messages': ['{"txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","op":"PREPARE","viewNo":0,"ppTime":1507741799,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"txnRootHash":null,"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","op":"PREPARE","viewNo":0,"ppTime":1507741799,"stateRootHash":null,"instId":1}'], 'signature': None} 2017-10-11 17:09:59,736 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","op":"PREPARE","viewNo":0,"ppTime":1507741799,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"txnRootHash":null,"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","op":"PREPARE","viewNo":0,"ppTime":1507741799,"stateRootHash":null,"instId":1}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:09:59,736 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","op":"PREPARE","viewNo":0,"ppTime":1507741799,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"txnRootHash":null,"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","op":"PREPARE","viewNo":0,"ppTime":1507741799,"stateRootHash":null,"instId":1}'], 'signature': None} 2017-10-11 17:09:59,737 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,737 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:09:59,737 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,737 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} 2017-10-11 17:09:59,737 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'stateRootHash': None, 'txnRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:09:59,737 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} 2017-10-11 17:09:59,738 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":null,"viewNo":0,"ppTime":1507741799,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:09:59,738 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":null,"viewNo":0,"ppTime":1507741799,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node4 2017-10-11 17:09:59,738 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":null,"viewNo":0,"ppTime":1507741799,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:09:59,738 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":null,"viewNo":0,"ppTime":1507741799,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node2 2017-10-11 17:09:59,739 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":null,"viewNo":0,"ppTime":1507741799,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:09:59,739 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"ppSeqNo":4,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","stateRootHash":null,"viewNo":0,"ppTime":1507741799,"op":"PREPARE","txnRootHash":null,"instId":1}' to Node3 2017-10-11 17:09:59,740 | DEBUG | replica.py ( 953) | doCommit | Node1:0 Sending COMMIT(0, 4) at 2270885.19741683 2017-10-11 17:09:59,740 | DEBUG | replica.py (2193) | send | Node1:0 sending Commit 2017-10-11 17:09:59,740 | TRACE | replica.py (2194) | send | Node1:0 sending COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} 2017-10-11 17:09:59,740 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:09:59,740 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 4) 2017-10-11 17:09:59,741 | DEBUG | replica.py ( 953) | doCommit | Node1:1 Sending COMMIT(0, 4) at 2270885.198596238 2017-10-11 17:09:59,741 | DEBUG | replica.py (2193) | send | Node1:1 sending Commit 2017-10-11 17:09:59,743 | TRACE | replica.py (2194) | send | Node1:1 sending COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} 2017-10-11 17:09:59,749 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:09:59,750 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 4) 2017-10-11 17:09:59,751 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:09:59,751 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} to all recipients: ['Node4', 'Node2', 'Node3'] 2017-10-11 17:09:59,752 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 4 messages through listener 2017-10-11 17:09:59,753 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} 2017-10-11 17:09:59,753 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 4, 'instId': 0, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:09:59,753 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} 2017-10-11 17:09:59,753 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'messages': ['{"ppSeqNo":4,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","ppTime":1507741799,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"}', '{"viewNo":0,"instId":0,"op":"COMMIT","ppSeqNo":4}', '{"ppSeqNo":4,"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"ppTime":1507741799,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","txnRootHash":null}'], 'signature': None} 2017-10-11 17:09:59,753 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"ppSeqNo":4,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","ppTime":1507741799,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"}', '{"viewNo":0,"instId":0,"op":"COMMIT","ppSeqNo":4}', '{"ppSeqNo":4,"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"ppTime":1507741799,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","txnRootHash":null}'], 'signature': None, 'op': 'BATCH'}, 'Node3') 2017-10-11 17:09:59,753 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'messages': ['{"ppSeqNo":4,"viewNo":0,"instId":0,"op":"PREPARE","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","ppTime":1507741799,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","txnRootHash":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"}', '{"viewNo":0,"instId":0,"op":"COMMIT","ppSeqNo":4}', '{"ppSeqNo":4,"viewNo":0,"instId":1,"op":"PREPARE","stateRootHash":null,"ppTime":1507741799,"digest":"71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12","txnRootHash":null}'], 'signature': None} 2017-10-11 17:09:59,754 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,754 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'op': 'PREPARE', 'instId': 0}, 'Node3') 2017-10-11 17:09:59,754 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,754 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} 2017-10-11 17:09:59,754 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'instId': 0, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:09:59,754 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} 2017-10-11 17:09:59,754 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} 2017-10-11 17:09:59,754 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'op': 'PREPARE', 'instId': 1}, 'Node3') 2017-10-11 17:09:59,755 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} 2017-10-11 17:09:59,755 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} 2017-10-11 17:09:59,755 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'op': 'COMMIT', 'viewNo': 0, 'instId': 0}, 'Node2') 2017-10-11 17:09:59,755 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} 2017-10-11 17:09:59,755 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} 2017-10-11 17:09:59,755 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'instId': 1, 'op': 'COMMIT', 'viewNo': 0}, 'Node3') 2017-10-11 17:09:59,755 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} 2017-10-11 17:09:59,756 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 2 msgs to Node4 into one transmission 2017-10-11 17:09:59,756 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"ppSeqNo":4,"op":"COMMIT","instId":0,"viewNo":0}', b'{"ppSeqNo":4,"op":"COMMIT","instId":1,"viewNo":0}']) 2017-10-11 17:09:59,756 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node4: b'{"messages":["{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"viewNo\\":0}","{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":1,\\"viewNo\\":0}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:09:59,757 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"viewNo\\":0}","{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":1,\\"viewNo\\":0}"],"signature":null,"op":"BATCH"}' to Node4 2017-10-11 17:09:59,757 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 2 msgs to Node2 into one transmission 2017-10-11 17:09:59,758 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"ppSeqNo":4,"op":"COMMIT","instId":0,"viewNo":0}', b'{"ppSeqNo":4,"op":"COMMIT","instId":1,"viewNo":0}']) 2017-10-11 17:09:59,758 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node2: b'{"messages":["{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"viewNo\\":0}","{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":1,\\"viewNo\\":0}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:09:59,758 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"viewNo\\":0}","{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":1,\\"viewNo\\":0}"],"signature":null,"op":"BATCH"}' to Node2 2017-10-11 17:09:59,758 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 2 msgs to Node3 into one transmission 2017-10-11 17:09:59,758 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"ppSeqNo":4,"op":"COMMIT","instId":0,"viewNo":0}', b'{"ppSeqNo":4,"op":"COMMIT","instId":1,"viewNo":0}']) 2017-10-11 17:09:59,760 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node3: b'{"messages":["{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"viewNo\\":0}","{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":1,\\"viewNo\\":0}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:09:59,760 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":0,\\"viewNo\\":0}","{\\"ppSeqNo\\":4,\\"op\\":\\"COMMIT\\",\\"instId\\":1,\\"viewNo\\":0}"],"signature":null,"op":"BATCH"}' to Node3 2017-10-11 17:09:59,761 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 4) from Node4:0 2017-10-11 17:09:59,761 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} commits where f is 1 2017-10-11 17:09:59,761 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 4) 2017-10-11 17:09:59,764 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ppTime': 1507741799, 'instId': 0} 2017-10-11 17:09:59,765 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (0, 4) 2017-10-11 17:09:59,765 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(0, 4) from Node3:0 2017-10-11 17:09:59,765 | TRACE | replica.py ( 925) | tryOrder | Node1:0 returning request to node 2017-10-11 17:09:59,765 | DEBUG | replica.py (1444) | doOrder | Node1:0 ordering COMMIT (0, 4) 2017-10-11 17:09:59,765 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:0 set last ordered as (0, 4) 2017-10-11 17:09:59,765 | DEBUG | replica.py (2193) | send | Node1:0 sending Ordered 2017-10-11 17:09:59,766 | TRACE | replica.py (2194) | send | Node1:0 sending ORDERED{'ppSeqNo': 4, 'txnRootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], 'instId': 0} 2017-10-11 17:09:59,766 | INFO | replica.py (1483) | order_3pc_key | Node1:0 ordered batch request, view no 0, ppSeqNo 4, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], discarded [] 2017-10-11 17:09:59,766 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(0, 4) 2017-10-11 17:09:59,766 | TRACE | message_processor.py ( 29) | discard | Node1:0 discarding message COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 0} because already ordered 3 phase message 2017-10-11 17:09:59,766 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'ppSeqNo': 4, 'digest': '71de6d0ae16dd33ae43d9d7ac08e235fc29742f10e0473400f30659958134f12', 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ppTime': 1507741799, 'instId': 1} 2017-10-11 17:09:59,766 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (0, 4) 2017-10-11 17:09:59,766 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 4) from Node3:1 2017-10-11 17:09:59,769 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} commits where f is 1 2017-10-11 17:09:59,769 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 4) 2017-10-11 17:09:59,769 | DEBUG | node.py (1847) | processOrdered | Node1 executing Ordered batch 0 4 of 1 requests 2017-10-11 17:09:59,776 | DEBUG | ledger.py ( 52) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:09:59,776 | DEBUG | node.py (2386) | sendReplyToClient | Node1 sending reply for ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892) to client 2017-10-11 17:09:59,776 | DEBUG | zstack.py ( 744) | transmitThroughListener | Node1C not sending message {'result': {'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 'reqId': 1507741799688892, 'type': '110', 'seqNo': 3, 'auditPath': ['4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7'], 'rootHash': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'signature': 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw', 'data': {'version': '1.1.40', 'action': 'in_progress'}, 'txnTime': 1507741799}, 'op': 'REPLY'} to b'Node1' 2017-10-11 17:09:59,790 | DEBUG | zstack.py ( 745) | transmitThroughListener | This is a temporary workaround for not being able to disconnect a ROUTER's remote 2017-10-11 17:09:59,790 | INFO | node.py (2290) | executeBatch | Node1 committed batch request, view no 0, ppSeqNo 4, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root 7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu, requests: [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)] 2017-10-11 17:09:59,790 | DEBUG | node.py (2305) | executeBatch | Node1 storing 3PC key (0, 4) for ledger 2 range (3, 3) 2017-10-11 17:09:59,791 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:09:59,791 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} 2017-10-11 17:09:59,791 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 4, 'op': 'COMMIT', 'viewNo': 0, 'instId': 1}, 'Node2') 2017-10-11 17:09:59,792 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} 2017-10-11 17:09:59,792 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} 2017-10-11 17:09:59,792 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'ppSeqNo': 4, 'instId': 1, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:09:59,792 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} 2017-10-11 17:09:59,794 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(0, 4) from Node2:1 2017-10-11 17:09:59,795 | TRACE | replica.py ( 925) | tryOrder | Node1:1 returning request to node 2017-10-11 17:09:59,796 | DEBUG | replica.py (1444) | doOrder | Node1:1 ordering COMMIT (0, 4) 2017-10-11 17:09:59,796 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:1 set last ordered as (0, 4) 2017-10-11 17:09:59,796 | DEBUG | replica.py (2193) | send | Node1:1 sending Ordered 2017-10-11 17:09:59,796 | TRACE | replica.py (2194) | send | Node1:1 sending ORDERED{'ppSeqNo': 4, 'txnRootHash': None, 'stateRootHash': None, 'viewNo': 0, 'ledgerId': 2, 'ppTime': 1507741799, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], 'instId': 1} 2017-10-11 17:09:59,796 | INFO | replica.py (1483) | order_3pc_key | Node1:1 ordered batch request, view no 0, ppSeqNo 4, ledger 2, state root None, txn root None, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741799688892)], discarded [] 2017-10-11 17:09:59,796 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(0, 4) 2017-10-11 17:09:59,796 | TRACE | message_processor.py ( 29) | discard | Node1:1 discarding message COMMIT{'ppSeqNo': 4, 'viewNo': 0, 'instId': 1} because already ordered 3 phase message 2017-10-11 17:09:59,799 | TRACE | node.py (1858) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:09:59,800 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:08,783 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 43 2017-10-11 17:10:08,783 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:10:08,783 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:10:08,783 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 51 to run in 60 seconds 2017-10-11 17:10:08,784 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 6 2017-10-11 17:10:08,784 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:10:08,784 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 7 to run in 60 seconds 2017-10-11 17:10:08,819 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 44 2017-10-11 17:10:08,820 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 52 to run in 60 seconds 2017-10-11 17:10:08,938 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:10:08,938 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:10:08,938 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:10:08,939 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:10:09,009 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 50 2017-10-11 17:10:09,010 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:10:09,010 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:10:09,010 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0127428759380463 is acceptable. 2017-10-11 17:10:09,010 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:09,010 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:10:09,010 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:10:09,010 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 53 to run in 10 seconds 2017-10-11 17:10:26,412 | DEBUG | node_runner.py ( 18) | run_node | You can find logs in /home/sovrin/.sovrin/Node1.log 2017-10-11 17:10:26,412 | DEBUG | node_runner.py ( 21) | run_node | Sovrin related env vars: [] 2017-10-11 17:10:27,499 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: git 2017-10-11 17:10:27,513 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: hg 2017-10-11 17:10:27,550 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: svn 2017-10-11 17:10:27,551 | DEBUG | __init__.py ( 60) | register | Registered VCS backend: bzr 2017-10-11 17:10:28,086 | DEBUG | selector_events.py ( 53) | __init__ | Using selector: EpollSelector 2017-10-11 17:10:28,086 | DEBUG | looper.py ( 125) | __init__ | Setting handler for SIGINT 2017-10-11 17:10:28,114 | DEBUG | ledger.py ( 200) | start | Starting ledger... 2017-10-11 17:10:28,120 | DEBUG | ledger.py ( 72) | recoverTree | Recovering tree from hash store of size 13 2017-10-11 17:10:28,120 | DEBUG | ledger.py ( 82) | recoverTree | Recovered tree in 0.000533882062882185 seconds 2017-10-11 17:10:28,134 | DEBUG | idr_cache.py ( 25) | __init__ | Initializing identity cache Node1 2017-10-11 17:10:28,138 | INFO | node.py (2420) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 17:10:28,178 | DEBUG | ledger.py ( 200) | start | Starting ledger... 2017-10-11 17:10:28,182 | DEBUG | ledger.py ( 72) | recoverTree | Recovering tree from hash store of size 4 2017-10-11 17:10:28,182 | DEBUG | ledger.py ( 82) | recoverTree | Recovered tree in 0.00044530071318149567 seconds 2017-10-11 17:10:28,182 | INFO | node.py (2420) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 17:10:28,227 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:10:28,227 | INFO | plugin_loader.py ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-10-11 17:10:28,227 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:10:28,227 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:10:28,227 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: ] because it does not have a 'pluginType' attribute 2017-10-11 17:10:28,227 | DEBUG | plugin_loader.py ( 96) | _load | skipping plugin plugin_firebase_stats_consumer[class: typing.Dict<~KT, ~VT>] because it does not have a 'pluginType' attribute 2017-10-11 17:10:28,228 | DEBUG | has_action_queue.py ( 79) | startRepeating | checkPerformance will be repeating every 60 seconds 2017-10-11 17:10:28,228 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 1 to run in 60 seconds 2017-10-11 17:10:28,228 | DEBUG | replica.py ( 313) | h | Node1:0 set watermarks as 0 300 2017-10-11 17:10:28,228 | DISPLAY | replicas.py ( 36) | grow | Node1 added replica Node1:0 to instance 0 (master) 2017-10-11 17:10:28,228 | DEBUG | replica.py ( 313) | h | Node1:1 set watermarks as 0 300 2017-10-11 17:10:28,228 | DISPLAY | replicas.py ( 36) | grow | Node1 added replica Node1:1 to instance 1 (backup) 2017-10-11 17:10:28,229 | DEBUG | has_action_queue.py ( 79) | startRepeating | checkPerformance will be repeating every 10 seconds 2017-10-11 17:10:28,229 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 1 to run in 10 seconds 2017-10-11 17:10:28,229 | DEBUG | has_action_queue.py ( 79) | startRepeating | checkNodeRequestSpike will be repeating every 60 seconds 2017-10-11 17:10:28,229 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 2 to run in 60 seconds 2017-10-11 17:10:28,229 | DEBUG | plugin_helper.py ( 24) | loadPlugins | Plugin loading started to load plugins from basedir: /home/sovrin/.sovrin 2017-10-11 17:10:28,229 | DEBUG | plugin_helper.py ( 68) | loadPlugins | Total plugins loaded from basedir /home/sovrin/.sovrin are : 0 2017-10-11 17:10:28,229 | DEBUG | node.py ( 340) | __init__ | total plugins loaded in node: 0 2017-10-11 17:10:28,268 | DEBUG | ledger.py ( 200) | start | Starting ledger... 2017-10-11 17:10:28,274 | DEBUG | ledger.py ( 72) | recoverTree | Recovering tree from hash store of size 3 2017-10-11 17:10:28,275 | DEBUG | ledger.py ( 82) | recoverTree | Recovered tree in 0.000536956824362278 seconds 2017-10-11 17:10:28,283 | INFO | upgrader.py ( 141) | check_upgrade_succeeded | Node 'Node1' successfully upgraded to version 1.1.40 2017-10-11 17:10:28,283 | INFO | node.py (2420) | initStateFromLedger | Node1 found state to be empty, recreating from ledger 2017-10-11 17:10:28,283 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from stopped to starting 2017-10-11 17:10:28,283 | DEBUG | ledger.py ( 198) | start | Ledger already started. 2017-10-11 17:10:28,284 | DEBUG | ledger.py ( 198) | start | Ledger already started. 2017-10-11 17:10:28,284 | DEBUG | ledger.py ( 198) | start | Ledger already started. 2017-10-11 17:10:28,284 | DEBUG | zstack.py ( 319) | start | Node1 starting with restricted as True and reSetupAuth as True 2017-10-11 17:10:28,284 | DEBUG | authenticator.py ( 31) | start | Starting ZAP at inproc://zeromq.zap.1 2017-10-11 17:10:28,284 | DEBUG | base.py ( 72) | allow | Allowing 0.0.0.0 2017-10-11 17:10:28,285 | DEBUG | base.py ( 112) | configure_curve | Configure curve: *[/home/sovrin/.sovrin/Node1/public_keys] 2017-10-11 17:10:28,285 | DEBUG | zstack.py ( 347) | open | Node1 will bind its listener at 9701 2017-10-11 17:10:28,285 | INFO | stacks.py ( 84) | start | CONNECTION: Node1 listening for other nodes at 0.0.0.0:9701 2017-10-11 17:10:28,286 | DEBUG | zstack.py ( 319) | start | Node1C starting with restricted as False and reSetupAuth as True 2017-10-11 17:10:28,286 | DEBUG | authenticator.py ( 31) | start | Starting ZAP at inproc://zeromq.zap.2 2017-10-11 17:10:28,286 | DEBUG | base.py ( 72) | allow | Allowing 0.0.0.0 2017-10-11 17:10:28,286 | DEBUG | base.py ( 112) | configure_curve | Configure curve: *[*] 2017-10-11 17:10:28,286 | DEBUG | zstack.py ( 347) | open | Node1C will bind its listener at 9702 2017-10-11 17:10:28,286 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action propose_view_change with id 3 to run in 60 seconds 2017-10-11 17:10:28,287 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 4 to run in 3 seconds 2017-10-11 17:10:28,287 | DEBUG | has_action_queue.py ( 79) | startRepeating | dump_json_file will be repeating every 60 seconds 2017-10-11 17:10:28,287 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 5 to run in 60 seconds 2017-10-11 17:10:28,287 | INFO | node.py ( 621) | start | Node1 first time running... 2017-10-11 17:10:28,288 | DEBUG | kit_zstack.py ( 97) | connectToMissing | CONNECTION: Node1 found the following missing connections: Node3, Node2, Node4 2017-10-11 17:10:28,288 | TRACE | remote.py ( 86) | connect | connecting socket 78 30767392 to remote Node3:HA(host='10.0.0.4', port=9705) 2017-10-11 17:10:28,289 | INFO | zstack.py ( 590) | connect | CONNECTION: Node1 looking for Node3 at 10.0.0.4:9705 2017-10-11 17:10:28,289 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 pinged Node3 2017-10-11 17:10:28,290 | TRACE | remote.py ( 86) | connect | connecting socket 80 30870976 to remote Node2:HA(host='10.0.0.3', port=9703) 2017-10-11 17:10:28,290 | INFO | zstack.py ( 590) | connect | CONNECTION: Node1 looking for Node2 at 10.0.0.3:9703 2017-10-11 17:10:28,290 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 pinged Node2 2017-10-11 17:10:28,291 | TRACE | remote.py ( 86) | connect | connecting socket 83 30908400 to remote Node4:HA(host='10.0.0.5', port=9707) 2017-10-11 17:10:28,291 | INFO | zstack.py ( 590) | connect | CONNECTION: Node1 looking for Node4 at 10.0.0.5:9707 2017-10-11 17:10:28,291 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 pinged Node4 2017-10-11 17:10:28,291 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:10:28,303 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'pi' to Node3 2017-10-11 17:10:28,303 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'pi' to Node3 2017-10-11 17:10:28,303 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'pi' to Node2 2017-10-11 17:10:28,303 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'pi' to Node2 2017-10-11 17:10:28,303 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'pi' to Node4 2017-10-11 17:10:28,303 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'pi' to Node4 2017-10-11 17:10:28,348 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 7 messages through listener 2017-10-11 17:10:28,348 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} 2017-10-11 17:10:28,348 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'op': 'INSTANCE_CHANGE', 'reason': 26}, 'Node3') 2017-10-11 17:10:28,349 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} 2017-10-11 17:10:28,349 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,349 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}']}, 'Node3') 2017-10-11 17:10:28,349 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,350 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,350 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 0}}, 'Node3') 2017-10-11 17:10:28,350 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,350 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,350 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 0}}, 'Node3') 2017-10-11 17:10:28,350 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,351 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,351 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}']}, 'Node3') 2017-10-11 17:10:28,351 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,351 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,351 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}}, 'Node3') 2017-10-11 17:10:28,351 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,351 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,351 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}}, 'Node3') 2017-10-11 17:10:28,351 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,352 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,352 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}']}, 'Node3') 2017-10-11 17:10:28,352 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,352 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,352 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}}, 'Node3') 2017-10-11 17:10:28,352 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,352 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,356 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}}, 'Node3') 2017-10-11 17:10:28,356 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,358 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,358 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']], 'op': 'VIEW_CHANGE_DONE'}, 'Node3') 2017-10-11 17:10:28,358 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,358 | DEBUG | zstack.py ( 663) | handlePingPong | Node1 got ping from Node3 2017-10-11 17:10:28,358 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 ponged Node3 2017-10-11 17:10:28,360 | DEBUG | zstack.py ( 669) | handlePingPong | Node1 got pong from Node3 2017-10-11 17:10:28,360 | DEBUG | node.py (1915) | processInstanceChange | Node1 received instance change request: INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} from Node3 2017-10-11 17:10:28,360 | DEBUG | node.py (1958) | do_view_change_if_possible | Node1 has no quorum for view 1 2017-10-11 17:10:28,360 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:10:28,360 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:28,360 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:10:28,360 | DEBUG | node.py (1946) | processInstanceChange | Node1 received instance change message INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} but did not find the master to be slow or has already sent an instance change message 2017-10-11 17:10:28,361 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'}, 'params': {'ledgerId': 0}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,361 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'}, 'params': {'ledgerId': 0}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,362 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,364 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,364 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'}, 'params': {'ledgerId': 1}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,364 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'}, 'params': {'ledgerId': 1}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,365 | DEBUG | node.py (1170) | msgHasAcceptableViewNo | Node1 stashing a message for a future view: VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,365 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:10:28,365 | DEBUG | zstack.py ( 663) | handlePingPong | Node1C got ping from b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:10:28,365 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'po' to b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' through listener socket 2017-10-11 17:10:28,365 | DEBUG | zstack.py ( 645) | sendPingPong | Node1C ponged b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:10:28,366 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 7 msgs to Node3 into one transmission 2017-10-11 17:10:28,366 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'po', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"msg":{"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ppSeqNo":null,"txnSeqNo":4,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"msg":{"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ppSeqNo":null,"txnSeqNo":4,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":null,"txnSeqNo":3,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":null,"txnSeqNo":3,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr","ppSeqNo":null,"txnSeqNo":13,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr","ppSeqNo":null,"txnSeqNo":13,"viewNo":null},"op":"MESSAGE_RESPONSE"}']) 2017-10-11 17:10:28,366 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node3: b'{"messages":["po","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,366 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["po","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' to Node3 2017-10-11 17:10:28,367 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from set() to {'Node3'} 2017-10-11 17:10:28,367 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node1 now connected to Node3 2017-10-11 17:10:28,367 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,367 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 14 messages through listener 2017-10-11 17:10:28,369 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} 2017-10-11 17:10:28,369 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'op': 'INSTANCE_CHANGE', 'reason': 26}, 'Node2') 2017-10-11 17:10:28,369 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} 2017-10-11 17:10:28,369 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} 2017-10-11 17:10:28,369 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'op': 'INSTANCE_CHANGE', 'reason': 26}, 'Node4') 2017-10-11 17:10:28,369 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} 2017-10-11 17:10:28,369 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":0}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":0}}']} 2017-10-11 17:10:28,369 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":0}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":0}}']}, 'Node2') 2017-10-11 17:10:28,370 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":0}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":0}}']} 2017-10-11 17:10:28,382 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,382 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}, 'op': 'MESSAGE_REQUEST'}, 'Node2') 2017-10-11 17:10:28,382 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,382 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,382 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}, 'op': 'MESSAGE_REQUEST'}, 'Node2') 2017-10-11 17:10:28,382 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,383 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,383 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:10:28,383 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,383 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,383 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 0}}, 'Node4') 2017-10-11 17:10:28,383 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,383 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,383 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 0}}, 'Node4') 2017-10-11 17:10:28,383 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 0}} 2017-10-11 17:10:28,384 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}']} 2017-10-11 17:10:28,384 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}']}, 'Node2') 2017-10-11 17:10:28,384 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":2}}']} 2017-10-11 17:10:28,384 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,385 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST'}, 'Node2') 2017-10-11 17:10:28,385 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,385 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,385 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'op': 'MESSAGE_REQUEST'}, 'Node2') 2017-10-11 17:10:28,385 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,385 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,385 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:10:28,385 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,385 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,386 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}}, 'Node4') 2017-10-11 17:10:28,386 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,386 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,386 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 2}}, 'Node4') 2017-10-11 17:10:28,386 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,387 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}']} 2017-10-11 17:10:28,387 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}']}, 'Node2') 2017-10-11 17:10:28,387 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}', '{"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST","params":{"ledgerId":1}}']} 2017-10-11 17:10:28,387 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,387 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST'}, 'Node2') 2017-10-11 17:10:28,387 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,387 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,387 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'op': 'MESSAGE_REQUEST'}, 'Node2') 2017-10-11 17:10:28,388 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,388 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,388 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:10:28,388 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}']} 2017-10-11 17:10:28,388 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,388 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}}, 'Node4') 2017-10-11 17:10:28,388 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,388 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,389 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_REQUEST', 'params': {'ledgerId': 1}}, 'Node4') 2017-10-11 17:10:28,389 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,389 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,389 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']], 'op': 'VIEW_CHANGE_DONE'}, 'Node2') 2017-10-11 17:10:28,389 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,390 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,390 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']], 'op': 'VIEW_CHANGE_DONE'}, 'Node4') 2017-10-11 17:10:28,390 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,390 | DEBUG | zstack.py ( 663) | handlePingPong | Node1 got ping from Node2 2017-10-11 17:10:28,390 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 ponged Node2 2017-10-11 17:10:28,390 | DEBUG | zstack.py ( 663) | handlePingPong | Node1 got ping from Node4 2017-10-11 17:10:28,390 | DEBUG | zstack.py ( 645) | sendPingPong | Node1 ponged Node4 2017-10-11 17:10:28,390 | DEBUG | zstack.py ( 669) | handlePingPong | Node1 got pong from Node2 2017-10-11 17:10:28,390 | DEBUG | zstack.py ( 669) | handlePingPong | Node1 got pong from Node4 2017-10-11 17:10:28,391 | DEBUG | node.py (1915) | processInstanceChange | Node1 received instance change request: INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} from Node2 2017-10-11 17:10:28,391 | DEBUG | node.py (1958) | do_view_change_if_possible | Node1 has no quorum for view 1 2017-10-11 17:10:28,391 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:10:28,391 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:28,391 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:10:28,391 | DEBUG | node.py (1946) | processInstanceChange | Node1 received instance change message INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} but did not find the master to be slow or has already sent an instance change message 2017-10-11 17:10:28,392 | DEBUG | node.py (1915) | processInstanceChange | Node1 received instance change request: INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} from Node4 2017-10-11 17:10:28,393 | INFO | node.py (1954) | do_view_change_if_possible | VIEW CHANGE: Node1 initiating a view change to 1 from 0 2017-10-11 17:10:28,394 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action _check_view_change_completed with id 6 to run in 60 seconds 2017-10-11 17:10:28,394 | DEBUG | replica.py ( 455) | on_view_change_start | Node1:0 setting last prepared for master to None 2017-10-11 17:10:28,394 | DEBUG | node.py (2146) | startViewChange | Node1 resetting monitor stats after view change 2017-10-11 17:10:28,394 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:10:28,395 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}, 'Node3') 2017-10-11 17:10:28,395 | DEBUG | node.py ( 996) | processStashedMsgsForView | Node1 processed 1 stashed msgs for view no 1 2017-10-11 17:10:28,395 | INFO | node.py (2159) | startViewChange | VIEW CHANGE: Node1 changed to view 1, will start catchup now 2017-10-11 17:10:28,396 | DEBUG | node.py (2185) | start_catchup | Node1 does not start the catchup procedure because it is already in this state 2017-10-11 17:10:28,396 | DEBUG | monitor.py ( 335) | isMasterThroughputTooLow | Node1 master throughput is not measurable. 2017-10-11 17:10:28,396 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:28,396 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:10:28,396 | DEBUG | node.py (1946) | processInstanceChange | Node1 received instance change message INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} but did not find the master to be slow or has already sent an instance change message 2017-10-11 17:10:28,397 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'}, 'params': {'ledgerId': 0}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,397 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'}, 'params': {'ledgerId': 0}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,397 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'}, 'params': {'ledgerId': 0}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,398 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'}, 'params': {'ledgerId': 0}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,398 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,398 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,399 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,399 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,400 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'}, 'params': {'ledgerId': 1}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,400 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'}, 'params': {'ledgerId': 1}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,400 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'}, 'params': {'ledgerId': 1}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,401 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'}, 'params': {'ledgerId': 1}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,401 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}, 'Node2') 2017-10-11 17:10:28,401 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}, 'Node4') 2017-10-11 17:10:28,402 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1C got 1 messages through listener 2017-10-11 17:10:28,402 | TRACE | node.py (1388) | validateClientMsg | Node1C received CLIENT message: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,402 | DEBUG | node.py (1434) | processClientInBox | Node1C processing b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' request LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,402 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} from b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' 2017-10-11 17:10:28,403 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,403 | TRACE | zstack.py ( 754) | transmitThroughListener | Node1C transmitting b'{"viewNo":null,"txnSeqNo":4,"ppSeqNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"op":"LEDGER_STATUS"}' to b'+>]}>OKfYQDWB?@taDN+?+{nc/ZoKRv9iH*/^F$a' through listener socket 2017-10-11 17:10:28,403 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,403 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,403 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"viewNo":null,"txnSeqNo":4,"ppSeqNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"op":"LEDGER_STATUS"}' to Node3 2017-10-11 17:10:28,405 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"viewNo":null,"txnSeqNo":4,"ppSeqNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"op":"LEDGER_STATUS"}' to Node3 2017-10-11 17:10:28,405 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 7 msgs to Node2 into one transmission 2017-10-11 17:10:28,405 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'po', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"msg":{"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ppSeqNo":null,"txnSeqNo":4,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"msg":{"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ppSeqNo":null,"txnSeqNo":4,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":null,"txnSeqNo":3,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":null,"txnSeqNo":3,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr","ppSeqNo":null,"txnSeqNo":13,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr","ppSeqNo":null,"txnSeqNo":13,"viewNo":null},"op":"MESSAGE_RESPONSE"}']) 2017-10-11 17:10:28,406 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node2: b'{"messages":["po","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,406 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["po","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' to Node2 2017-10-11 17:10:28,407 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 7 msgs to Node4 into one transmission 2017-10-11 17:10:28,408 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'po', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"msg":{"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ppSeqNo":null,"txnSeqNo":4,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":0},"msg":{"ledgerId":0,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ppSeqNo":null,"txnSeqNo":4,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":null,"txnSeqNo":3,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":null,"txnSeqNo":3,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr","ppSeqNo":null,"txnSeqNo":13,"viewNo":null},"op":"MESSAGE_RESPONSE"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr","ppSeqNo":null,"txnSeqNo":13,"viewNo":null},"op":"MESSAGE_RESPONSE"}']) 2017-10-11 17:10:28,408 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node4: b'{"messages":["po","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,408 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["po","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":0},\\"msg\\":{\\"ledgerId\\":0,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":4,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"msg\\":{\\"ledgerId\\":2,\\"merkleRoot\\":\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":3,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"msg\\":{\\"ledgerId\\":1,\\"merkleRoot\\":\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\",\\"ppSeqNo\\":null,\\"txnSeqNo\\":13,\\"viewNo\\":null},\\"op\\":\\"MESSAGE_RESPONSE\\"}"],"signature":null,"op":"BATCH"}' to Node4 2017-10-11 17:10:28,409 | DEBUG | keep_in_touch.py ( 68) | conns | Node1's connections changed from {'Node3'} to {'Node3', 'Node2', 'Node4'} 2017-10-11 17:10:28,409 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node1 now connected to Node2 2017-10-11 17:10:28,409 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node1 now connected to Node4 2017-10-11 17:10:28,410 | DEBUG | motor.py ( 34) | set_status | Node1 changing status from starting to started 2017-10-11 17:10:28,410 | DEBUG | node.py ( 941) | checkInstances | Node1 choosing to start election on the basis of count 4 and nodes {'Node3', 'Node2', 'Node4'} 2017-10-11 17:10:28,410 | DEBUG | primary_selector.py ( 77) | get_msgs_for_lagged_nodes | Node1 has no ViewChangeDone message to send for view 1 2017-10-11 17:10:28,410 | DEBUG | node.py ( 902) | send_current_state_to_lagging_node | Node1 sending current state CURRENT_STATE{'viewNo': 1, 'primary': []} to lagged node Node2 2017-10-11 17:10:28,410 | DEBUG | node.py (2608) | send | Node1 sending message CURRENT_STATE{'viewNo': 1, 'primary': []} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,411 | DEBUG | primary_selector.py ( 77) | get_msgs_for_lagged_nodes | Node1 has no ViewChangeDone message to send for view 1 2017-10-11 17:10:28,411 | DEBUG | node.py ( 902) | send_current_state_to_lagging_node | Node1 sending current state CURRENT_STATE{'viewNo': 1, 'primary': []} to lagged node Node4 2017-10-11 17:10:28,411 | DEBUG | node.py (2608) | send | Node1 sending message CURRENT_STATE{'viewNo': 1, 'primary': []} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,411 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,411 | DEBUG | node.py (2608) | send | Node1 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,412 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:10:28,413 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"viewNo":1,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"op":"CURRENT_STATE"}', '{"txnSeqNo":4,"viewNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","op":"LEDGER_STATUS","ledgerId":0,"ppSeqNo":null}', '{"txnSeqNo":13,"viewNo":0,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","op":"LEDGER_STATUS","ledgerId":1,"ppSeqNo":2}', '{"txnSeqNo":3,"viewNo":0,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","op":"LEDGER_STATUS","ledgerId":2,"ppSeqNo":4}']} 2017-10-11 17:10:28,413 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"viewNo":1,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"op":"CURRENT_STATE"}', '{"txnSeqNo":4,"viewNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","op":"LEDGER_STATUS","ledgerId":0,"ppSeqNo":null}', '{"txnSeqNo":13,"viewNo":0,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","op":"LEDGER_STATUS","ledgerId":1,"ppSeqNo":2}', '{"txnSeqNo":3,"viewNo":0,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","op":"LEDGER_STATUS","ledgerId":2,"ppSeqNo":4}']}, 'Node3') 2017-10-11 17:10:28,413 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"viewNo":1,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"op":"CURRENT_STATE"}', '{"txnSeqNo":4,"viewNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","op":"LEDGER_STATUS","ledgerId":0,"ppSeqNo":null}', '{"txnSeqNo":13,"viewNo":0,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","op":"LEDGER_STATUS","ledgerId":1,"ppSeqNo":2}', '{"txnSeqNo":3,"viewNo":0,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","op":"LEDGER_STATUS","ledgerId":2,"ppSeqNo":4}']} 2017-10-11 17:10:28,413 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} 2017-10-11 17:10:28,414 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}], 'op': 'CURRENT_STATE'}, 'Node3') 2017-10-11 17:10:28,414 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} 2017-10-11 17:10:28,414 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,414 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': None, 'txnSeqNo': 4, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:10:28,414 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,414 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} 2017-10-11 17:10:28,415 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'txnSeqNo': 13, 'ppSeqNo': 2, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:10:28,415 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} 2017-10-11 17:10:28,418 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} 2017-10-11 17:10:28,418 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'txnSeqNo': 3, 'ppSeqNo': 4, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node3') 2017-10-11 17:10:28,418 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} 2017-10-11 17:10:28,418 | DEBUG | node.py ( 907) | process_current_state_message | Node1 processing current state CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} from Node3 2017-10-11 17:10:28,419 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}, 'Node3') 2017-10-11 17:10:28,419 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} from Node3 2017-10-11 17:10:28,419 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,419 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,420 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node3 2017-10-11 17:10:28,420 | DEBUG | ledger_manager.py (1016) | stashLedgerStatus | Node1 stashing ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node3 2017-10-11 17:10:28,420 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node3 2017-10-11 17:10:28,420 | DEBUG | ledger_manager.py (1016) | stashLedgerStatus | Node1 stashing ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node3 2017-10-11 17:10:28,420 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node3 : VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 190) | _hasViewChangeQuorum | Node1 needs 2 ViewChangeDone messages 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 266) | _startSelection | Node1 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node2 : VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 190) | _hasViewChangeQuorum | Node1 needs 1 ViewChangeDone messages 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 266) | _startSelection | Node1 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node4 : VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 196) | _hasViewChangeQuorum | Node1 got view change quorum (3 >= 3) 2017-10-11 17:10:28,421 | DEBUG | primary_selector.py ( 215) | has_view_change_from_primary | Node1 received ViewChangeDone from primary Node2 2017-10-11 17:10:28,422 | DEBUG | primary_selector.py ( 238) | has_sufficient_same_view_change_done_messages | Node1 found acceptable primary Node2 and ledger info ((0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'), (2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu')) 2017-10-11 17:10:28,422 | DEBUG | primary_selector.py ( 271) | _startSelection | Node1 cannot start primary selection since mode is 100 2017-10-11 17:10:28,422 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node3 : VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,422 | DEBUG | primary_selector.py ( 271) | _startSelection | Node1 cannot start primary selection since mode is 100 2017-10-11 17:10:28,422 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 2 msgs to Node2 into one transmission 2017-10-11 17:10:28,422 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"viewNo":1,"primary":[],"op":"CURRENT_STATE"}', b'{"viewNo":null,"txnSeqNo":4,"ppSeqNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"op":"LEDGER_STATUS"}']) 2017-10-11 17:10:28,423 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node2: b'{"messages":["{\\"viewNo\\":1,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"viewNo\\":null,\\"txnSeqNo\\":4,\\"ppSeqNo\\":null,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,423 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"viewNo\\":1,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"viewNo\\":null,\\"txnSeqNo\\":4,\\"ppSeqNo\\":null,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' to Node2 2017-10-11 17:10:28,423 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 2 msgs to Node4 into one transmission 2017-10-11 17:10:28,423 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"viewNo":1,"primary":[],"op":"CURRENT_STATE"}', b'{"viewNo":null,"txnSeqNo":4,"ppSeqNo":null,"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"op":"LEDGER_STATUS"}']) 2017-10-11 17:10:28,424 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node4: b'{"messages":["{\\"viewNo\\":1,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"viewNo\\":null,\\"txnSeqNo\\":4,\\"ppSeqNo\\":null,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,424 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"viewNo\\":1,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"viewNo\\":null,\\"txnSeqNo\\":4,\\"ppSeqNo\\":null,\\"merkleRoot\\":\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\",\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}"],"signature":null,"op":"BATCH"}' to Node4 2017-10-11 17:10:28,441 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:10:28,442 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"viewNo":1,"op":"CURRENT_STATE"}', '{"op":"LEDGER_STATUS","merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"ledgerId":0,"txnSeqNo":4}', '{"op":"LEDGER_STATUS","merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","viewNo":0,"ppSeqNo":2,"ledgerId":1,"txnSeqNo":13}', '{"op":"LEDGER_STATUS","merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","viewNo":0,"ppSeqNo":4,"ledgerId":2,"txnSeqNo":3}']} 2017-10-11 17:10:28,442 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"viewNo":1,"op":"CURRENT_STATE"}', '{"op":"LEDGER_STATUS","merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"ledgerId":0,"txnSeqNo":4}', '{"op":"LEDGER_STATUS","merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","viewNo":0,"ppSeqNo":2,"ledgerId":1,"txnSeqNo":13}', '{"op":"LEDGER_STATUS","merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","viewNo":0,"ppSeqNo":4,"ledgerId":2,"txnSeqNo":3}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:10:28,442 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"viewNo":1,"op":"CURRENT_STATE"}', '{"op":"LEDGER_STATUS","merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","viewNo":null,"ppSeqNo":null,"ledgerId":0,"txnSeqNo":4}', '{"op":"LEDGER_STATUS","merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","viewNo":0,"ppSeqNo":2,"ledgerId":1,"txnSeqNo":13}', '{"op":"LEDGER_STATUS","merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","viewNo":0,"ppSeqNo":4,"ledgerId":2,"txnSeqNo":3}']} 2017-10-11 17:10:28,444 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} 2017-10-11 17:10:28,444 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}], 'op': 'CURRENT_STATE'}, 'Node4') 2017-10-11 17:10:28,445 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} 2017-10-11 17:10:28,445 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,445 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': None, 'txnSeqNo': 4, 'ppSeqNo': None, 'op': 'LEDGER_STATUS', 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'}, 'Node4') 2017-10-11 17:10:28,445 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,445 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} 2017-10-11 17:10:28,445 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'txnSeqNo': 13, 'ppSeqNo': 2, 'op': 'LEDGER_STATUS', 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'Node4') 2017-10-11 17:10:28,445 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} 2017-10-11 17:10:28,446 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} 2017-10-11 17:10:28,446 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'txnSeqNo': 3, 'ppSeqNo': 4, 'op': 'LEDGER_STATUS', 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'Node4') 2017-10-11 17:10:28,446 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} 2017-10-11 17:10:28,446 | DEBUG | node.py ( 907) | process_current_state_message | Node1 processing current state CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} from Node4 2017-10-11 17:10:28,446 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}, 'Node4') 2017-10-11 17:10:28,447 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} from Node4 2017-10-11 17:10:28,447 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,447 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,447 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node4'} that its ledger of type 0 is latest 2017-10-11 17:10:28,447 | DEBUG | ledger_manager.py ( 315) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} that it does not need catchup 2017-10-11 17:10:28,447 | DEBUG | node.py (1479) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 17:10:28,447 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 17:10:28,447 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 17:10:28,447 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 17:10:28,447 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 17:10:28,447 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:10:28,448 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:10:28,448 | INFO | node.py (1489) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 0 2017-10-11 17:10:28,448 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,448 | DEBUG | node.py ( 941) | checkInstances | Node1 choosing to start election on the basis of count 4 and nodes {'Node3', 'Node2', 'Node4'} 2017-10-11 17:10:28,449 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,449 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node1 for ledger status of ledger 2 2017-10-11 17:10:28,449 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,449 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node2 for ledger status of ledger 2 2017-10-11 17:10:28,449 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,450 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node3 for ledger status of ledger 2 2017-10-11 17:10:28,450 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,450 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node4 for ledger status of ledger 2 2017-10-11 17:10:28,450 | DEBUG | ledger_manager.py (1027) | processStashedLedgerStatuses | Node1 going to process 1 stashed ledger statuses for ledger 2 2017-10-11 17:10:28,450 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node3 2017-10-11 17:10:28,450 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,450 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,450 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node1 completed catching up ledger 0, caught up 0 in total 2017-10-11 17:10:28,451 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node4 2017-10-11 17:10:28,452 | DEBUG | ledger_manager.py (1016) | stashLedgerStatus | Node1 stashing ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node4 2017-10-11 17:10:28,452 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node4 2017-10-11 17:10:28,452 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,453 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,453 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node4'} that its ledger of type 2 is latest 2017-10-11 17:10:28,453 | DEBUG | ledger_manager.py ( 315) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} that it does not need catchup 2017-10-11 17:10:28,453 | DEBUG | node.py (1479) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 17:10:28,453 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 17:10:28,453 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 17:10:28,453 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 17:10:28,453 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 17:10:28,453 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:10:28,453 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:10:28,453 | INFO | node.py (1489) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 2 2017-10-11 17:10:28,454 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,454 | DEBUG | pool_config.py ( 35) | processLedger | processing config ledger for any POOL_CONFIGs 2017-10-11 17:10:28,457 | DEBUG | upgrader.py ( 208) | processLedger | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv processing config ledger for any upgrades 2017-10-11 17:10:28,458 | INFO | upgrader.py ( 214) | processLedger | Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv found upgrade START txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade40'), ('reinstall', False), ('reqId', 1507741607555952), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-10-11T17:25:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-10-11T17:15:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-10-11T17:20:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-10-11T17:10:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i'), ('timeout', 10), ('txnTime', 1507741607), ('type', '109'), ('version', '1.1.40'), ('seqNo', 2)]) 2017-10-11 17:10:28,459 | INFO | upgrader.py ( 292) | handleUpgradeTxn | Node 'Node1' handles upgrade txn OrderedDict([('action', 'start'), ('force', False), ('identifier', 'V4SGRU86Z58d6TV7PBUe6f'), ('justification', None), ('name', 'upgrade40'), ('reinstall', False), ('reqId', 1507741607555952), ('schedule', OrderedDict([('4PS3EDQ3dW1tci1Bp6543CfuuebjFrg36kLAUcskGfaA', '2017-10-11T17:25:00.258870+00:00'), ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', '2017-10-11T17:15:00.258870+00:00'), ('DKVxG2fXXTU8yT5N7hGEbXB3dfdAnYv1JczDUHpmDxya', '2017-10-11T17:20:00.258870+00:00'), ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', '2017-10-11T17:10:00.258870+00:00')])), ('sha256', 'ed0a366b4ef36d40c055672a8b83679e99246fec71a706b4ae4cb7958feace3f'), ('signature', '3ACoow9Gjp7Ta2XUW2CBYqytPQ7LThnAp9b8cwdCvd3bFHPWry15xbHSeGLjUWMfRRfPm4X98q9ccGtqCutqj1i'), ('timeout', 10), ('txnTime', 1507741607), ('type', '109'), ('version', '1.1.40'), ('seqNo', 2)]) 2017-10-11 17:10:28,459 | INFO | upgrader.py ( 312) | handleUpgradeTxn | Node 'Node1' has already performed an upgrade with upgrade_id 15077416075559522. Last recorded event is ('succeeded', datetime.datetime(2017, 10, 11, 17, 10, 0, 258870, tzinfo=tzlocal()), '1.1.40', '15077416075559522') 2017-10-11 17:10:28,459 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,459 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node1 for ledger status of ledger 1 2017-10-11 17:10:28,459 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node2'] 2017-10-11 17:10:28,459 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node2 for ledger status of ledger 1 2017-10-11 17:10:28,460 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node3'] 2017-10-11 17:10:28,460 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node3 for ledger status of ledger 1 2017-10-11 17:10:28,460 | DEBUG | node.py (2608) | send | Node1 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node4'] 2017-10-11 17:10:28,460 | DEBUG | node.py ( 855) | _ask_for_ledger_status | Node1 asking Node4 for ledger status of ledger 1 2017-10-11 17:10:28,460 | DEBUG | ledger_manager.py (1027) | processStashedLedgerStatuses | Node1 going to process 2 stashed ledger statuses for ledger 1 2017-10-11 17:10:28,460 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node3 2017-10-11 17:10:28,460 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,460 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,460 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node4 2017-10-11 17:10:28,461 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,461 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,461 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node4'} that its ledger of type 1 is latest 2017-10-11 17:10:28,461 | DEBUG | ledger_manager.py ( 315) | processLedgerStatus | Node1 found from ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} that it does not need catchup 2017-10-11 17:10:28,462 | DEBUG | node.py (1479) | preLedgerCatchUp | Node1 going to process any ordered requests before starting catchup. 2017-10-11 17:10:28,462 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 17:10:28,462 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 0 before starting catch up 2017-10-11 17:10:28,462 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:1 going to remove 0 Ordered messages from outbox 2017-10-11 17:10:28,462 | DEBUG | node.py (1888) | force_process_ordered | Node1 processed 0 Ordered batches for instance 1 before starting catch up 2017-10-11 17:10:28,462 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:10:28,463 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:10:28,463 | INFO | node.py (1489) | preLedgerCatchUp | Node1 reverted 0 batches before starting catch up for ledger 1 2017-10-11 17:10:28,463 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,463 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node1 completed catching up ledger 1, caught up 0 in total 2017-10-11 17:10:28,463 | DEBUG | node.py (1609) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 17:10:28,463 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:0 set last ordered as (0, 4) 2017-10-11 17:10:28,463 | DEBUG | replica.py (2234) | _remove_till_caught_up_3pc | Node1:0 going to remove messages for 0 3PC keys 2017-10-11 17:10:28,463 | DEBUG | replica.py (2259) | _remove_ordered_from_queue | Node1:0 going to remove 0 Ordered messages from outbox 2017-10-11 17:10:28,463 | INFO | node.py (1525) | allLedgersCaughtUp | CATCH-UP: Node1 caught up till (0, 4) 2017-10-11 17:10:28,463 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:10:28,464 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:10:28,464 | DEBUG | primary_selector.py ( 196) | _hasViewChangeQuorum | Node1 got view change quorum (3 >= 3) 2017-10-11 17:10:28,464 | DEBUG | node.py (1552) | is_catchup_needed | Node1 is not caught up for the current view 1 2017-10-11 17:10:28,464 | DEBUG | node.py (1609) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 17:10:28,464 | DEBUG | node.py (1557) | is_catchup_needed | Node1 ordered till last prepared certificate 2017-10-11 17:10:28,464 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node1 does not need any more catchups 2017-10-11 17:10:28,465 | DEBUG | primary_decider.py ( 134) | send | Node1's elector sending VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 13, 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'), (2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu')]} 2017-10-11 17:10:28,465 | DEBUG | primary_selector.py ( 281) | _startSelection | Node1 starting selection 2017-10-11 17:10:28,465 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node1:0 selected primary Node2:0 for instance 0 (view 1) 2017-10-11 17:10:28,465 | INFO | node.py ( 510) | start_participating | Node1 started participating 2017-10-11 17:10:28,465 | DEBUG | replica.py ( 408) | primaryName | Node1:0 setting primaryName for view no 1 to: Node2:0 2017-10-11 17:10:28,465 | DEBUG | replica.py (1694) | _gc | Node1:0 cleaning up till (0, 4) 2017-10-11 17:10:28,465 | DEBUG | replica.py (1709) | _gc | Node1:0 found 0 3-phase keys to clean 2017-10-11 17:10:28,465 | DEBUG | replica.py (1711) | _gc | Node1:0 found 0 request keys to clean 2017-10-11 17:10:28,466 | DEBUG | replica.py ( 313) | h | Node1:0 set watermarks as 0 300 2017-10-11 17:10:28,466 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node1:0 declares view change 1 as completed for instance 0, new primary is Node2:0, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 13, 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'), (2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu')] 2017-10-11 17:10:28,466 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node1:1 selected primary Node3:1 for instance 1 (view 1) 2017-10-11 17:10:28,466 | DEBUG | replica.py ( 408) | primaryName | Node1:1 setting primaryName for view no 1 to: Node3:1 2017-10-11 17:10:28,466 | DEBUG | replica.py (1694) | _gc | Node1:1 cleaning up till (0, 0) 2017-10-11 17:10:28,466 | DEBUG | replica.py (1709) | _gc | Node1:1 found 0 3-phase keys to clean 2017-10-11 17:10:28,466 | DEBUG | replica.py (1711) | _gc | Node1:1 found 0 request keys to clean 2017-10-11 17:10:28,466 | DEBUG | replica.py ( 313) | h | Node1:1 set watermarks as 0 300 2017-10-11 17:10:28,467 | DEBUG | replica.py ( 514) | _setup_for_non_master | Node1:1 Setting last ordered for non-master as (0, 0) 2017-10-11 17:10:28,467 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:1 set last ordered as (1, 0) 2017-10-11 17:10:28,467 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node1:1 declares view change 1 as completed for instance 1, new primary is Node3:1, ledger info is [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 13, 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'), (2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu')] 2017-10-11 17:10:28,467 | INFO | upgrader.py ( 150) | should_notify_about_upgrade_result | Node's 'Node1' last upgrade txn is OrderedDict([('data', OrderedDict([('action', 'in_progress'), ('version', '1.1.40')])), ('identifier', 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'), ('reqId', 1507741799688892), ('signature', 'JJhLp5nsb5W7rNHn9RbBUQPsEcgyFynvBs5tRt62GLsoVt4wZZDRtTKM8kX2Mm8d39Xa5L5QVKsjVmmz6ngyYGw'), ('txnTime', 1507741799), ('type', '110'), ('seqNo', 3)]) 2017-10-11 17:10:28,467 | DEBUG | node.py ( 237) | acknowledge_upgrade | Node1 found the first run after upgrade, sending NODE_UPGRADE 2017-10-11 17:10:28,469 | DEBUG | node.py (2608) | send | Node1 sending message Request: {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'action': 'complete', 'version': '1.1.40'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,469 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node1 completed catching up ledger 2, caught up 0 in total 2017-10-11 17:10:28,469 | DEBUG | node.py (1609) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 17:10:28,469 | DEBUG | node.py (2469) | processStashedOrderedReqs | Node1 processed 0 stashed ordered requests 2017-10-11 17:10:28,469 | DEBUG | monitor.py ( 192) | reset | Node1's Monitor being reset 2017-10-11 17:10:28,469 | DEBUG | primary_selector.py ( 196) | _hasViewChangeQuorum | Node1 got view change quorum (4 >= 2) 2017-10-11 17:10:28,470 | DEBUG | node.py (1552) | is_catchup_needed | Node1 is not caught up for the current view 1 2017-10-11 17:10:28,470 | DEBUG | node.py (1609) | num_txns_caught_up_in_last_catchup | Node1 caught up to 0 txns in the last catchup 2017-10-11 17:10:28,470 | DEBUG | node.py (1557) | is_catchup_needed | Node1 ordered till last prepared certificate 2017-10-11 17:10:28,470 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node1 does not need any more catchups 2017-10-11 17:10:28,470 | DEBUG | primary_selector.py ( 281) | _startSelection | Node1 starting selection 2017-10-11 17:10:28,470 | DEBUG | primary_selector.py ( 284) | _startSelection | Node1:0 already has a primary 2017-10-11 17:10:28,470 | DEBUG | primary_selector.py ( 284) | _startSelection | Node1:1 already has a primary 2017-10-11 17:10:28,470 | INFO | node.py (1624) | no_more_catchups_needed | Node1 starting to participate since catchup is done, primaries are selected but mode was not set to participating 2017-10-11 17:10:28,470 | INFO | node.py ( 510) | start_participating | Node1 started participating 2017-10-11 17:10:28,471 | DEBUG | node.py (2608) | send | Node1 sending message VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [(0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 13, 'B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr'), (2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu')]} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,471 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node4 : VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,471 | DEBUG | message_processor.py ( 29) | discard | Node1 discarding message VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} because it already decided primary which is Node2:0 2017-10-11 17:10:28,471 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 6 msgs to Node3 into one transmission 2017-10-11 17:10:28,471 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', b'{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"action":"complete","version":"1.1.40"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"}', b'{"viewNo":1,"name":"Node2","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"op":"VIEW_CHANGE_DONE"}']) 2017-10-11 17:10:28,471 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node3: b'{"messages":["{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"reqId\\":1507741828468646,\\"signature\\":\\"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm\\",\\"operation\\":{\\"type\\":\\"110\\",\\"data\\":{\\"action\\":\\"complete\\",\\"version\\":\\"1.1.40\\"},\\"signature\\":\\"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ\\"},\\"identifier\\":\\"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv\\"}","{\\"viewNo\\":1,\\"name\\":\\"Node2\\",\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,13,\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\"],[2,3,\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\"]],\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,472 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"reqId\\":1507741828468646,\\"signature\\":\\"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm\\",\\"operation\\":{\\"type\\":\\"110\\",\\"data\\":{\\"action\\":\\"complete\\",\\"version\\":\\"1.1.40\\"},\\"signature\\":\\"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ\\"},\\"identifier\\":\\"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv\\"}","{\\"viewNo\\":1,\\"name\\":\\"Node2\\",\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,13,\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\"],[2,3,\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\"]],\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' to Node3 2017-10-11 17:10:28,475 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 6 msgs to Node2 into one transmission 2017-10-11 17:10:28,475 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', b'{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"action":"complete","version":"1.1.40"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"}', b'{"viewNo":1,"name":"Node2","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"op":"VIEW_CHANGE_DONE"}']) 2017-10-11 17:10:28,475 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node2: b'{"messages":["{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"reqId\\":1507741828468646,\\"signature\\":\\"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm\\",\\"operation\\":{\\"type\\":\\"110\\",\\"data\\":{\\"action\\":\\"complete\\",\\"version\\":\\"1.1.40\\"},\\"signature\\":\\"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ\\"},\\"identifier\\":\\"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv\\"}","{\\"viewNo\\":1,\\"name\\":\\"Node2\\",\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,13,\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\"],[2,3,\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\"]],\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,475 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"reqId\\":1507741828468646,\\"signature\\":\\"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm\\",\\"operation\\":{\\"type\\":\\"110\\",\\"data\\":{\\"action\\":\\"complete\\",\\"version\\":\\"1.1.40\\"},\\"signature\\":\\"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ\\"},\\"identifier\\":\\"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv\\"}","{\\"viewNo\\":1,\\"name\\":\\"Node2\\",\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,13,\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\"],[2,3,\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\"]],\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' to Node2 2017-10-11 17:10:28,476 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 6 msgs to Node4 into one transmission 2017-10-11 17:10:28,476 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', b'{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_REQUEST"}', b'{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"action":"complete","version":"1.1.40"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"}', b'{"viewNo":1,"name":"Node2","ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"op":"VIEW_CHANGE_DONE"}']) 2017-10-11 17:10:28,476 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node4: b'{"messages":["{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"reqId\\":1507741828468646,\\"signature\\":\\"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm\\",\\"operation\\":{\\"type\\":\\"110\\",\\"data\\":{\\"action\\":\\"complete\\",\\"version\\":\\"1.1.40\\"},\\"signature\\":\\"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ\\"},\\"identifier\\":\\"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv\\"}","{\\"viewNo\\":1,\\"name\\":\\"Node2\\",\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,13,\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\"],[2,3,\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\"]],\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,476 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":2},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"msg_type\\":\\"LEDGER_STATUS\\",\\"params\\":{\\"ledgerId\\":1},\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"reqId\\":1507741828468646,\\"signature\\":\\"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm\\",\\"operation\\":{\\"type\\":\\"110\\",\\"data\\":{\\"action\\":\\"complete\\",\\"version\\":\\"1.1.40\\"},\\"signature\\":\\"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ\\"},\\"identifier\\":\\"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv\\"}","{\\"viewNo\\":1,\\"name\\":\\"Node2\\",\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,13,\\"B2WPnH16xNgc2or78LRvBb8FmmgB9gx15yB3ww4u36Mr\\"],[2,3,\\"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu\\"]],\\"op\\":\\"VIEW_CHANGE_DONE\\"}"],"signature":null,"op":"BATCH"}' to Node4 2017-10-11 17:10:28,477 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:10:28,477 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"op":"CURRENT_STATE","viewNo":1}', '{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"viewNo":null}', '{"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ledgerId":1,"ppSeqNo":2,"op":"LEDGER_STATUS","txnSeqNo":13,"viewNo":0}', '{"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ledgerId":2,"ppSeqNo":4,"op":"LEDGER_STATUS","txnSeqNo":3,"viewNo":0}']} 2017-10-11 17:10:28,477 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"op":"CURRENT_STATE","viewNo":1}', '{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"viewNo":null}', '{"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ledgerId":1,"ppSeqNo":2,"op":"LEDGER_STATUS","txnSeqNo":13,"viewNo":0}', '{"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ledgerId":2,"ppSeqNo":4,"op":"LEDGER_STATUS","txnSeqNo":3,"viewNo":0}']}, 'Node2') 2017-10-11 17:10:28,478 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,13,"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn"],[2,3,"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu"]],"name":"Node2","viewNo":1}],"op":"CURRENT_STATE","viewNo":1}', '{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","ledgerId":0,"ppSeqNo":null,"op":"LEDGER_STATUS","txnSeqNo":4,"viewNo":null}', '{"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ledgerId":1,"ppSeqNo":2,"op":"LEDGER_STATUS","txnSeqNo":13,"viewNo":0}', '{"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ledgerId":2,"ppSeqNo":4,"op":"LEDGER_STATUS","txnSeqNo":3,"viewNo":0}']} 2017-10-11 17:10:28,478 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} 2017-10-11 17:10:28,478 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}], 'op': 'CURRENT_STATE'}, 'Node2') 2017-10-11 17:10:28,478 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} 2017-10-11 17:10:28,478 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,478 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': None, 'txnSeqNo': 4, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:10:28,478 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} 2017-10-11 17:10:28,479 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} 2017-10-11 17:10:28,479 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'txnSeqNo': 13, 'ppSeqNo': 2, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn', 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:10:28,479 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} 2017-10-11 17:10:28,479 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} 2017-10-11 17:10:28,479 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 0, 'txnSeqNo': 3, 'ppSeqNo': 4, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu', 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node2') 2017-10-11 17:10:28,479 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} 2017-10-11 17:10:28,479 | DEBUG | node.py ( 907) | process_current_state_message | Node1 processing current state CURRENT_STATE{'viewNo': 1, 'primary': [{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}]} from Node2 2017-10-11 17:10:28,480 | DEBUG | node.py (1205) | sendToElector | Node1 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]}, 'Node2') 2017-10-11 17:10:28,480 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'txnSeqNo': 4, 'ledgerId': 0, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'} from Node2 2017-10-11 17:10:28,480 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,480 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 0 of size 4 with 4 2017-10-11 17:10:28,480 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node2 2017-10-11 17:10:28,481 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,481 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,481 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node2 2017-10-11 17:10:28,481 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,481 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,482 | DEBUG | primary_selector.py ( 106) | _processViewChangeDoneMessage | Node1's primary selector started processing of ViewChangeDone msg from Node2 : VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} 2017-10-11 17:10:28,482 | DEBUG | message_processor.py ( 29) | discard | Node1 discarding message VIEW_CHANGE_DONE{'viewNo': 1, 'name': 'Node2', 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 13, 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'], [2, 3, '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu']]} because it already decided primary which is Node2:0 2017-10-11 17:10:28,495 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:10:28,496 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0}}', '{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ","data":{"version":"1.1.40","action":"complete"}},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}']} 2017-10-11 17:10:28,496 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0}}', '{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ","data":{"version":"1.1.40","action":"complete"}},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}']}, 'Node3') 2017-10-11 17:10:28,496 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":2},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0}}', '{"msg_type":"LEDGER_STATUS","params":{"ledgerId":1},"op":"MESSAGE_RESPONSE","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0}}', '{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ","data":{"version":"1.1.40","action":"complete"}},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}']} 2017-10-11 17:10:28,496 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,496 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}}, 'Node3') 2017-10-11 17:10:28,497 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,497 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,497 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}}, 'Node3') 2017-10-11 17:10:28,497 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,497 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,497 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}}, 'Node3') 2017-10-11 17:10:28,497 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,498 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,498 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}}, 'Node3') 2017-10-11 17:10:28,498 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,499 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv signature on propagate request 1507741828468646 2017-10-11 17:10:28,499 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,499 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}, 'op': 'PROPAGATE'}, 'Node3') 2017-10-11 17:10:28,499 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,499 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node3 2017-10-11 17:10:28,500 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,500 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,500 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2'} that its ledger of type 2 is latest 2017-10-11 17:10:28,501 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node3 2017-10-11 17:10:28,501 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,501 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,501 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2'} that its ledger of type 2 is latest 2017-10-11 17:10:28,501 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node3 2017-10-11 17:10:28,501 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,502 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,502 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2'} that its ledger of type 1 is latest 2017-10-11 17:10:28,502 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node3 2017-10-11 17:10:28,502 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,502 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,502 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2'} that its ledger of type 1 is latest 2017-10-11 17:10:28,502 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,503 | TRACE | propagator.py ( 165) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'} 2017-10-11 17:10:28,503 | INFO | propagator.py ( 148) | propagate | Node1 propagating request ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646) from client Node1 2017-10-11 17:10:28,503 | DEBUG | node.py (2608) | send | Node1 sending message PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,510 | DEBUG | propagator.py ( 214) | forward | Node1 forwarding request ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646) to 0 replicas 2017-10-11 17:10:28,511 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}' to Node3 2017-10-11 17:10:28,511 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}' to Node3 2017-10-11 17:10:28,512 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}' to Node2 2017-10-11 17:10:28,512 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}' to Node2 2017-10-11 17:10:28,512 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}' to Node4 2017-10-11 17:10:28,512 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"senderClient":"Node1","request":{"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv"},"op":"PROPAGATE"}' to Node4 2017-10-11 17:10:28,513 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 3 messages through listener 2017-10-11 17:10:28,513 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"op":"PROPAGATE","request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm"},"senderClient":"Node1"}']} 2017-10-11 17:10:28,513 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"op":"PROPAGATE","request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm"},"senderClient":"Node1"}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:10:28,514 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"params":{"ledgerId":2},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"params":{"ledgerId":1},"op":"MESSAGE_RESPONSE"}', '{"op":"PROPAGATE","request":{"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","operation":{"type":"110","data":{"version":"1.1.40","action":"complete"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"reqId":1507741828468646,"signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm"},"senderClient":"Node1"}']} 2017-10-11 17:10:28,514 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,514 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}}, 'Node4') 2017-10-11 17:10:28,514 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,514 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,514 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}}, 'Node4') 2017-10-11 17:10:28,514 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,515 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,515 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}}, 'Node4') 2017-10-11 17:10:28,515 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,515 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,515 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'op': 'MESSAGE_RESPONSE', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}}, 'Node4') 2017-10-11 17:10:28,515 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,516 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv signature on propagate request 1507741828468646 2017-10-11 17:10:28,516 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,516 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}, 'op': 'PROPAGATE'}, 'Node4') 2017-10-11 17:10:28,517 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,517 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"senderClient":"Node1","request":{"operation":{"type":"110","data":{"action":"complete","version":"1.1.40"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","reqId":1507741828468646},"op":"PROPAGATE"}']} 2017-10-11 17:10:28,517 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"senderClient":"Node1","request":{"operation":{"type":"110","data":{"action":"complete","version":"1.1.40"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","reqId":1507741828468646},"op":"PROPAGATE"}']}, 'Node2') 2017-10-11 17:10:28,517 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu","ppSeqNo":4,"txnSeqNo":3,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn","ppSeqNo":2,"txnSeqNo":13,"viewNo":0},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE","params":{"ledgerId":1}}', '{"senderClient":"Node1","request":{"operation":{"type":"110","data":{"action":"complete","version":"1.1.40"},"signature":"4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ"},"identifier":"Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv","signature":"4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm","reqId":1507741828468646},"op":"PROPAGATE"}']} 2017-10-11 17:10:28,517 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,517 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 17:10:28,517 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,518 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,518 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 17:10:28,518 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 4, 'viewNo': 0, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'}, 'params': {'ledgerId': 2}} 2017-10-11 17:10:28,518 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,518 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 17:10:28,518 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,518 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,518 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'op': 'MESSAGE_RESPONSE'}, 'Node2') 2017-10-11 17:10:28,519 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'msg': {'ppSeqNo': 2, 'viewNo': 0, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'}, 'params': {'ledgerId': 1}} 2017-10-11 17:10:28,519 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv signature on propagate request 1507741828468646 2017-10-11 17:10:28,520 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'action': 'complete', 'version': '1.1.40'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,524 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'action': 'complete', 'version': '1.1.40'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}, 'op': 'PROPAGATE'}, 'Node2') 2017-10-11 17:10:28,524 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'action': 'complete', 'version': '1.1.40'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,525 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646]], 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'ppSeqNo': 1, 'discarded': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'ledgerId': 2} 2017-10-11 17:10:28,525 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646]], 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'ppSeqNo': 1, 'discarded': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'ledgerId': 2, 'op': 'PREPREPARE'}, 'Node2') 2017-10-11 17:10:28,525 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646]], 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'ppSeqNo': 1, 'discarded': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'ledgerId': 2} 2017-10-11 17:10:28,525 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node4 2017-10-11 17:10:28,526 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,526 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,526 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 2 is latest 2017-10-11 17:10:28,526 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node4 2017-10-11 17:10:28,526 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,526 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,527 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 2 is latest 2017-10-11 17:10:28,527 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node4 2017-10-11 17:10:28,527 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,527 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,527 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 1 is latest 2017-10-11 17:10:28,527 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node4 2017-10-11 17:10:28,528 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,528 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,528 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 1 is latest 2017-10-11 17:10:28,528 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,528 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'} 2017-10-11 17:10:28,528 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'complete'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'} to its replicas since already forwarded 2017-10-11 17:10:28,529 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node2 2017-10-11 17:10:28,529 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,529 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,529 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 2 is latest 2017-10-11 17:10:28,529 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 4, 'txnSeqNo': 3, 'ledgerId': 2, 'merkleRoot': '7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu'} from Node2 2017-10-11 17:10:28,530 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,530 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 2 of size 3 with 3 2017-10-11 17:10:28,530 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 2 is latest 2017-10-11 17:10:28,530 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node2 2017-10-11 17:10:28,530 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,530 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,530 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 1 is latest 2017-10-11 17:10:28,531 | DEBUG | ledger_manager.py ( 246) | processLedgerStatus | Node1 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 2, 'txnSeqNo': 13, 'ledgerId': 1, 'merkleRoot': 'DQdu5vrv4a9kqbELDnF7SiXYcktCHTAQV786DE6hKrxn'} from Node2 2017-10-11 17:10:28,531 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,531 | DEBUG | ledger_manager.py ( 984) | _compareLedger | Node1 comparing its ledger 1 of size 13 with 13 2017-10-11 17:10:28,531 | DEBUG | ledger_manager.py ( 312) | processLedgerStatus | Node1 found out from {'Node3', 'Node2', 'Node4'} that its ledger of type 1 is latest 2017-10-11 17:10:28,531 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGATE{'senderClient': 'Node1', 'request': {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'action': 'complete', 'version': '1.1.40'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}} 2017-10-11 17:10:28,532 | TRACE | propagator.py ( 141) | propagate | Node1 already propagated SafeRequest: {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'action': 'complete', 'version': '1.1.40'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'} 2017-10-11 17:10:28,532 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'reqId': 1507741828468646, 'signature': '4ed6Uw3oKPy8UVB8K3Pk9TiLjkjeREjycjhCnefmMzdjV8A6fZnsP91D5RWoFiq3iz3if4FjosezWWk11JPJstgm', 'operation': {'type': '110', 'data': {'action': 'complete', 'version': '1.1.40'}, 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ'}, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'} to its replicas since already forwarded 2017-10-11 17:10:28,536 | DEBUG | replica.py ( 823) | processPrePrepare | Node1:0 received PRE-PREPARE(1, 1) from Node2:0 at 2270913.99434995 2017-10-11 17:10:28,537 | DEBUG | replica.py (1034) | validate_pre_prepare | Node1:0 state root before processing PREPREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)], 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'ppSeqNo': 1, 'discarded': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'ledgerId': 2} is b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f', 7243ccvhfvHyHtPipBrTmwRrpp4LdNT2V6zeEuFUZsXu 2017-10-11 17:10:28,538 | DEBUG | replica.py ( 934) | doPrepare | Node1:0 Sending PREPARE(1, 1) at 2270913.995874036 2017-10-11 17:10:28,543 | DEBUG | replica.py (2193) | send | Node1:0 sending Prepare 2017-10-11 17:10:28,543 | TRACE | replica.py (2194) | send | Node1:0 sending PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} 2017-10-11 17:10:28,543 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} 2017-10-11 17:10:28,543 | DEBUG | replica.py ( 614) | trackBatches | Node1:0 tracking batch for PREPREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)], 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'ppSeqNo': 1, 'discarded': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'ledgerId': 2} with state root b'\xbc q\xa4\xde\x84o(W\x02D\x7f%\x89\xdd\x166x\xe0\x97*\x8a\x1b\r(\xb0N\xd5\xc0\x94T\x7f' 2017-10-11 17:10:28,544 | DEBUG | replica.py ( 842) | processPrePrepare | Node1:0 processed incoming PRE-PREPARE(1, 1) 2017-10-11 17:10:28,544 | DEBUG | node.py (2608) | send | Node1 sending message PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,544 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 6 messages through listener 2017-10-11 17:10:28,545 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646]], 'stateRootHash': None, 'ppSeqNo': 1, 'discarded': 1, 'instId': 1, 'txnRootHash': None, 'ledgerId': 2} 2017-10-11 17:10:28,545 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646]], 'ledgerId': 2, 'ppSeqNo': 1, 'discarded': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None, 'op': 'PREPREPARE'}, 'Node3') 2017-10-11 17:10:28,545 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [['Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646]], 'stateRootHash': None, 'ppSeqNo': 1, 'discarded': 1, 'instId': 1, 'txnRootHash': None, 'ledgerId': 2} 2017-10-11 17:10:28,545 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} 2017-10-11 17:10:28,545 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node2') 2017-10-11 17:10:28,545 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} 2017-10-11 17:10:28,545 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: BATCH{'signature': None, 'messages': ['{"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","ppSeqNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","op":"PREPARE","viewNo":1,"ppTime":1507741828,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"op":"COMMIT","instId":0,"viewNo":1,"ppSeqNo":1}', '{"txnRootHash":null,"ppSeqNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","op":"PREPARE","viewNo":1,"ppTime":1507741828,"stateRootHash":null,"instId":1}']} 2017-10-11 17:10:28,546 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'messages': ['{"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","ppSeqNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","op":"PREPARE","viewNo":1,"ppTime":1507741828,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"op":"COMMIT","instId":0,"viewNo":1,"ppSeqNo":1}', '{"txnRootHash":null,"ppSeqNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","op":"PREPARE","viewNo":1,"ppTime":1507741828,"stateRootHash":null,"instId":1}'], 'signature': None, 'op': 'BATCH'}, 'Node4') 2017-10-11 17:10:28,546 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","ppSeqNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","op":"PREPARE","viewNo":1,"ppTime":1507741828,"stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","instId":0}', '{"op":"COMMIT","instId":0,"viewNo":1,"ppSeqNo":1}', '{"txnRootHash":null,"ppSeqNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","op":"PREPARE","viewNo":1,"ppTime":1507741828,"stateRootHash":null,"instId":1}']} 2017-10-11 17:10:28,547 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} 2017-10-11 17:10:28,547 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:10:28,547 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} 2017-10-11 17:10:28,547 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} 2017-10-11 17:10:28,547 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppSeqNo': 1, 'instId': 0, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:10:28,547 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} 2017-10-11 17:10:28,548 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} 2017-10-11 17:10:28,548 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node4') 2017-10-11 17:10:28,548 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} 2017-10-11 17:10:28,548 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} 2017-10-11 17:10:28,548 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'op': 'PREPARE'}, 'Node3') 2017-10-11 17:10:28,548 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} 2017-10-11 17:10:28,548 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} 2017-10-11 17:10:28,548 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppSeqNo': 1, 'instId': 1, 'op': 'COMMIT'}, 'Node4') 2017-10-11 17:10:28,549 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} 2017-10-11 17:10:28,549 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: BATCH{'signature': None, 'messages': ['{"viewNo":1,"instId":0,"op":"COMMIT","ppSeqNo":1}', '{"viewNo":1,"instId":1,"op":"COMMIT","ppSeqNo":1}']} 2017-10-11 17:10:28,549 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"viewNo":1,"instId":0,"op":"COMMIT","ppSeqNo":1}', '{"viewNo":1,"instId":1,"op":"COMMIT","ppSeqNo":1}']}, 'Node3') 2017-10-11 17:10:28,549 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"viewNo":1,"instId":0,"op":"COMMIT","ppSeqNo":1}', '{"viewNo":1,"instId":1,"op":"COMMIT","ppSeqNo":1}']} 2017-10-11 17:10:28,549 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} 2017-10-11 17:10:28,549 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppSeqNo': 1, 'instId': 0, 'op': 'COMMIT'}, 'Node3') 2017-10-11 17:10:28,549 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} 2017-10-11 17:10:28,549 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} 2017-10-11 17:10:28,549 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'viewNo': 1, 'ppSeqNo': 1, 'instId': 1, 'op': 'COMMIT'}, 'Node3') 2017-10-11 17:10:28,550 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} 2017-10-11 17:10:28,550 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":0,"ppSeqNo":1,"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","op":"PREPARE"}' to Node3 2017-10-11 17:10:28,551 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":0,"ppSeqNo":1,"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","op":"PREPARE"}' to Node3 2017-10-11 17:10:28,551 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":0,"ppSeqNo":1,"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","op":"PREPARE"}' to Node2 2017-10-11 17:10:28,551 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":0,"ppSeqNo":1,"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","op":"PREPARE"}' to Node2 2017-10-11 17:10:28,551 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":0,"ppSeqNo":1,"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","op":"PREPARE"}' to Node4 2017-10-11 17:10:28,551 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":0,"ppSeqNo":1,"txnRootHash":"Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw","stateRootHash":"DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA","op":"PREPARE"}' to Node4 2017-10-11 17:10:28,552 | DEBUG | replica.py ( 953) | doCommit | Node1:0 Sending COMMIT(1, 1) at 2270914.009748622 2017-10-11 17:10:28,552 | DEBUG | replica.py (2193) | send | Node1:0 sending Commit 2017-10-11 17:10:28,552 | TRACE | replica.py (2194) | send | Node1:0 sending COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} 2017-10-11 17:10:28,552 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} commits where f is 1 2017-10-11 17:10:28,552 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (1, 1) 2017-10-11 17:10:28,552 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(1, 1) from Node4:0 2017-10-11 17:10:28,553 | DEBUG | replica.py ( 929) | tryOrder | Node1:0 cannot return request to node: no quorum (3): COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} commits where f is 1 2017-10-11 17:10:28,553 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(1, 1) 2017-10-11 17:10:28,553 | DEBUG | replica.py ( 917) | tryCommit | Node1:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA'} 2017-10-11 17:10:28,553 | DEBUG | replica.py ( 876) | processPrepare | Node1:0 processed incoming PREPARE (1, 1) 2017-10-11 17:10:28,553 | DEBUG | replica.py ( 894) | processCommit | Node1:0 received COMMIT(1, 1) from Node3:0 2017-10-11 17:10:28,553 | TRACE | replica.py ( 925) | tryOrder | Node1:0 returning request to node 2017-10-11 17:10:28,553 | DEBUG | replica.py (1444) | doOrder | Node1:0 ordering COMMIT (1, 1) 2017-10-11 17:10:28,553 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:0 set last ordered as (1, 1) 2017-10-11 17:10:28,554 | DEBUG | replica.py (2193) | send | Node1:0 sending Ordered 2017-10-11 17:10:28,554 | TRACE | replica.py (2194) | send | Node1:0 sending ORDERED{'viewNo': 1, 'ppTime': 1507741828, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)], 'stateRootHash': 'DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA', 'ppSeqNo': 1, 'instId': 0, 'txnRootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'ledgerId': 2} 2017-10-11 17:10:28,554 | INFO | replica.py (1483) | order_3pc_key | Node1:0 ordered batch request, view no 1, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)], discarded [] 2017-10-11 17:10:28,554 | DEBUG | replica.py (1556) | _newCheckpointState | Node1:0 adding new checkpoint state for (1, 100) 2017-10-11 17:10:28,554 | DEBUG | replica.py ( 905) | processCommit | Node1:0 processed incoming COMMIT(1, 1) 2017-10-11 17:10:28,554 | DEBUG | replica.py ( 823) | processPrePrepare | Node1:1 received PRE-PREPARE(1, 1) from Node3:1 at 2270914.012055558 2017-10-11 17:10:28,555 | DEBUG | replica.py ( 934) | doPrepare | Node1:1 Sending PREPARE(1, 1) at 2270914.012459446 2017-10-11 17:10:28,555 | DEBUG | replica.py (2193) | send | Node1:1 sending Prepare 2017-10-11 17:10:28,555 | TRACE | replica.py (2194) | send | Node1:1 sending PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} 2017-10-11 17:10:28,555 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} 2017-10-11 17:10:28,555 | DEBUG | replica.py ( 614) | trackBatches | Node1:1 tracking batch for PREPREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)], 'stateRootHash': None, 'ppSeqNo': 1, 'discarded': 1, 'instId': 1, 'txnRootHash': None, 'ledgerId': 2} with state root None 2017-10-11 17:10:28,555 | DEBUG | replica.py ( 842) | processPrePrepare | Node1:1 processed incoming PRE-PREPARE(1, 1) 2017-10-11 17:10:28,556 | DEBUG | replica.py ( 953) | doCommit | Node1:1 Sending COMMIT(1, 1) at 2270914.013553646 2017-10-11 17:10:28,556 | DEBUG | replica.py (2193) | send | Node1:1 sending Commit 2017-10-11 17:10:28,556 | TRACE | replica.py (2194) | send | Node1:1 sending COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} 2017-10-11 17:10:28,556 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} commits where f is 1 2017-10-11 17:10:28,556 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (1, 1) 2017-10-11 17:10:28,556 | DEBUG | replica.py ( 917) | tryCommit | Node1:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} 2017-10-11 17:10:28,556 | DEBUG | replica.py ( 876) | processPrepare | Node1:1 processed incoming PREPARE (1, 1) 2017-10-11 17:10:28,557 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(1, 1) from Node4:1 2017-10-11 17:10:28,557 | DEBUG | replica.py ( 929) | tryOrder | Node1:1 cannot return request to node: no quorum (3): COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} commits where f is 1 2017-10-11 17:10:28,557 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(1, 1) 2017-10-11 17:10:28,557 | DEBUG | replica.py ( 894) | processCommit | Node1:1 received COMMIT(1, 1) from Node3:1 2017-10-11 17:10:28,557 | TRACE | replica.py ( 925) | tryOrder | Node1:1 returning request to node 2017-10-11 17:10:28,557 | DEBUG | replica.py (1444) | doOrder | Node1:1 ordering COMMIT (1, 1) 2017-10-11 17:10:28,557 | DEBUG | replica.py ( 323) | last_ordered_3pc | Node1:1 set last ordered as (1, 1) 2017-10-11 17:10:28,557 | DEBUG | replica.py (2193) | send | Node1:1 sending Ordered 2017-10-11 17:10:28,557 | TRACE | replica.py (2194) | send | Node1:1 sending ORDERED{'viewNo': 1, 'ppTime': 1507741828, 'reqIdr': [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)], 'stateRootHash': None, 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'ledgerId': 2} 2017-10-11 17:10:28,558 | INFO | replica.py (1483) | order_3pc_key | Node1:1 ordered batch request, view no 1, ppSeqNo 1, ledger 2, state root None, txn root None, requests ordered [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)], discarded [] 2017-10-11 17:10:28,558 | DEBUG | replica.py (1556) | _newCheckpointState | Node1:1 adding new checkpoint state for (1, 100) 2017-10-11 17:10:28,558 | DEBUG | replica.py ( 905) | processCommit | Node1:1 processed incoming COMMIT(1, 1) 2017-10-11 17:10:28,558 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,558 | DEBUG | node.py (1847) | processOrdered | Node1 executing Ordered batch 1 1 of 1 requests 2017-10-11 17:10:28,561 | DEBUG | ledger.py ( 52) | commitTxns | Committed 1 txns, 0 are uncommitted 2017-10-11 17:10:28,561 | DEBUG | node.py (2386) | sendReplyToClient | Node1 sending reply for ('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646) to client 2017-10-11 17:10:28,562 | DEBUG | zstack.py ( 744) | transmitThroughListener | Node1C not sending message {'result': {'txnTime': 1507741828, 'data': {'version': '1.1.40', 'action': 'complete'}, 'rootHash': 'Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw', 'auditPath': ['FFusKBPJVV3mVuKVMNsZ1KcJ77B3EPoyMZoXiPdLTKdv', '4EqhrnCKJRxQSAQ8cjfpZUr71oCijptQWfbLq59MwGE7'], 'signature': '4KPg3heM8TgwKUPvegXDuLkE8UYiUNP162dfCtncKFwgmuQo4hgCgV3ptu8KWVNFQd68w9mJkhQFcdTY9YgqProZ', 'reqId': 1507741828468646, 'type': '110', 'seqNo': 4, 'identifier': 'Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv'}, 'op': 'REPLY'} to b'Node1' 2017-10-11 17:10:28,562 | DEBUG | zstack.py ( 745) | transmitThroughListener | This is a temporary workaround for not being able to disconnect a ROUTER's remote 2017-10-11 17:10:28,562 | INFO | node.py (2290) | executeBatch | Node1 committed batch request, view no 1, ppSeqNo 1, ledger 2, state root DfNLmH4DAHTKv63YPFJzuRdeEtVwF5RtVnvKYHd8iLEA, txn root Ee5vA4vbk6gKvwoVh471Z3NsPFyKiYXbCjf3sFHi2nZw, requests: [('Gw6pDLhcBcoQesN72qfotTgFa7cbuqZpkX3Xo6pLhPhv', 1507741828468646)] 2017-10-11 17:10:28,562 | DEBUG | node.py (2305) | executeBatch | Node1 storing 3PC key (1, 1) for ledger 2 range (4, 4) 2017-10-11 17:10:28,562 | DEBUG | node.py (2608) | send | Node1 sending message PREPARE{'viewNo': 1, 'ppTime': 1507741828, 'digest': '9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171', 'ppSeqNo': 1, 'instId': 1, 'txnRootHash': None, 'stateRootHash': None} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,562 | DEBUG | node.py (2608) | send | Node1 sending message COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:10:28,563 | TRACE | node.py (1858) | processOrdered | Node1 got ordered requests from backup replica 1 2017-10-11 17:10:28,563 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:28,563 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:10:28,564 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":1,"op":"COMMIT","viewNo":1}', '{"instId":1,"ppSeqNo":1,"op":"COMMIT","viewNo":1}']} 2017-10-11 17:10:28,564 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'op': 'BATCH', 'signature': None, 'messages': ['{"instId":0,"ppSeqNo":1,"op":"COMMIT","viewNo":1}', '{"instId":1,"ppSeqNo":1,"op":"COMMIT","viewNo":1}']}, 'Node2') 2017-10-11 17:10:28,564 | DEBUG | node.py (1272) | unpackNodeMsg | Node1 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":1,"op":"COMMIT","viewNo":1}', '{"instId":1,"ppSeqNo":1,"op":"COMMIT","viewNo":1}']} 2017-10-11 17:10:28,564 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} 2017-10-11 17:10:28,564 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'viewNo': 1, 'instId': 0, 'op': 'COMMIT'}, 'Node2') 2017-10-11 17:10:28,564 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} 2017-10-11 17:10:28,564 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node2: COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} 2017-10-11 17:10:28,565 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'ppSeqNo': 1, 'viewNo': 1, 'instId': 1, 'op': 'COMMIT'}, 'Node2') 2017-10-11 17:10:28,565 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} 2017-10-11 17:10:28,565 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 3 msgs to Node3 into one transmission 2017-10-11 17:10:28,565 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"viewNo":1,"ppSeqNo":1,"instId":0,"op":"COMMIT"}', b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":1,"ppSeqNo":1,"txnRootHash":null,"stateRootHash":null,"op":"PREPARE"}', b'{"viewNo":1,"ppSeqNo":1,"instId":1,"op":"COMMIT"}']) 2017-10-11 17:10:28,565 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node3: b'{"messages":["{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":0,\\"op\\":\\"COMMIT\\"}","{\\"viewNo\\":1,\\"digest\\":\\"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171\\",\\"ppTime\\":1507741828,\\"instId\\":1,\\"ppSeqNo\\":1,\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\"}","{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":1,\\"op\\":\\"COMMIT\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,565 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":0,\\"op\\":\\"COMMIT\\"}","{\\"viewNo\\":1,\\"digest\\":\\"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171\\",\\"ppTime\\":1507741828,\\"instId\\":1,\\"ppSeqNo\\":1,\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\"}","{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":1,\\"op\\":\\"COMMIT\\"}"],"signature":null,"op":"BATCH"}' to Node3 2017-10-11 17:10:28,566 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 3 msgs to Node2 into one transmission 2017-10-11 17:10:28,566 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"viewNo":1,"ppSeqNo":1,"instId":0,"op":"COMMIT"}', b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":1,"ppSeqNo":1,"txnRootHash":null,"stateRootHash":null,"op":"PREPARE"}', b'{"viewNo":1,"ppSeqNo":1,"instId":1,"op":"COMMIT"}']) 2017-10-11 17:10:28,566 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node2: b'{"messages":["{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":0,\\"op\\":\\"COMMIT\\"}","{\\"viewNo\\":1,\\"digest\\":\\"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171\\",\\"ppTime\\":1507741828,\\"instId\\":1,\\"ppSeqNo\\":1,\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\"}","{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":1,\\"op\\":\\"COMMIT\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,566 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":0,\\"op\\":\\"COMMIT\\"}","{\\"viewNo\\":1,\\"digest\\":\\"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171\\",\\"ppTime\\":1507741828,\\"instId\\":1,\\"ppSeqNo\\":1,\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\"}","{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":1,\\"op\\":\\"COMMIT\\"}"],"signature":null,"op":"BATCH"}' to Node2 2017-10-11 17:10:28,566 | DEBUG | batched.py ( 100) | flushOutBoxes | Node1 batching 3 msgs to Node4 into one transmission 2017-10-11 17:10:28,567 | TRACE | batched.py ( 101) | flushOutBoxes | messages: deque([b'{"viewNo":1,"ppSeqNo":1,"instId":0,"op":"COMMIT"}', b'{"viewNo":1,"digest":"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171","ppTime":1507741828,"instId":1,"ppSeqNo":1,"txnRootHash":null,"stateRootHash":null,"op":"PREPARE"}', b'{"viewNo":1,"ppSeqNo":1,"instId":1,"op":"COMMIT"}']) 2017-10-11 17:10:28,567 | TRACE | batched.py ( 110) | flushOutBoxes | Node1 sending payload to Node4: b'{"messages":["{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":0,\\"op\\":\\"COMMIT\\"}","{\\"viewNo\\":1,\\"digest\\":\\"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171\\",\\"ppTime\\":1507741828,\\"instId\\":1,\\"ppSeqNo\\":1,\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\"}","{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":1,\\"op\\":\\"COMMIT\\"}"],"signature":null,"op":"BATCH"}' 2017-10-11 17:10:28,567 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"messages":["{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":0,\\"op\\":\\"COMMIT\\"}","{\\"viewNo\\":1,\\"digest\\":\\"9d617548384ccf6c1f17e5d7d5bd4d2d1b03325dafd8844bea4c566a86ba0171\\",\\"ppTime\\":1507741828,\\"instId\\":1,\\"ppSeqNo\\":1,\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\"}","{\\"viewNo\\":1,\\"ppSeqNo\\":1,\\"instId\\":1,\\"op\\":\\"COMMIT\\"}"],"signature":null,"op":"BATCH"}' to Node4 2017-10-11 17:10:28,568 | TRACE | message_processor.py ( 29) | discard | Node1:0 discarding message COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 0} because already ordered 3 phase message 2017-10-11 17:10:28,568 | TRACE | message_processor.py ( 29) | discard | Node1:1 discarding message COMMIT{'viewNo': 1, 'ppSeqNo': 1, 'instId': 1} because already ordered 3 phase message 2017-10-11 17:10:31,293 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 4 2017-10-11 17:10:38,231 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 1 2017-10-11 17:10:38,231 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:10:38,231 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:10:38,232 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:10:38,232 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:38,232 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:10:38,232 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:10:38,232 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 7 to run in 10 seconds 2017-10-11 17:10:43,297 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:10:43,297 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:10:43,297 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:10:43,298 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:10:48,240 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 7 2017-10-11 17:10:48,241 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:10:48,241 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:10:48,241 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:10:48,241 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:48,241 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:10:48,241 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:10:48,241 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 8 to run in 10 seconds 2017-10-11 17:10:58,244 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 8 2017-10-11 17:10:58,244 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:10:58,244 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:10:58,244 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:10:58,244 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:10:58,244 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:10:58,244 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:10:58,245 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 9 to run in 10 seconds 2017-10-11 17:10:58,302 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:10:58,303 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:10:58,303 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:10:58,303 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:11:08,248 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 9 2017-10-11 17:11:08,248 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:11:08,248 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:11:08,248 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:11:08,248 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:11:08,248 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:11:08,249 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:11:08,249 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 10 to run in 10 seconds 2017-10-11 17:11:13,312 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:11:13,312 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:11:13,313 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:11:13,313 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:11:18,260 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 10 2017-10-11 17:11:18,260 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:11:18,260 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:11:18,260 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:11:18,260 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:11:18,260 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:11:18,260 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:11:18,260 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 11 to run in 10 seconds 2017-10-11 17:11:28,237 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 2 2017-10-11 17:11:28,237 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:11:28,237 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:11:28,237 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 12 to run in 60 seconds 2017-10-11 17:11:28,237 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 1 2017-10-11 17:11:28,237 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:11:28,237 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 2 to run in 60 seconds 2017-10-11 17:11:28,261 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 11 2017-10-11 17:11:28,262 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:11:28,262 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:11:28,262 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:11:28,262 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:11:28,262 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:11:28,262 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:11:28,262 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 13 to run in 10 seconds 2017-10-11 17:11:28,289 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 5 2017-10-11 17:11:28,290 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 14 to run in 60 seconds 2017-10-11 17:11:28,290 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action propose_view_change with id 3 2017-10-11 17:11:28,290 | TRACE | node.py (2097) | propose_view_change | The primary is already connected so view change will not be proposed 2017-10-11 17:11:28,313 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:11:28,314 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:11:28,314 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:11:28,314 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:11:28,399 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action _check_view_change_completed with id 6 2017-10-11 17:11:28,399 | DEBUG | node.py (1011) | _check_view_change_completed | Node1 running the scheduled check for view change completion 2017-10-11 17:11:28,400 | DEBUG | node.py (1013) | _check_view_change_completed | Node1 already completion view change 2017-10-11 17:11:38,262 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 13 2017-10-11 17:11:38,263 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:11:38,263 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:11:38,263 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:11:38,263 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:11:38,263 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:11:38,263 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:11:38,263 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 15 to run in 10 seconds 2017-10-11 17:11:43,323 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:11:43,323 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:11:43,323 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:11:43,324 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:11:48,272 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 15 2017-10-11 17:11:48,272 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:11:48,272 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:11:48,272 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:11:48,272 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:11:48,272 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:11:48,272 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:11:48,272 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 16 to run in 10 seconds 2017-10-11 17:11:58,275 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 16 2017-10-11 17:11:58,275 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:11:58,275 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:11:58,275 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:11:58,275 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:11:58,275 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:11:58,275 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:11:58,275 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 17 to run in 10 seconds 2017-10-11 17:11:58,334 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:11:58,334 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:11:58,334 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:11:58,334 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:12:08,287 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 17 2017-10-11 17:12:08,288 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:12:08,288 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:12:08,288 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:12:08,288 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:12:08,288 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:12:08,289 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:12:08,289 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 18 to run in 10 seconds 2017-10-11 17:12:13,335 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:12:13,335 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:12:13,335 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:12:13,336 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:12:18,291 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 18 2017-10-11 17:12:18,292 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:12:18,292 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:12:18,292 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:12:18,292 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:12:18,292 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:12:18,292 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:12:18,292 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 19 to run in 10 seconds 2017-10-11 17:12:28,247 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 12 2017-10-11 17:12:28,247 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:12:28,247 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:12:28,247 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 20 to run in 60 seconds 2017-10-11 17:12:28,247 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 2 2017-10-11 17:12:28,248 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:12:28,248 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 3 to run in 60 seconds 2017-10-11 17:12:28,296 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 19 2017-10-11 17:12:28,296 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:12:28,296 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:12:28,296 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:12:28,297 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:12:28,297 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:12:28,297 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:12:28,297 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 21 to run in 10 seconds 2017-10-11 17:12:28,297 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 14 2017-10-11 17:12:28,297 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 22 to run in 60 seconds 2017-10-11 17:12:28,344 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:12:28,344 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:12:28,345 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:12:28,345 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:12:38,300 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 21 2017-10-11 17:12:38,300 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:12:38,300 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:12:38,300 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:12:38,300 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:12:38,300 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:12:38,300 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:12:38,301 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 23 to run in 10 seconds 2017-10-11 17:12:43,351 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:12:43,352 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:12:43,352 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:12:43,352 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:12:48,308 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 23 2017-10-11 17:12:48,308 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:12:48,308 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:12:48,308 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:12:48,308 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:12:48,308 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:12:48,308 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:12:48,308 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 24 to run in 10 seconds 2017-10-11 17:12:58,319 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 24 2017-10-11 17:12:58,319 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:12:58,319 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:12:58,319 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:12:58,319 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:12:58,319 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:12:58,319 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:12:58,320 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 25 to run in 10 seconds 2017-10-11 17:12:58,355 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:12:58,355 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:12:58,355 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:12:58,356 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:13:08,324 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 25 2017-10-11 17:13:08,324 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:13:08,325 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:13:08,325 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:13:08,325 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:13:08,325 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:13:08,325 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:13:08,325 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 26 to run in 10 seconds 2017-10-11 17:13:13,364 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:13:13,364 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:13:13,365 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:13:13,365 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:13:18,334 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 26 2017-10-11 17:13:18,334 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:13:18,334 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:13:18,334 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:13:18,335 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:13:18,335 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:13:18,335 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:13:18,335 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 27 to run in 10 seconds 2017-10-11 17:13:28,252 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 20 2017-10-11 17:13:28,253 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:13:28,253 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:13:28,253 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 28 to run in 60 seconds 2017-10-11 17:13:28,253 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 3 2017-10-11 17:13:28,254 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:13:28,254 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 4 to run in 60 seconds 2017-10-11 17:13:28,305 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 22 2017-10-11 17:13:28,306 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 29 to run in 60 seconds 2017-10-11 17:13:28,341 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 27 2017-10-11 17:13:28,341 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:13:28,341 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:13:28,342 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:13:28,342 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:13:28,342 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:13:28,342 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:13:28,342 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 30 to run in 10 seconds 2017-10-11 17:13:28,366 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:13:28,366 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:13:28,367 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:13:28,367 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:13:38,351 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 30 2017-10-11 17:13:38,351 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:13:38,351 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:13:38,351 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:13:38,351 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:13:38,351 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:13:38,351 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:13:38,352 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 31 to run in 10 seconds 2017-10-11 17:13:43,376 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:13:43,376 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:13:43,376 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:13:43,377 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:13:48,361 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 31 2017-10-11 17:13:48,361 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:13:48,361 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:13:48,361 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:13:48,362 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:13:48,362 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:13:48,362 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:13:48,362 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 32 to run in 10 seconds 2017-10-11 17:13:58,362 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 32 2017-10-11 17:13:58,362 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:13:58,362 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:13:58,363 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:13:58,363 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:13:58,363 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:13:58,363 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:13:58,363 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 33 to run in 10 seconds 2017-10-11 17:13:58,389 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:13:58,389 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:13:58,389 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:13:58,390 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:14:08,367 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 33 2017-10-11 17:14:08,368 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:14:08,368 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:14:08,369 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:14:08,369 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:14:08,369 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:14:08,369 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:14:08,369 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 34 to run in 10 seconds 2017-10-11 17:14:13,397 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:14:13,397 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:14:13,397 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:14:13,398 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:14:18,371 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 34 2017-10-11 17:14:18,371 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:14:18,371 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:14:18,371 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:14:18,371 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:14:18,372 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:14:18,372 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:14:18,372 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 35 to run in 10 seconds 2017-10-11 17:14:28,256 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkNodeRequestSpike with id 28 2017-10-11 17:14:28,256 | DEBUG | node.py (1999) | checkNodeRequestSpike | Node1 checking its request amount 2017-10-11 17:14:28,256 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:14:28,256 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkNodeRequestSpike with id 36 to run in 60 seconds 2017-10-11 17:14:28,256 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 4 2017-10-11 17:14:28,257 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike 2017-10-11 17:14:28,257 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 5 to run in 60 seconds 2017-10-11 17:14:28,314 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action dump_json_file with id 29 2017-10-11 17:14:28,315 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action dump_json_file with id 37 to run in 60 seconds 2017-10-11 17:14:28,372 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 35 2017-10-11 17:14:28,372 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:14:28,372 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:14:28,373 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:14:28,373 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:14:28,373 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:14:28,373 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:14:28,373 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 38 to run in 10 seconds 2017-10-11 17:14:28,397 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:14:28,397 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:14:28,397 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:14:28,398 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:14:38,377 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 38 2017-10-11 17:14:38,378 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:14:38,378 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:14:38,378 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:14:38,378 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:14:38,378 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:14:38,378 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:14:38,379 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 39 to run in 10 seconds 2017-10-11 17:14:43,401 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:14:43,401 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:14:43,401 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:14:43,402 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:14:48,387 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 39 2017-10-11 17:14:48,387 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:14:48,388 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:14:48,388 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:14:48,388 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:14:48,388 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:14:48,388 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:14:48,388 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 40 to run in 10 seconds 2017-10-11 17:14:58,398 | TRACE | has_action_queue.py ( 66) | _serviceActions | Node1 running action checkPerformance with id 40 2017-10-11 17:14:58,398 | TRACE | node.py (1978) | checkPerformance | Node1 checking its performance 2017-10-11 17:14:58,398 | DEBUG | notifier_plugin_manager.py ( 74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike 2017-10-11 17:14:58,398 | TRACE | monitor.py ( 344) | isMasterThroughputTooLow | Node1 master throughput ratio 1.0106105669142882 is acceptable. 2017-10-11 17:14:58,398 | TRACE | monitor.py ( 361) | isMasterReqLatencyTooHigh | Node1 found master's latency to be lower than the threshold for all requests. 2017-10-11 17:14:58,398 | TRACE | monitor.py ( 391) | isMasterAvgReqLatencyTooHigh | Node1 found difference between master and backups avg latencies to be acceptable 2017-10-11 17:14:58,398 | DEBUG | node.py (1995) | checkPerformance | Node1's master has higher performance than backups 2017-10-11 17:14:58,398 | TRACE | has_action_queue.py ( 36) | _schedule | Node1 scheduling action checkPerformance with id 41 to run in 10 seconds 2017-10-11 17:14:58,413 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node3 HA(host='10.0.0.4', port=9705) 2017-10-11 17:14:58,413 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node2 HA(host='10.0.0.3', port=9703) 2017-10-11 17:14:58,413 | DEBUG | kit_zstack.py ( 67) | reconcileNodeReg | Node1 matched remote Node4 HA(host='10.0.0.5', port=9707) 2017-10-11 17:14:58,413 | DEBUG | kit_zstack.py ( 50) | maintainConnections | Node1 next check for retries in 15.00 seconds 2017-10-11 17:14:59,442 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 1 messages through listener 2017-10-11 17:14:59,442 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'action': 'in_progress', 'version': '1.1.40'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}, 'Node2') 2017-10-11 17:14:59,442 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox SafeRequest: {'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'action': 'in_progress', 'version': '1.1.40'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'} 2017-10-11 17:14:59,443 | TRACE | propagator.py ( 165) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'action': 'in_progress', 'version': '1.1.40'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'} 2017-10-11 17:14:59,443 | INFO | propagator.py ( 148) | propagate | Node1 propagating request ('8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb', 1507742099432084) from client Node2 2017-10-11 17:14:59,444 | DEBUG | node.py (2608) | send | Node1 sending message PROPAGATE{'senderClient': 'Node2', 'request': {'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'action': 'in_progress', 'version': '1.1.40'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}} to all recipients: ['Node3', 'Node2', 'Node4'] 2017-10-11 17:14:59,444 | DEBUG | propagator.py ( 246) | tryForwarding | Node1 not forwarding request SafeRequest: {'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'action': 'in_progress', 'version': '1.1.40'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'} to its replicas since not finalised 2017-10-11 17:14:59,444 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"senderClient":"Node2","request":{"reqId":1507742099432084,"signature":"5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB","operation":{"type":"110","data":{"action":"in_progress","version":"1.1.40"},"signature":"3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS"},"identifier":"8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"},"op":"PROPAGATE"}' to Node3 2017-10-11 17:14:59,445 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"senderClient":"Node2","request":{"reqId":1507742099432084,"signature":"5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB","operation":{"type":"110","data":{"action":"in_progress","version":"1.1.40"},"signature":"3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS"},"identifier":"8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"},"op":"PROPAGATE"}' to Node3 2017-10-11 17:14:59,445 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"senderClient":"Node2","request":{"reqId":1507742099432084,"signature":"5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB","operation":{"type":"110","data":{"action":"in_progress","version":"1.1.40"},"signature":"3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS"},"identifier":"8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"},"op":"PROPAGATE"}' to Node2 2017-10-11 17:14:59,445 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"senderClient":"Node2","request":{"reqId":1507742099432084,"signature":"5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB","operation":{"type":"110","data":{"action":"in_progress","version":"1.1.40"},"signature":"3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS"},"identifier":"8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"},"op":"PROPAGATE"}' to Node2 2017-10-11 17:14:59,446 | DEBUG | zstack.py ( 723) | transmit | Node1 transmitting message b'{"senderClient":"Node2","request":{"reqId":1507742099432084,"signature":"5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB","operation":{"type":"110","data":{"action":"in_progress","version":"1.1.40"},"signature":"3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS"},"identifier":"8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"},"op":"PROPAGATE"}' to Node4 2017-10-11 17:14:59,446 | TRACE | batched.py ( 96) | flushOutBoxes | Node1 sending msg b'{"senderClient":"Node2","request":{"reqId":1507742099432084,"signature":"5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB","operation":{"type":"110","data":{"action":"in_progress","version":"1.1.40"},"signature":"3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS"},"identifier":"8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb"},"op":"PROPAGATE"}' to Node4 2017-10-11 17:14:59,446 | TRACE | zstack.py ( 479) | _receiveFromListener | Node1 got 2 messages through listener 2017-10-11 17:14:59,448 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb signature on propagate request 1507742099432084 2017-10-11 17:14:59,448 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node3: PROPAGATE{'senderClient': 'Node2', 'request': {'reqId': 1507742099432084, 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}} 2017-10-11 17:14:59,448 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'senderClient': 'Node2', 'request': {'reqId': 1507742099432084, 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}, 'op': 'PROPAGATE'}, 'Node3') 2017-10-11 17:14:59,448 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'senderClient': 'Node2', 'request': {'reqId': 1507742099432084, 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}} 2017-10-11 17:14:59,449 | DEBUG | node.py (2236) | verifySignature | Node1 authenticated 8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb signature on propagate request 1507742099432084 2017-10-11 17:14:59,449 | DEBUG | node.py (1257) | validateNodeMsg | Node1 received node message from Node4: PROPAGATE{'senderClient': 'Node2', 'request': {'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}} 2017-10-11 17:14:59,449 | DEBUG | node.py (1219) | handleOneNodeMsg | Node1 msg validated ({'senderClient': 'Node2', 'request': {'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}, 'op': 'PROPAGATE'}, 'Node4') 2017-10-11 17:14:59,449 | DEBUG | node.py (1286) | postToNodeInBox | Node1 appending to nodeInbox PROPAGATE{'senderClient': 'Node2', 'request': {'reqId': 1507742099432084, 'signature': '5hEHFtUkz1JBGstkZSUKhaeUqJbnjBSLimd7yJWuPbHdpc5iZ8yqXL3TSU5obg9MbeNt2WmnaJyZFQDx5AY3JHVB', 'operation': {'type': '110', 'data': {'version': '1.1.40', 'action': 'in_progress'}, 'signature': '3gvc5cqDb5BFedoR6WCt2MwWeqzvhUSNPEmYTiFbWM41hKVQXqCae8xotWNUPRo1E9c5LN7D4Too7v7P4mr4tXRS'}, 'identifier': '8ECVSk179mjsjKRLWiQtssMLgp6EPhWXtaYyStWPSGAb'}} 2017-10-11 17:14:59,449 | DEBUG | node.py (1757) | processPropagate | Node Node1 received propagated request: PROPAGA