2017-09-21 09:17:25,292 | INFO | node.py (2431) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-09-21 09:17:25,337 | INFO | node.py (2431) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-09-21 09:17:25,385 | INFO | plugin_loader.py ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer 2017-09-21 09:17:25,386 | DISPLAY | replicas.py ( 36) | grow | Node2 added replica Node2:0 to instance 0 (master) 2017-09-21 09:17:25,430 | INFO | node.py (2431) | initStateFromLedger | Node2 found state to be empty, recreating from ledger 2017-09-21 09:17:25,431 | INFO | stacks.py ( 84) | start | CONNECTION: Node2 listening for other nodes at 0.0.0.0:9703 2017-09-21 09:17:25,431 | INFO | node.py ( 621) | start | Node2 first time running... 2017-09-21 09:17:25,432 | INFO | zstack.py ( 590) | connect | CONNECTION: Node2 looking for Node1 at 10.0.0.2:9701 2017-09-21 09:17:25,434 | INFO | zstack.py ( 590) | connect | CONNECTION: Node2 looking for Node3 at 10.0.0.4:9705 2017-09-21 09:17:30,159 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-09-21 09:17:31,443 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-09-21 09:17:31,443 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-09-21 09:17:31,445 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node3 2017-09-21 09:17:31,459 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-09-21 09:17:31,459 | INFO | upgrader.py ( 150) | should_notify_about_upgrade_result | Node's 'Node2' last upgrade txn is None 2017-09-21 09:17:31,459 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-09-21 09:17:31,475 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-09-21 09:17:31,476 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-09-21 09:17:31,476 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-09-21 09:17:31,477 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node2:0 selected primary Node1:0 for instance 0 (view 0) 2017-09-21 09:17:31,477 | INFO | node.py ( 510) | start_participating | Node2 started participating 2017-09-21 09:17:31,477 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node2:0 declares view change 0 as completed for instance 0, new primary is Node1:0, ledger info is [(0, 3, '9pM4rmVSZGgU1muLXeHittuQNM8V9Z3hswV3DZDmb5Rq'), (1, 14, 'CnPftVUXmiwX5XjaV6mHoJkhLajsuQui9ZKYJeULhAPC'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')] 2017-09-21 09:19:15,879 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1505985555857988) from client b'h8K^THiG$X*:fkTsq^bmTi@*ez5N%<]0Kt0M0)^i' 2017-09-21 09:19:15,941 | INFO | replica.py (1483) | order_3pc_key | Node2:0 ordered batch request, view no 0, ppSeqNo 1, ledger 1, state root 8a8ECkXpQBrxdBxmvCNaB238932RjYzPJgS5jqHNy6cu, txn root 29XGdr2VnKcHDtiphHc7MvB7H1ocjZksTbZ3rTbK7udz, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1505985555857988)], discarded [] 2017-09-21 09:19:15,942 | INFO | node.py (2301) | executeBatch | Node2 committed batch request, view no 0, ppSeqNo 1, ledger 1, state root 8a8ECkXpQBrxdBxmvCNaB238932RjYzPJgS5jqHNy6cu, txn root 29XGdr2VnKcHDtiphHc7MvB7H1ocjZksTbZ3rTbK7udz, requests: [('V4SGRU86Z58d6TV7PBUe6f', 1505985555857988)] 2017-09-21 09:20:15,004 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node1 2017-09-21 09:20:15,004 | INFO | node.py ( 823) | onConnsChanged | Node2 lost connection to primary of master 2017-09-21 09:20:17,013 | INFO | node.py (2057) | sendInstanceChange | VIEW CHANGE: Node2 sending an instance change with view_no 1 since Primary of master protocol instance disconnected 2017-09-21 09:20:17,014 | INFO | node.py (2060) | sendInstanceChange | MONITORING: Node2 metrics for monitor: Node2 Monitor metrics:: None Delta: 0.4 Lambda: 60 Omega: 5 instances started: [514530.84430358] ordered request counts: {0: 1} ordered request durations: {0: 0.06310337601462379} master request latencies: {} client avg request latencies: [{'V4SGRU86Z58d6TV7PBUe6f': (1, 0.06310337601462379)}] throughput: {0: 15.84701268230493} master throughput: 15.84701268230493 total requests: 1 avg backup throughput: None master throughput ratio: None 2017-09-21 09:20:17,015 | INFO | node.py (2117) | propose_view_change | Node2 sent view change since was disconnected from primary for too long 2017-09-21 09:20:17,019 | INFO | node.py (1963) | do_view_change_if_possible | VIEW CHANGE: Node2 initiating a view change to 1 from 0 2017-09-21 09:20:17,021 | INFO | node.py (2168) | startViewChange | VIEW CHANGE: Node2 changed to view 1, will start catchup now 2017-09-21 09:20:23,904 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1505985623888664) from client b'h8K^THiG$X*:fkTsq^bmTi@*ez5N%<]0Kt0M0)^i' 2017-09-21 09:20:35,142 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 0 2017-09-21 09:20:35,142 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 0, caught up 0 in total 2017-09-21 09:20:35,145 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-09-21 09:20:35,159 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 2 2017-09-21 09:20:35,160 | INFO | upgrader.py ( 150) | should_notify_about_upgrade_result | Node's 'Node2' last upgrade txn is None 2017-09-21 09:20:35,160 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 2, caught up 0 in total 2017-09-21 09:20:35,178 | INFO | node.py (1489) | preLedgerCatchUp | Node2 reverted 0 batches before starting catch up for ledger 1 2017-09-21 09:20:35,178 | INFO | ledger_manager.py ( 848) | catchupCompleted | CATCH-UP: Node2 completed catching up ledger 1, caught up 0 in total 2017-09-21 09:20:35,178 | INFO | node.py (1537) | allLedgersCaughtUp | CATCH-UP: Node2 does not need any more catchups 2017-09-21 09:20:35,274 | DISPLAY | primary_selector.py ( 291) | _startSelection | PRIMARY SELECTION: Node2:0 selected primary Node2:0 for instance 0 (view 1) 2017-09-21 09:20:35,275 | INFO | node.py ( 510) | start_participating | Node2 started participating 2017-09-21 09:20:35,275 | DISPLAY | primary_selector.py ( 315) | _startSelection | VIEW CHANGE: Node2:0 declares view change 1 as completed for instance 0, new primary is Node2:0, ledger info is [(0, 3, '9pM4rmVSZGgU1muLXeHittuQNM8V9Z3hswV3DZDmb5Rq'), (1, 15, '29XGdr2VnKcHDtiphHc7MvB7H1ocjZksTbZ3rTbK7udz'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')] 2017-09-21 09:20:35,326 | INFO | replica.py (1483) | order_3pc_key | Node2:0 ordered batch request, view no 1, ppSeqNo 1, ledger 1, state root 2Rjm2BCifbKHnx4xC2dgXSQtW4rcqFCNrhBLwgfw19ho, txn root VoyJUvKZz2msmjKVwCfEqQEDoQnCTV1vHNzQvyiY3iJ, requests ordered [('V4SGRU86Z58d6TV7PBUe6f', 1505985623888664)], discarded [] 2017-09-21 09:20:35,329 | INFO | node.py (2301) | executeBatch | Node2 committed batch request, view no 1, ppSeqNo 1, ledger 1, state root 2Rjm2BCifbKHnx4xC2dgXSQtW4rcqFCNrhBLwgfw19ho, txn root VoyJUvKZz2msmjKVwCfEqQEDoQnCTV1vHNzQvyiY3iJ, requests: [('V4SGRU86Z58d6TV7PBUe6f', 1505985623888664)] 2017-09-21 09:20:42,046 | INFO | keep_in_touch.py ( 93) | _connsChanged | CONNECTION: Node2 disconnected from Node1 2017-09-21 09:20:50,278 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1505985650243643) from client b'h8K^THiG$X*:fkTsq^bmTi@*ez5N%<]0Kt0M0)^i' 2017-09-21 09:33:28,177 | INFO | keep_in_touch.py ( 98) | _connsChanged | CONNECTION: Node2 now connected to Node1 2017-09-21 09:33:49,696 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1505986429682183) from client b'h8K^THiG$X*:fkTsq^bmTi@*ez5N%<]0Kt0M0)^i' 2017-09-21 09:45:52,245 | INFO | propagator.py ( 179) | propagate | Node2 propagating request ('V4SGRU86Z58d6TV7PBUe6f', 1505987152222177) from client h8K^THiG$X*:fkTsq^bmTi@*ez5N%<]0Kt0M0)^i