2017-08-28 15:16:42,338 | DEBUG    | node_runner.py       (  18) | run_node | You can find logs in /home/sovrin/.sovrin/Node6.log
2017-08-28 15:16:42,339 | DEBUG    | node_runner.py       (  21) | run_node | Sovrin related env vars: []
2017-08-28 15:16:56,551 | DEBUG    | __init__.py          (  60) | register | Registered VCS backend: git
2017-08-28 15:16:57,078 | DEBUG    | __init__.py          (  60) | register | Registered VCS backend: hg
2017-08-28 15:16:57,780 | DEBUG    | __init__.py          (  60) | register | Registered VCS backend: svn
2017-08-28 15:16:57,814 | DEBUG    | __init__.py          (  60) | register | Registered VCS backend: bzr
2017-08-28 15:17:05,808 | DEBUG    | selector_events.py   (  53) | __init__ | Using selector: EpollSelector
2017-08-28 15:17:05,823 | DEBUG    | looper.py            ( 125) | __init__ | Setting handler for SIGINT
2017-08-28 15:17:05,905 | DEBUG    | ledger.py            ( 200) | start | Starting ledger...
2017-08-28 15:17:05,933 | DEBUG    | ledger.py            (  67) | recoverTree | Recovering tree from transaction log
2017-08-28 15:17:05,996 | DEBUG    | ledger.py            (  82) | recoverTree | Recovered tree in 0.06267848197603598 seconds
2017-08-28 15:17:06,087 | DEBUG    | idr_cache.py         (  25) | __init__ | Initializing identity cache Node6
2017-08-28 15:17:06,110 | INFO     | node.py              (2378) | initStateFromLedger | Node6 found state to be empty, recreating from ledger
2017-08-28 15:17:06,217 | DEBUG    | ledger.py            ( 200) | start | Starting ledger...
2017-08-28 15:17:06,249 | DEBUG    | ledger.py            (  67) | recoverTree | Recovering tree from transaction log
2017-08-28 15:17:06,305 | DEBUG    | ledger.py            (  82) | recoverTree | Recovered tree in 0.05615487799514085 seconds
2017-08-28 15:17:06,324 | INFO     | node.py              (2378) | initStateFromLedger | Node6 found state to be empty, recreating from ledger
2017-08-28 15:17:06,775 | INFO     | plugin_loader.py     ( 117) | _load | plugin FirebaseStatsConsumer successfully loaded from module plugin_firebase_stats_consumer
2017-08-28 15:17:06,777 | DEBUG    | plugin_loader.py     (  96) | _load | skipping plugin plugin_firebase_stats_consumer[class: <class 'plenum.server.plugin_loader.HasDynamicallyImportedModules'>] because it does not have a 'pluginType' attribute
2017-08-28 15:17:06,777 | DEBUG    | plugin_loader.py     (  96) | _load | skipping plugin plugin_firebase_stats_consumer[class: <enum 'Topic'>] because it does not have a 'pluginType' attribute
2017-08-28 15:17:06,790 | DEBUG    | plugin_loader.py     (  96) | _load | skipping plugin plugin_firebase_stats_consumer[class: typing.Dict<~KT, ~VT>] because it does not have a 'pluginType' attribute
2017-08-28 15:17:06,791 | DEBUG    | plugin_loader.py     (  96) | _load | skipping plugin plugin_firebase_stats_consumer[class: <class 'plenum.server.stats_consumer.StatsConsumer'>] because it does not have a 'pluginType' attribute
2017-08-28 15:17:06,800 | DEBUG    | plugin_loader.py     (  96) | _load | skipping plugin plugin_firebase_stats_consumer[class: <class 'plenum.server.plugin.stats_consumer.stats_publisher.StatsPublisher'>] because it does not have a 'pluginType' attribute
2017-08-28 15:17:06,801 | DEBUG    | has_action_queue.py  (  79) | startRepeating | checkPerformance will be repeating every 60 seconds
2017-08-28 15:17:06,803 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 1 to run in 60 seconds
2017-08-28 15:17:06,807 | DEBUG    | replica.py           ( 311) | h | Node6:0 set watermarks as 0 300
2017-08-28 15:17:06,808 | DISPLAY  | replicas.py          (  36) | grow | Node6 added replica Node6:0 to instance 0 (master)
2017-08-28 15:17:06,808 | DEBUG    | replica.py           ( 311) | h | Node6:1 set watermarks as 0 300
2017-08-28 15:17:06,808 | DISPLAY  | replicas.py          (  36) | grow | Node6 added replica Node6:1 to instance 1 (backup)
2017-08-28 15:17:06,809 | DEBUG    | has_action_queue.py  (  79) | startRepeating | checkPerformance will be repeating every 10 seconds
2017-08-28 15:17:06,824 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 1 to run in 10 seconds
2017-08-28 15:17:06,824 | DEBUG    | has_action_queue.py  (  79) | startRepeating | checkNodeRequestSpike will be repeating every 60 seconds
2017-08-28 15:17:06,824 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 2 to run in 60 seconds
2017-08-28 15:17:06,825 | DEBUG    | plugin_helper.py     (  24) | loadPlugins | Plugin loading started to load plugins from basedir: /home/sovrin/.sovrin
2017-08-28 15:17:06,825 | DEBUG    | plugin_helper.py     (  33) | loadPlugins | Plugin directory created at: /home/sovrin/.sovrin/plugins
2017-08-28 15:17:06,825 | DEBUG    | plugin_helper.py     (  68) | loadPlugins | Total plugins loaded from basedir /home/sovrin/.sovrin are : 0
2017-08-28 15:17:06,825 | DEBUG    | node.py              ( 337) | __init__ | total plugins loaded in node: 0
2017-08-28 15:17:06,914 | DEBUG    | ledger.py            ( 200) | start | Starting ledger...
2017-08-28 15:17:06,943 | DEBUG    | ledger.py            (  67) | recoverTree | Recovering tree from transaction log
2017-08-28 15:17:06,999 | DEBUG    | ledger.py            (  82) | recoverTree | Recovered tree in 0.05602344899671152 seconds
2017-08-28 15:17:07,027 | DEBUG    | upgrader.py          ( 118) | check_upgrade_succeeded | Node Node6 has no upgrade events
2017-08-28 15:17:07,028 | INFO     | node.py              (2378) | initStateFromLedger | Node6 found state to be empty, recreating from ledger
2017-08-28 15:17:07,028 | DEBUG    | motor.py             (  34) | set_status | Node6 changing status from stopped to starting
2017-08-28 15:17:07,028 | DEBUG    | ledger.py            ( 198) | start | Ledger already started.
2017-08-28 15:17:07,031 | DEBUG    | ledger.py            ( 198) | start | Ledger already started.
2017-08-28 15:17:07,031 | DEBUG    | ledger.py            ( 198) | start | Ledger already started.
2017-08-28 15:17:07,035 | DEBUG    | zstack.py            ( 319) | start | Node6 starting with restricted as True and reSetupAuth as True
2017-08-28 15:17:07,036 | DEBUG    | authenticator.py     (  31) | start | Starting ZAP at inproc://zeromq.zap.1
2017-08-28 15:17:07,036 | DEBUG    | base.py              (  72) | allow | Allowing 0.0.0.0
2017-08-28 15:17:07,039 | DEBUG    | base.py              ( 112) | configure_curve | Configure curve: *[/home/sovrin/.sovrin/Node6/public_keys]
2017-08-28 15:17:07,047 | DEBUG    | zstack.py            ( 347) | open | Node6 will bind its listener at 9701
2017-08-28 15:17:07,048 | INFO     | stacks.py            (  84) | start | CONNECTION: Node6 listening for other nodes at 0.0.0.0:9701
2017-08-28 15:17:07,048 | DEBUG    | zstack.py            ( 319) | start | Node6C starting with restricted as False and reSetupAuth as True
2017-08-28 15:17:07,048 | DEBUG    | authenticator.py     (  31) | start | Starting ZAP at inproc://zeromq.zap.2
2017-08-28 15:17:07,067 | DEBUG    | base.py              (  72) | allow | Allowing 0.0.0.0
2017-08-28 15:17:07,067 | DEBUG    | base.py              ( 112) | configure_curve | Configure curve: *[*]
2017-08-28 15:17:07,080 | DEBUG    | zstack.py            ( 347) | open | Node6C will bind its listener at 9702
2017-08-28 15:17:07,081 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action propose_view_change with id 3 to run in 60 seconds
2017-08-28 15:17:07,081 | INFO     | node.py              ( 612) | start | Node6 first time running...
2017-08-28 15:17:07,174 | DEBUG    | kit_zstack.py        (  97) | connectToMissing | CONNECTION: Node6 found the following missing connections: Node1, Node3, Node2, Node4
2017-08-28 15:17:07,180 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408624 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:17:07,181 | INFO     | zstack.py            ( 588) | connect | CONNECTION: Node6 looking for Node1 at 10.0.0.2:9701
2017-08-28 15:17:07,202 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:17:07,242 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45414992 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:17:07,242 | INFO     | zstack.py            ( 588) | connect | CONNECTION: Node6 looking for Node3 at 10.0.0.4:9705
2017-08-28 15:17:07,243 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:17:07,250 | TRACE    | remote.py            (  84) | connect | connecting socket 83 45419616 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:17:07,260 | INFO     | zstack.py            ( 588) | connect | CONNECTION: Node6 looking for Node2 at 10.0.0.3:9703
2017-08-28 15:17:07,260 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:17:07,280 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45599872 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:17:07,280 | INFO     | zstack.py            ( 588) | connect | CONNECTION: Node6 looking for Node4 at 10.0.0.5:9707
2017-08-28 15:17:07,281 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:17:07,294 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:17:07,300 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.204 seconds
2017-08-28 15:17:07,355 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:17:07,355 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:17:07,356 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:17:07,360 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:17:07,361 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:17:07,362 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:17:07,362 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:17:07,377 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:17:16,854 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 1
2017-08-28 15:17:16,855 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:17:16,863 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 4 to run in 10 seconds
2017-08-28 15:17:22,175 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:17:22,181 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:17:22,182 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:17:22,182 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:17:22,182 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:17:22,182 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:17:22,185 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:17:22,185 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:17:22,185 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:17:22,267 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:17:22,272 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:17:22,273 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:17:22,276 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:17:22,279 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:17:22,288 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:17:22,288 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:17:22,295 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:17:26,869 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 4
2017-08-28 15:17:26,869 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:17:26,876 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 5 to run in 10 seconds
2017-08-28 15:17:36,890 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 5
2017-08-28 15:17:36,911 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:17:36,912 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 6 to run in 10 seconds
2017-08-28 15:17:37,176 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:17:37,176 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:17:37,179 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:17:37,180 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:17:37,183 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:17:37,187 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:17:37,188 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408624
2017-08-28 15:17:37,191 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:17:37,192 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:17:37,193 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:17:37,236 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:17:37,237 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45414992
2017-08-28 15:17:37,237 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45657040 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:17:37,238 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:17:37,238 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:17:37,238 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:17:37,263 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 83 45419616
2017-08-28 15:17:37,264 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45692640 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:17:37,265 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:17:37,265 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:17:37,266 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:17:37,270 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45599872
2017-08-28 15:17:37,270 | TRACE    | remote.py            (  84) | connect | connecting socket 89 45728480 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:17:37,270 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:17:37,271 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:17:37,288 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.125 seconds
2017-08-28 15:17:37,350 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:17:37,351 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:17:37,351 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:17:37,351 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:17:37,363 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:17:37,364 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:17:37,364 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:17:37,372 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:17:46,938 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 6
2017-08-28 15:17:46,938 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:17:46,939 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 7 to run in 10 seconds
2017-08-28 15:17:52,198 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:17:52,208 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:17:52,208 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:17:52,208 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:17:52,209 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:17:52,216 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:17:52,217 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:17:52,224 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:17:52,224 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:17:52,255 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:17:52,256 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:17:52,256 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:17:52,266 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:17:52,267 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:17:52,267 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:17:52,267 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:17:52,268 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:17:56,950 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 7
2017-08-28 15:17:56,962 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:17:56,963 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 8 to run in 10 seconds
2017-08-28 15:18:06,835 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 2
2017-08-28 15:18:06,835 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:18:06,836 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 9 to run in 60 seconds
2017-08-28 15:18:06,836 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 1
2017-08-28 15:18:06,836 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:18:06,837 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 2 to run in 60 seconds
2017-08-28 15:18:06,970 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 8
2017-08-28 15:18:06,978 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:18:06,979 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 10 to run in 10 seconds
2017-08-28 15:18:07,098 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action propose_view_change with id 3
2017-08-28 15:18:07,108 | DEBUG    | throttler.py         (  31) | acquire | now: 249783.257324465, len(actionsLog): 0
2017-08-28 15:18:07,109 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 0
2017-08-28 15:18:07,110 | DEBUG    | throttler.py         (  39) | acquire | len(actionsLog) was 0, after append, len(actionsLog): 1
2017-08-28 15:18:07,110 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 1 since Primary of master protocol instance disconnected
2017-08-28 15:18:07,126 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 0}
            ordered request durations: {0: 0, 1: 0}
            master request latencies: {}
            client avg request latencies: [{}, {}]
            throughput: {0: 0, 1: 0}
            master throughput: None
            total requests: 0
            avg backup throughput: None
            master throughput ratio: None
2017-08-28 15:18:07,135 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 1, 'reason': 26} to all recipients: ['Node1', 'Node3', 'Node2', 'Node4']
2017-08-28 15:18:07,143 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 1
2017-08-28 15:18:07,144 | INFO     | node.py              (2089) | propose_view_change | Node6 sent view change since was disconnected from primary for too long
2017-08-28 15:18:07,144 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 1
2017-08-28 15:18:07,146 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:18:07,146 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node1 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:18:07,149 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:18:07,150 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:18:07,150 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node3 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:18:07,150 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:18:07,150 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:18:07,150 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node2 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:18:07,162 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:18:07,163 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:18:07,163 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node4 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:18:07,163 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":1,"reason":26,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:18:07,197 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.101 seconds
2017-08-28 15:18:07,232 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:07,232 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:07,236 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:07,236 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:07,236 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:07,239 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:07,239 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:18:07,266 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:07,266 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:18:07,266 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:07,268 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:07,268 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45657040
2017-08-28 15:18:07,269 | TRACE    | remote.py            (  84) | connect | connecting socket 84 45535904 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:07,271 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:18:07,272 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:07,272 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:07,278 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45692640
2017-08-28 15:18:07,283 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45571504 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:07,290 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:18:07,290 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:07,310 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:07,314 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 89 45728480
2017-08-28 15:18:07,315 | TRACE    | remote.py            (  84) | connect | connecting socket 83 45641136 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:07,327 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:18:07,328 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:18:07,329 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.119 seconds
2017-08-28 15:18:07,354 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:18:07,406 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:18:07,407 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:18:07,412 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:18:07,412 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:18:07,412 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:18:07,413 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:18:07,423 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:18:16,993 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 10
2017-08-28 15:18:17,010 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:18:17,011 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 11 to run in 10 seconds
2017-08-28 15:18:22,235 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:22,236 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:22,236 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:22,236 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:22,237 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:18:22,237 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:18:22,238 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:18:22,238 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:18:22,244 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:18:22,276 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:18:22,280 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:18:22,280 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:18:22,281 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:18:22,281 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:18:22,281 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:18:22,282 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:18:22,283 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:18:27,024 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 11
2017-08-28 15:18:27,025 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:18:27,025 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 12 to run in 10 seconds
2017-08-28 15:18:37,031 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 12
2017-08-28 15:18:37,038 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:18:37,038 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 13 to run in 10 seconds
2017-08-28 15:18:37,246 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:37,267 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:37,268 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:37,268 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:37,268 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:37,269 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:37,269 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:18:37,272 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:37,283 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:18:37,301 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:37,302 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:37,302 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 84 45535904
2017-08-28 15:18:37,308 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45629936 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:37,324 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:18:37,324 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:37,353 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:37,353 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45571504
2017-08-28 15:18:37,354 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45568736 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:37,354 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:18:37,373 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:37,374 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:37,389 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 83 45641136
2017-08-28 15:18:37,392 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45631584 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:37,407 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:18:37,415 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:18:37,423 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.182 seconds
2017-08-28 15:18:37,480 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:18:37,480 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:18:37,483 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:18:37,494 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:18:37,495 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:18:37,514 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:18:37,515 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:18:37,515 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:18:47,059 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 13
2017-08-28 15:18:47,059 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:18:47,064 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 14 to run in 10 seconds
2017-08-28 15:18:52,260 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:18:52,271 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:18:52,271 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:18:52,272 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:18:52,272 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:18:52,272 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:18:52,279 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:18:52,282 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:18:52,283 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:18:52,311 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:18:52,319 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:18:52,320 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:18:52,320 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:18:52,327 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:18:52,327 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:18:52,328 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:18:52,328 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:18:57,066 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 14
2017-08-28 15:18:57,077 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:18:57,078 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 15 to run in 10 seconds
2017-08-28 15:19:06,838 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 9
2017-08-28 15:19:06,838 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:19:06,840 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 16 to run in 60 seconds
2017-08-28 15:19:06,840 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 2
2017-08-28 15:19:06,840 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:19:06,840 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 3 to run in 60 seconds
2017-08-28 15:19:07,089 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 15
2017-08-28 15:19:07,089 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:19:07,097 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 17 to run in 10 seconds
2017-08-28 15:19:07,270 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:07,279 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:07,280 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:07,280 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:07,281 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:07,281 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:07,282 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:19:07,308 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45571456 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:07,309 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:19:07,310 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:07,310 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:07,310 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45629936
2017-08-28 15:19:07,316 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45629936 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:07,352 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:19:07,352 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:07,352 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:07,363 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45568736
2017-08-28 15:19:07,364 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45408384 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:07,364 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:19:07,364 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:07,365 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:07,365 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45631584
2017-08-28 15:19:07,383 | TRACE    | remote.py            (  84) | connect | connecting socket 83 45410032 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:07,384 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:19:07,384 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:19:07,413 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.146 seconds
2017-08-28 15:19:07,435 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:19:07,477 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:19:07,478 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:19:07,478 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:19:07,478 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:19:07,479 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:19:07,480 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:19:07,480 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:19:17,116 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 17
2017-08-28 15:19:17,126 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:19:17,126 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 18 to run in 10 seconds
2017-08-28 15:19:22,285 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:22,290 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:22,298 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:22,298 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:22,299 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:19:22,303 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:19:22,312 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:19:22,312 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:19:22,312 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:19:22,340 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:19:22,341 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:19:22,341 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:19:22,342 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:19:22,345 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:19:22,347 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:19:22,349 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:19:22,350 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:19:27,130 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 18
2017-08-28 15:19:27,140 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:19:27,141 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 19 to run in 10 seconds
2017-08-28 15:19:37,145 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 19
2017-08-28 15:19:37,148 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:19:37,148 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 20 to run in 10 seconds
2017-08-28 15:19:37,294 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:37,299 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:37,300 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:37,300 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:37,300 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:37,309 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:37,309 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45571456
2017-08-28 15:19:37,311 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45632512 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:37,325 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:19:37,325 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:37,325 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:37,330 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45629936
2017-08-28 15:19:37,332 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45624640 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:37,333 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:19:37,365 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:37,365 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:37,365 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45408384
2017-08-28 15:19:37,366 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45629936 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:37,387 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:19:37,387 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:37,388 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:37,388 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 83 45410032
2017-08-28 15:19:37,391 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45571856 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:37,391 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:19:37,392 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:19:37,392 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.113 seconds
2017-08-28 15:19:37,433 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:19:37,474 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:19:37,475 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:19:37,475 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:19:37,481 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:19:37,485 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:19:37,485 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:19:37,486 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:19:47,154 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 20
2017-08-28 15:19:47,167 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:19:47,167 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 21 to run in 10 seconds
2017-08-28 15:19:52,310 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:19:52,314 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:19:52,314 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:19:52,315 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:19:52,318 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:19:52,320 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:19:52,320 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:19:52,325 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:19:52,326 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:19:52,358 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:19:52,358 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:19:52,359 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:19:52,360 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:19:52,368 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:19:52,372 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:19:52,372 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:19:52,372 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:19:57,182 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 21
2017-08-28 15:19:57,185 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:19:57,188 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 22 to run in 10 seconds
2017-08-28 15:20:06,860 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 16
2017-08-28 15:20:06,867 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:20:06,868 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 23 to run in 60 seconds
2017-08-28 15:20:06,868 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 3
2017-08-28 15:20:06,868 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:20:06,868 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 4 to run in 60 seconds
2017-08-28 15:20:07,214 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 22
2017-08-28 15:20:07,215 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:20:07,216 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 24 to run in 10 seconds
2017-08-28 15:20:07,323 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:07,328 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:07,328 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:07,329 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:07,339 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:07,340 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:07,340 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45632512
2017-08-28 15:20:07,344 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45632000 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:07,347 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:20:07,350 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:07,351 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:07,352 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45624640
2017-08-28 15:20:07,364 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45408384 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:07,365 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:20:07,365 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:07,365 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:07,372 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45629936
2017-08-28 15:20:07,381 | TRACE    | remote.py            (  84) | connect | connecting socket 82 45410032 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:07,381 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:20:07,382 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:07,417 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:07,417 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45571856
2017-08-28 15:20:07,418 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45629936 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:07,440 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:20:07,441 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:20:07,442 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.131 seconds
2017-08-28 15:20:07,474 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:20:07,490 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:20:07,492 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:20:07,493 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:20:07,499 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:20:07,506 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:20:07,514 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:20:07,515 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:20:17,222 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 24
2017-08-28 15:20:17,223 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:20:17,224 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 25 to run in 10 seconds
2017-08-28 15:20:22,331 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:22,334 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:22,339 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:22,340 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:22,340 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:20:22,340 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:20:22,351 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:20:22,352 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:20:22,353 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:20:22,387 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:20:22,387 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:20:22,389 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:20:22,390 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:20:22,390 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:20:22,390 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:20:22,396 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:20:22,396 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:20:27,235 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 25
2017-08-28 15:20:27,241 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:20:27,241 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 26 to run in 10 seconds
2017-08-28 15:20:37,243 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 26
2017-08-28 15:20:37,248 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:20:37,254 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 27 to run in 10 seconds
2017-08-28 15:20:37,356 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:37,356 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:37,357 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:37,357 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:37,357 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:37,358 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:37,358 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45632000
2017-08-28 15:20:37,375 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45634160 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:37,375 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:20:37,379 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:37,380 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:37,380 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45408384
2017-08-28 15:20:37,383 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45408384 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:37,386 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:20:37,386 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:37,394 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:37,394 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 82 45410032
2017-08-28 15:20:37,394 | TRACE    | remote.py            (  84) | connect | connecting socket 82 45410032 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:37,395 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:20:37,395 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:37,395 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:37,395 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45629936
2017-08-28 15:20:37,440 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45571856 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:37,440 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:20:37,444 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:20:37,444 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.108 seconds
2017-08-28 15:20:37,488 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:20:37,509 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:20:37,509 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:20:37,511 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:20:37,512 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:20:37,512 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:20:37,533 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:20:37,533 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:20:47,283 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 27
2017-08-28 15:20:47,288 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:20:47,288 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 28 to run in 10 seconds
2017-08-28 15:20:52,364 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:20:52,364 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:20:52,374 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:20:52,375 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:20:52,375 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:20:52,379 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:20:52,386 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:20:52,387 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:20:52,387 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:20:52,408 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:20:52,409 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:20:52,411 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:20:52,412 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:20:52,412 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:20:52,412 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:20:52,413 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:20:52,413 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:20:57,306 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 28
2017-08-28 15:20:57,309 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:20:57,311 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 29 to run in 10 seconds
2017-08-28 15:21:06,878 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 23
2017-08-28 15:21:06,889 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:21:06,889 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 30 to run in 60 seconds
2017-08-28 15:21:06,892 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 4
2017-08-28 15:21:06,892 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:21:06,892 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 5 to run in 60 seconds
2017-08-28 15:21:07,316 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 29
2017-08-28 15:21:07,319 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:21:07,334 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 31 to run in 10 seconds
2017-08-28 15:21:07,368 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:07,371 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:07,371 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:07,385 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:07,386 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:07,389 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:07,391 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45634160
2017-08-28 15:21:07,392 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45634032 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:07,393 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:21:07,393 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:07,393 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:07,413 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45408384
2017-08-28 15:21:07,422 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45408384 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:07,423 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:21:07,423 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:07,453 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:07,453 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 82 45410032
2017-08-28 15:21:07,454 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45629936 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:07,454 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:21:07,454 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:07,455 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:07,455 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45571856
2017-08-28 15:21:07,455 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45706288 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:07,456 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:21:07,475 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:21:07,476 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.110 seconds
2017-08-28 15:21:07,535 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:21:07,552 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:21:07,552 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:21:07,552 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:21:07,564 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:21:07,564 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:21:07,566 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:21:07,567 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:21:17,351 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 31
2017-08-28 15:21:17,352 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:21:17,357 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 32 to run in 10 seconds
2017-08-28 15:21:22,369 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:22,383 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:22,383 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:22,384 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:22,384 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:21:22,394 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:21:22,395 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:21:22,395 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:21:22,395 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:21:22,431 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:21:22,434 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:21:22,435 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:21:22,442 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:21:22,444 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:21:22,444 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:21:22,445 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:21:22,445 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:21:27,377 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 32
2017-08-28 15:21:27,378 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:21:27,378 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 33 to run in 10 seconds
2017-08-28 15:21:37,383 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 33
2017-08-28 15:21:37,390 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:21:37,391 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 34 to run in 10 seconds
2017-08-28 15:21:37,396 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:37,407 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:37,408 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:37,408 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:37,408 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:37,410 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:37,423 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45634032
2017-08-28 15:21:37,427 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45410032 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:37,467 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:21:37,468 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:37,468 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:37,475 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45408384
2017-08-28 15:21:37,484 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45570736 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:37,484 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:21:37,484 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:37,485 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:37,485 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45629936
2017-08-28 15:21:37,501 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45590336 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:37,502 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:21:37,502 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:37,504 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:37,514 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45706288
2017-08-28 15:21:37,542 | TRACE    | remote.py            (  84) | connect | connecting socket 84 45408384 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:37,542 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:21:37,542 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:21:37,578 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:21:37,579 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:21:37,583 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:21:37,584 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:21:37,584 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:21:37,584 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:21:37,608 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:21:37,618 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:21:37,628 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.251 seconds
2017-08-28 15:21:47,405 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 34
2017-08-28 15:21:47,407 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:21:47,417 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 35 to run in 10 seconds
2017-08-28 15:21:52,419 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:21:52,420 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:21:52,424 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:21:52,425 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:21:52,433 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:21:52,433 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:21:52,434 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:21:52,434 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:21:52,443 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:21:52,474 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:21:52,475 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:21:52,484 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:21:52,484 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:21:52,487 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:21:52,495 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:21:52,496 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:21:52,496 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:21:57,422 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 35
2017-08-28 15:21:57,423 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:21:57,430 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 36 to run in 10 seconds
2017-08-28 15:22:06,903 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 30
2017-08-28 15:22:06,911 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:22:06,911 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 37 to run in 60 seconds
2017-08-28 15:22:06,914 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 5
2017-08-28 15:22:06,922 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:22:06,929 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 6 to run in 60 seconds
2017-08-28 15:22:07,440 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 36
2017-08-28 15:22:07,462 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:22:07,462 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 38 to run in 10 seconds
2017-08-28 15:22:07,463 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:07,468 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:07,469 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:07,469 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:07,471 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:07,472 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:07,484 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45410032
2017-08-28 15:22:07,484 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45410032 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:07,488 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:22:07,488 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:07,488 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:07,489 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45570736
2017-08-28 15:22:07,501 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45570736 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:07,502 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:22:07,503 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:07,515 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:07,551 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45590336
2017-08-28 15:22:07,551 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45629936 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:07,552 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:22:07,552 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:07,552 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:07,552 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 84 45408384
2017-08-28 15:22:07,571 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45632704 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:07,571 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:22:07,596 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:22:07,621 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:22:07,647 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:22:07,649 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:22:07,649 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:22:07,650 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:22:07,651 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:22:07,658 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:22:07,663 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:22:07,664 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.226 seconds
2017-08-28 15:22:17,463 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 38
2017-08-28 15:22:17,468 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:22:17,468 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 39 to run in 10 seconds
2017-08-28 15:22:22,474 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:22,479 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:22,480 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:22,482 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:22,499 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:22:22,500 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:22:22,500 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:22:22,500 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:22:22,504 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:22:22,536 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:22:22,536 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:22:22,543 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:22:22,551 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:22:22,556 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:22:22,556 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:22:22,559 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:22:22,560 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:22:27,499 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 39
2017-08-28 15:22:27,519 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:22:27,523 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 40 to run in 10 seconds
2017-08-28 15:22:37,489 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:37,492 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:37,493 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:37,494 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:37,494 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:37,495 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:37,512 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45410032
2017-08-28 15:22:37,514 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:37,523 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:22:37,523 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:37,533 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:37,533 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45570736
2017-08-28 15:22:37,534 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45410032 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:37,534 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:22:37,552 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:37,588 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:37,589 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45629936
2017-08-28 15:22:37,590 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45658160 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:37,591 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:22:37,591 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:37,619 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:37,620 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45632704
2017-08-28 15:22:37,620 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45659808 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:37,621 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:22:37,621 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:22:37,622 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.151 seconds
2017-08-28 15:22:37,658 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 40
2017-08-28 15:22:37,691 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:22:37,692 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 41 to run in 10 seconds
2017-08-28 15:22:37,692 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:22:37,705 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:22:37,723 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:22:37,723 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:22:37,724 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:22:37,724 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:22:37,724 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:22:37,726 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:22:37,790 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.157 seconds
2017-08-28 15:22:47,720 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 41
2017-08-28 15:22:47,723 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:22:47,724 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 42 to run in 10 seconds
2017-08-28 15:22:52,493 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:22:52,500 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:22:52,500 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:22:52,500 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:22:52,501 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:22:52,501 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:22:52,515 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:22:52,515 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:22:52,516 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:22:52,546 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:22:52,546 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:22:52,546 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:22:52,547 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:22:52,547 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:22:52,547 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:22:52,547 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:22:52,551 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:22:57,725 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 42
2017-08-28 15:22:57,727 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:22:57,731 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 43 to run in 10 seconds
2017-08-28 15:23:06,920 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 37
2017-08-28 15:23:06,931 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:23:06,932 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 44 to run in 60 seconds
2017-08-28 15:23:06,932 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 6
2017-08-28 15:23:06,940 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:23:06,941 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 7 to run in 60 seconds
2017-08-28 15:23:07,504 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:07,508 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:07,508 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:07,508 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:07,509 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:07,524 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:07,524 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:23:07,532 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:07,532 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:23:07,556 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:07,556 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:07,556 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45410032
2017-08-28 15:23:07,583 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45410032 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:07,584 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:23:07,610 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:07,611 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:07,623 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45658160
2017-08-28 15:23:07,624 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45629936 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:07,624 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:23:07,628 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:07,630 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:07,631 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45659808
2017-08-28 15:23:07,643 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45632704 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:07,644 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:23:07,644 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:23:07,672 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.187 seconds
2017-08-28 15:23:07,678 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:23:07,708 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:23:07,710 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:23:07,711 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:23:07,724 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:23:07,724 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:23:07,729 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:23:07,729 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:23:07,750 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 43
2017-08-28 15:23:07,808 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:23:07,809 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 45 to run in 10 seconds
2017-08-28 15:23:17,855 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 45
2017-08-28 15:23:17,857 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:23:17,875 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 46 to run in 10 seconds
2017-08-28 15:23:22,504 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:22,518 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:22,519 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:22,520 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:22,520 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:23:22,521 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:23:22,522 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:23:22,524 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:23:22,542 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:23:22,578 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:23:22,579 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:23:22,580 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:23:22,584 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:23:22,592 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:23:22,598 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:23:22,599 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:23:22,608 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:23:27,896 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 46
2017-08-28 15:23:27,899 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:23:27,899 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 47 to run in 10 seconds
2017-08-28 15:23:37,506 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:37,507 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:37,511 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:37,511 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:37,512 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:37,512 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:37,513 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:23:37,519 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:37,529 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:23:37,529 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:37,529 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:37,531 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45410032
2017-08-28 15:23:37,546 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45410032 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:37,546 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:23:37,575 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:37,576 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:37,576 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45629936
2017-08-28 15:23:37,577 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45623680 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:37,577 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:23:37,595 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:37,600 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:37,602 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45632704
2017-08-28 15:23:37,602 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45603568 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:37,603 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:23:37,606 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:23:37,606 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.121 seconds
2017-08-28 15:23:37,667 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:23:37,677 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:23:37,677 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:23:37,677 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:23:37,679 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:23:37,679 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:23:37,687 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:23:37,710 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:23:37,912 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 47
2017-08-28 15:23:37,912 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:23:37,912 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 48 to run in 10 seconds
2017-08-28 15:23:47,927 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 48
2017-08-28 15:23:47,931 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:23:47,932 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 49 to run in 10 seconds
2017-08-28 15:23:52,516 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:23:52,517 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:23:52,517 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:23:52,522 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:23:52,523 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:23:52,523 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:23:52,527 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:23:52,527 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:23:52,528 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:23:52,555 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:23:52,556 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:23:52,556 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:23:52,556 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:23:52,567 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:23:52,568 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:23:52,568 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:23:52,571 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:23:57,948 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 49
2017-08-28 15:23:57,948 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:23:57,950 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 50 to run in 10 seconds
2017-08-28 15:24:06,943 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 44
2017-08-28 15:24:06,944 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:24:06,959 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 51 to run in 60 seconds
2017-08-28 15:24:06,960 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 7
2017-08-28 15:24:06,960 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:24:06,960 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 8 to run in 60 seconds
2017-08-28 15:24:07,518 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:07,521 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:07,521 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:07,522 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:07,522 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:07,527 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:07,536 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:24:07,537 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:07,541 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:24:07,541 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:07,545 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:07,545 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45410032
2017-08-28 15:24:07,546 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45410032 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:07,575 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:24:07,594 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:07,594 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:07,594 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45623680
2017-08-28 15:24:07,612 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45631536 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:07,612 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:24:07,614 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:07,614 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:07,615 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45603568
2017-08-28 15:24:07,621 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45621632 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:07,622 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:24:07,622 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:24:07,622 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.107 seconds
2017-08-28 15:24:07,686 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:24:07,691 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:24:07,697 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:24:07,697 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:24:07,697 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:24:07,712 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:24:07,713 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:24:07,713 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:24:07,958 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 50
2017-08-28 15:24:07,969 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:24:07,969 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 52 to run in 10 seconds
2017-08-28 15:24:17,997 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 52
2017-08-28 15:24:18,000 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:24:18,001 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 53 to run in 10 seconds
2017-08-28 15:24:22,526 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:22,547 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:22,547 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:22,547 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:22,547 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:24:22,548 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:24:22,548 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:24:22,548 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:24:22,548 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:24:22,590 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:24:22,591 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:24:22,595 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:24:22,595 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:24:22,595 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:24:22,596 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:24:22,596 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:24:22,598 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:24:28,027 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 53
2017-08-28 15:24:28,027 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:24:28,028 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 54 to run in 10 seconds
2017-08-28 15:24:37,533 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:37,533 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:37,534 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:37,543 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:37,544 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:37,544 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:37,544 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:24:37,555 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:37,556 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:24:37,556 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:37,556 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:37,558 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45410032
2017-08-28 15:24:37,561 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45410032 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:37,561 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:24:37,563 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:37,597 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:37,598 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45631536
2017-08-28 15:24:37,615 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45723824 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:37,616 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:24:37,636 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:37,636 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:37,637 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45621632
2017-08-28 15:24:37,640 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45623280 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:37,640 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:24:37,652 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:24:37,654 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.128 seconds
2017-08-28 15:24:37,673 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:24:37,702 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:24:37,702 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:24:37,703 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:24:37,703 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:24:37,713 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:24:37,714 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:24:37,714 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:24:38,048 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 54
2017-08-28 15:24:38,048 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:24:38,049 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 55 to run in 10 seconds
2017-08-28 15:24:48,058 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 55
2017-08-28 15:24:48,066 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:24:48,069 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 56 to run in 10 seconds
2017-08-28 15:24:52,540 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:24:52,544 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:24:52,544 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:24:52,544 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:24:52,544 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:24:52,545 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:24:52,567 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:24:52,567 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:24:52,567 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:24:52,607 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:24:52,607 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:24:52,608 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:24:52,608 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:24:52,612 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:24:52,612 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:24:52,622 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:24:52,622 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:24:58,079 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 56
2017-08-28 15:24:58,083 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:24:58,084 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 57 to run in 10 seconds
2017-08-28 15:25:06,970 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 51
2017-08-28 15:25:06,975 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:25:06,977 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 58 to run in 60 seconds
2017-08-28 15:25:06,980 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 8
2017-08-28 15:25:06,980 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:25:06,981 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 9 to run in 60 seconds
2017-08-28 15:25:07,547 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:07,547 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:07,548 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:07,548 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:07,548 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:07,550 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:07,550 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:25:07,570 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:07,571 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:25:07,575 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:07,606 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:07,609 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45410032
2017-08-28 15:25:07,616 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45631536 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:07,616 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:25:07,642 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:07,642 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:07,642 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45723824
2017-08-28 15:25:07,645 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45621376 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:07,655 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:25:07,664 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:07,664 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:07,664 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45623280
2017-08-28 15:25:07,665 | TRACE    | remote.py            (  84) | connect | connecting socket 82 45410032 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:07,665 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:25:07,665 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:25:07,666 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.139 seconds
2017-08-28 15:25:07,738 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:25:07,757 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:25:07,757 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:25:07,758 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:25:07,772 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:25:07,776 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:25:07,776 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:25:07,797 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:25:08,108 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 57
2017-08-28 15:25:08,110 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:25:08,112 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 59 to run in 10 seconds
2017-08-28 15:25:18,127 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 59
2017-08-28 15:25:18,132 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:25:18,132 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 60 to run in 10 seconds
2017-08-28 15:25:22,549 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:22,562 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:22,563 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:22,563 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:22,563 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:25:22,564 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:25:22,564 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:25:22,567 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:25:22,567 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:25:22,603 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:25:22,604 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:25:22,604 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:25:22,607 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:25:22,607 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:25:22,613 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:25:22,613 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:25:22,614 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:25:28,134 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 60
2017-08-28 15:25:28,148 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:25:28,148 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 61 to run in 10 seconds
2017-08-28 15:25:37,560 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:37,571 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:37,571 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:37,572 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:37,572 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:37,572 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:37,583 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:25:37,611 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45408384 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:37,611 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:25:37,612 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:37,626 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:37,627 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45631536
2017-08-28 15:25:37,634 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45722704 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:37,642 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:25:37,642 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:37,647 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:37,658 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45621376
2017-08-28 15:25:37,661 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45623280 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:37,661 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:25:37,666 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:37,666 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:37,672 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 82 45410032
2017-08-28 15:25:37,685 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45624928 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:37,698 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:25:37,699 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:25:37,700 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.142 seconds
2017-08-28 15:25:37,736 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:25:37,751 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:25:37,751 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:25:37,751 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:25:37,752 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:25:37,752 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:25:37,774 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:25:37,774 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:25:38,181 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 61
2017-08-28 15:25:38,185 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:25:38,186 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 62 to run in 10 seconds
2017-08-28 15:25:48,204 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 62
2017-08-28 15:25:48,204 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:25:48,204 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 63 to run in 10 seconds
2017-08-28 15:25:52,592 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:25:52,592 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:25:52,595 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:25:52,598 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:25:52,598 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:25:52,609 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:25:52,610 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:25:52,611 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:25:52,612 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:25:52,655 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:25:52,655 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:25:52,656 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:25:52,656 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:25:52,659 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:25:52,663 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:25:52,666 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:25:52,671 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:25:58,206 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 63
2017-08-28 15:25:58,223 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:25:58,224 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 64 to run in 10 seconds
2017-08-28 15:26:06,978 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 58
2017-08-28 15:26:06,978 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:26:06,978 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 65 to run in 60 seconds
2017-08-28 15:26:06,991 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 9
2017-08-28 15:26:06,992 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:26:07,009 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 10 to run in 60 seconds
2017-08-28 15:26:07,619 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:07,620 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:07,622 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:07,631 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:07,632 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:07,635 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:07,641 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45408384
2017-08-28 15:26:07,645 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45621376 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:07,646 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:26:07,646 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:07,658 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:07,658 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45722704
2017-08-28 15:26:07,683 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45408384 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:07,684 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:26:07,687 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:07,708 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:07,708 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45623280
2017-08-28 15:26:07,723 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45630416 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:07,724 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:26:07,724 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:07,727 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:07,732 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45624928
2017-08-28 15:26:07,741 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45706288 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:07,758 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:26:07,758 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:26:07,760 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.159 seconds
2017-08-28 15:26:07,783 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:26:07,830 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:26:07,836 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:26:07,836 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:26:07,843 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:26:07,846 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:26:07,852 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:26:07,852 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:26:08,225 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 64
2017-08-28 15:26:08,235 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:26:08,236 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 66 to run in 10 seconds
2017-08-28 15:26:18,254 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 66
2017-08-28 15:26:18,255 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:26:18,257 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 67 to run in 10 seconds
2017-08-28 15:26:22,624 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:22,624 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:22,627 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:22,627 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:22,627 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:26:22,641 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:26:22,641 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:26:22,642 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:26:22,643 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:26:22,670 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:26:22,670 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:26:22,681 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:26:22,681 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:26:22,682 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:26:22,682 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:26:22,682 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:26:22,683 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:26:28,281 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 67
2017-08-28 15:26:28,281 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:26:28,281 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 68 to run in 10 seconds
2017-08-28 15:26:37,631 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:37,632 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:37,633 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:37,634 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:37,634 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:37,634 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:37,636 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45621376
2017-08-28 15:26:37,650 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45623280 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:37,651 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:26:37,652 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:37,663 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:37,664 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45408384
2017-08-28 15:26:37,700 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45624928 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:37,702 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:26:37,703 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:37,703 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:37,717 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45630416
2017-08-28 15:26:37,717 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45408384 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:37,718 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:26:37,718 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:37,746 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:37,746 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45706288
2017-08-28 15:26:37,747 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45619984 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:37,763 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:26:37,776 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:26:37,777 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.164 seconds
2017-08-28 15:26:37,852 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:26:37,853 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:26:37,853 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:26:37,853 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:26:37,854 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:26:37,870 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:26:37,886 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:26:37,886 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:26:38,285 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 68
2017-08-28 15:26:38,307 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:26:38,315 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 69 to run in 10 seconds
2017-08-28 15:26:48,331 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 69
2017-08-28 15:26:48,337 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:26:48,337 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 70 to run in 10 seconds
2017-08-28 15:26:52,638 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:26:52,639 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:26:52,639 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:26:52,647 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:26:52,648 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:26:52,648 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:26:52,648 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:26:52,648 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:26:52,654 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:26:52,692 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:26:52,692 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:26:52,692 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:26:52,693 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:26:52,699 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:26:52,700 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:26:52,700 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:26:52,700 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:26:58,342 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 70
2017-08-28 15:26:58,356 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:26:58,356 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 71 to run in 10 seconds
2017-08-28 15:27:06,981 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 65
2017-08-28 15:27:06,994 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:27:06,995 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 72 to run in 60 seconds
2017-08-28 15:27:07,014 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 10
2017-08-28 15:27:07,017 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:27:07,019 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 11 to run in 60 seconds
2017-08-28 15:27:07,640 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:07,649 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:07,657 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:07,658 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:07,658 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:07,668 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:07,668 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45623280
2017-08-28 15:27:07,673 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45623280 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:07,677 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:27:07,677 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:07,678 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:07,679 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45624928
2017-08-28 15:27:07,701 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45624928 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:07,702 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:27:07,702 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:07,711 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:07,712 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45408384
2017-08-28 15:27:07,712 | TRACE    | remote.py            (  84) | connect | connecting socket 84 45631472 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:07,712 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:27:07,712 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:07,713 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:07,713 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45619984
2017-08-28 15:27:07,754 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45408384 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:07,776 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:27:07,777 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:27:07,796 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.178 seconds
2017-08-28 15:27:07,839 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:27:07,856 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:27:07,857 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:27:07,857 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:27:07,857 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:27:07,858 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:27:07,858 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:27:07,858 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:27:08,361 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 71
2017-08-28 15:27:08,378 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:27:08,379 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 73 to run in 10 seconds
2017-08-28 15:27:18,398 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 73
2017-08-28 15:27:18,401 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:27:18,401 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 74 to run in 10 seconds
2017-08-28 15:27:22,649 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:22,650 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:22,650 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:22,651 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:22,655 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:27:22,667 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:27:22,667 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:27:22,669 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:27:22,669 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:27:22,688 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:27:22,691 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:27:22,706 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:27:22,706 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:27:22,706 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:27:22,707 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:27:22,707 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:27:22,712 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:27:28,414 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 74
2017-08-28 15:27:28,423 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:27:28,430 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 75 to run in 10 seconds
2017-08-28 15:27:37,660 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:37,661 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:37,661 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:37,661 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:37,662 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:37,662 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:37,662 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45623280
2017-08-28 15:27:37,695 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45623280 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:37,696 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:27:37,696 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:37,696 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:37,720 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45624928
2017-08-28 15:27:37,728 | TRACE    | remote.py            (  84) | connect | connecting socket 87 45619984 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:37,729 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:27:37,740 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:37,740 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:37,741 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 84 45631472
2017-08-28 15:27:37,747 | TRACE    | remote.py            (  84) | connect | connecting socket 88 45706288 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:37,755 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:27:37,756 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:37,758 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:37,766 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45408384
2017-08-28 15:27:37,771 | TRACE    | remote.py            (  84) | connect | connecting socket 84 45624928 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:37,778 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:27:37,790 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:27:37,800 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.147 seconds
2017-08-28 15:27:37,843 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:27:37,859 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:27:37,859 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:27:37,860 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:27:37,860 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:27:37,862 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:27:37,863 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:27:37,863 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:27:38,432 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 75
2017-08-28 15:27:38,435 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:27:38,443 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 76 to run in 10 seconds
2017-08-28 15:27:40,125 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.237 seconds
2017-08-28 15:27:48,462 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 76
2017-08-28 15:27:48,464 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:27:48,464 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 77 to run in 10 seconds
2017-08-28 15:27:52,670 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:27:52,684 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:27:52,685 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:27:52,685 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:27:52,685 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:27:52,691 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:27:52,691 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:27:52,692 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:27:52,692 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:27:52,736 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:27:52,736 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:27:52,743 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:27:52,744 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:27:52,745 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:27:52,753 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:27:52,754 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:27:52,754 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:27:58,476 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 77
2017-08-28 15:27:58,477 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:27:58,477 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 78 to run in 10 seconds
2017-08-28 15:28:07,023 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 72
2017-08-28 15:28:07,024 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:28:07,024 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 79 to run in 60 seconds
2017-08-28 15:28:07,024 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 11
2017-08-28 15:28:07,024 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:28:07,025 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 12 to run in 60 seconds
2017-08-28 15:28:07,682 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:07,688 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:07,693 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:07,693 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:07,705 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:07,706 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:07,706 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45623280
2017-08-28 15:28:07,707 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45623280 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:07,707 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:28:07,716 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:07,717 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:07,717 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 87 45619984
2017-08-28 15:28:07,753 | TRACE    | remote.py            (  84) | connect | connecting socket 81 45409504 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:07,753 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:28:07,754 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:07,775 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:07,776 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 88 45706288
2017-08-28 15:28:07,776 | TRACE    | remote.py            (  84) | connect | connecting socket 85 45630352 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:07,776 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:28:07,777 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:07,777 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:07,781 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 84 45624928
2017-08-28 15:28:07,784 | TRACE    | remote.py            (  84) | connect | connecting socket 86 45632000 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:07,787 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:28:07,791 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:28:07,800 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.120 seconds
2017-08-28 15:28:07,853 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node1
2017-08-28 15:28:07,853 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node1
2017-08-28 15:28:07,854 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node3
2017-08-28 15:28:07,854 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node3
2017-08-28 15:28:07,854 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node2
2017-08-28 15:28:07,877 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node2
2017-08-28 15:28:07,877 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node4
2017-08-28 15:28:07,877 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node4
2017-08-28 15:28:08,485 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 78
2017-08-28 15:28:08,486 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:28:08,486 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 80 to run in 10 seconds
2017-08-28 15:28:10,486 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 2 messages through listener
2017-08-28 15:28:10,519 | DEBUG    | zstack.py            ( 661) | handlePingPong | Node6 got ping from Node2
2017-08-28 15:28:10,520 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 ponged Node2
2017-08-28 15:28:10,520 | DEBUG    | zstack.py            ( 661) | handlePingPong | Node6 got ping from Node3
2017-08-28 15:28:10,520 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 ponged Node3
2017-08-28 15:28:10,521 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'po' to Node3
2017-08-28 15:28:10,521 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'po' to Node3
2017-08-28 15:28:10,521 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'po' to Node2
2017-08-28 15:28:10,522 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'po' to Node2
2017-08-28 15:28:10,562 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:10,573 | DEBUG    | zstack.py            ( 661) | handlePingPong | Node6 got ping from Node1
2017-08-28 15:28:10,590 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 ponged Node1
2017-08-28 15:28:10,618 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'po' to Node1
2017-08-28 15:28:10,619 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'po' to Node1
2017-08-28 15:28:10,630 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 2 messages through listener
2017-08-28 15:28:10,658 | DEBUG    | zstack.py            ( 661) | handlePingPong | Node6 got ping from Node4
2017-08-28 15:28:10,659 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 ponged Node4
2017-08-28 15:28:10,659 | DEBUG    | zstack.py            ( 661) | handlePingPong | Node6 got ping from b'BEhc!q7/{^9)vgy&p$f!kx.sTfa9gxeT9(BqKVil'
2017-08-28 15:28:10,666 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 ponged b'BEhc!q7/{^9)vgy&p$f!kx.sTfa9gxeT9(BqKVil'
2017-08-28 15:28:10,679 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'po' to Node4
2017-08-28 15:28:10,681 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'po' to Node4
2017-08-28 15:28:10,681 | WARNING  | batched.py           ( 121) | flushOutBoxes | CONNECTION: Node6 rid b'BEhc!q7/{^9)vgy&p$f!kx.sTfa9gxeT9(BqKVil' has been removed
2017-08-28 15:28:10,681 | DEBUG    | message_processor.py (  29) | discard | Node6 discarding message deque([b'po']) because CONNECTION: rid b'BEhc!q7/{^9)vgy&p$f!kx.sTfa9gxeT9(BqKVil' no longer available
2017-08-28 15:28:10,704 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6C got 1 messages through listener
2017-08-28 15:28:10,704 | DEBUG    | zstack.py            ( 661) | handlePingPong | Node6C got ping from b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:28:10,705 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'po' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:28:10,712 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6C ponged b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:28:10,735 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:28:10,749 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"CURRENT_STATE","primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2}', '{"ledgerId":0,"txnSeqNo":7,"viewNo":2,"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"ledgerId":1,"txnSeqNo":15,"viewNo":0,"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}', '{"ledgerId":2,"txnSeqNo":0,"viewNo":null,"op":"LEDGER_STATUS","ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"}']}
2017-08-28 15:28:10,760 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"CURRENT_STATE","primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2}', '{"ledgerId":0,"txnSeqNo":7,"viewNo":2,"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"ledgerId":1,"txnSeqNo":15,"viewNo":0,"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}', '{"ledgerId":2,"txnSeqNo":0,"viewNo":null,"op":"LEDGER_STATUS","ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"}'], 'op': 'BATCH'}, 'Node3')
2017-08-28 15:28:10,760 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"CURRENT_STATE","primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2}', '{"ledgerId":0,"txnSeqNo":7,"viewNo":2,"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"ledgerId":1,"txnSeqNo":15,"viewNo":0,"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}', '{"ledgerId":2,"txnSeqNo":0,"viewNo":null,"op":"LEDGER_STATUS","ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"}']}
2017-08-28 15:28:10,761 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:10,779 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}], 'op': 'CURRENT_STATE'}, 'Node3')
2017-08-28 15:28:10,796 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:10,797 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:10,797 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'txnSeqNo': 7, 'viewNo': 2, 'ppSeqNo': 6, 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node3')
2017-08-28 15:28:10,808 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:10,808 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:10,808 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'txnSeqNo': 15, 'viewNo': 0, 'ppSeqNo': 6, 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node3')
2017-08-28 15:28:10,815 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:10,815 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:10,816 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'txnSeqNo': 0, 'viewNo': None, 'ppSeqNo': None, 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node3')
2017-08-28 15:28:10,821 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:10,821 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: BATCH{'signature': None, 'messages': ['{"op":"CURRENT_STATE","viewNo":2,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}]}', '{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","op":"LEDGER_STATUS","txnSeqNo":7,"viewNo":2,"ledgerId":0,"ppSeqNo":6}', '{"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"LEDGER_STATUS","txnSeqNo":15,"viewNo":0,"ledgerId":1,"ppSeqNo":6}', '{"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","op":"LEDGER_STATUS","txnSeqNo":0,"viewNo":null,"ledgerId":2,"ppSeqNo":null}']}
2017-08-28 15:28:10,822 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"CURRENT_STATE","viewNo":2,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}]}', '{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","op":"LEDGER_STATUS","txnSeqNo":7,"viewNo":2,"ledgerId":0,"ppSeqNo":6}', '{"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"LEDGER_STATUS","txnSeqNo":15,"viewNo":0,"ledgerId":1,"ppSeqNo":6}', '{"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","op":"LEDGER_STATUS","txnSeqNo":0,"viewNo":null,"ledgerId":2,"ppSeqNo":null}'], 'op': 'BATCH'}, 'Node2')
2017-08-28 15:28:10,823 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"CURRENT_STATE","viewNo":2,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}]}', '{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","op":"LEDGER_STATUS","txnSeqNo":7,"viewNo":2,"ledgerId":0,"ppSeqNo":6}', '{"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"LEDGER_STATUS","txnSeqNo":15,"viewNo":0,"ledgerId":1,"ppSeqNo":6}', '{"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","op":"LEDGER_STATUS","txnSeqNo":0,"viewNo":null,"ledgerId":2,"ppSeqNo":null}']}
2017-08-28 15:28:10,824 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:10,846 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}], 'op': 'CURRENT_STATE'}, 'Node2')
2017-08-28 15:28:10,846 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:10,847 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:10,847 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'txnSeqNo': 7, 'viewNo': 2, 'ppSeqNo': 6, 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node2')
2017-08-28 15:28:10,848 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:10,850 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:10,852 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'txnSeqNo': 15, 'viewNo': 0, 'ppSeqNo': 6, 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node2')
2017-08-28 15:28:10,853 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:10,853 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:10,853 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'txnSeqNo': 0, 'viewNo': None, 'ppSeqNo': None, 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node2')
2017-08-28 15:28:10,853 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:10,870 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: BATCH{'signature': None, 'messages': ['{"op":"CURRENT_STATE","primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2}', '{"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"ledgerId":0,"viewNo":2}', '{"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","txnSeqNo":15,"ledgerId":1,"viewNo":0}', '{"op":"LEDGER_STATUS","ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","txnSeqNo":0,"ledgerId":2,"viewNo":null}']}
2017-08-28 15:28:10,871 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"CURRENT_STATE","primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2}', '{"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"ledgerId":0,"viewNo":2}', '{"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","txnSeqNo":15,"ledgerId":1,"viewNo":0}', '{"op":"LEDGER_STATUS","ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","txnSeqNo":0,"ledgerId":2,"viewNo":null}'], 'op': 'BATCH'}, 'Node1')
2017-08-28 15:28:10,875 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"CURRENT_STATE","primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2}', '{"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"ledgerId":0,"viewNo":2}', '{"op":"LEDGER_STATUS","ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","txnSeqNo":15,"ledgerId":1,"viewNo":0}', '{"op":"LEDGER_STATUS","ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","txnSeqNo":0,"ledgerId":2,"viewNo":null}']}
2017-08-28 15:28:10,877 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:10,880 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}], 'op': 'CURRENT_STATE'}, 'Node1')
2017-08-28 15:28:10,882 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:10,883 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:10,883 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'txnSeqNo': 7, 'viewNo': 2, 'ppSeqNo': 6, 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node1')
2017-08-28 15:28:10,886 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:10,903 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:10,907 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'txnSeqNo': 15, 'viewNo': 0, 'ppSeqNo': 6, 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node1')
2017-08-28 15:28:10,908 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:10,909 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:10,910 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'txnSeqNo': 0, 'viewNo': None, 'ppSeqNo': None, 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node1')
2017-08-28 15:28:10,912 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:10,912 | DEBUG    | node.py              ( 889) | process_current_state_message | Node6 processing current state CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]} from Node3
2017-08-28 15:28:10,924 | DEBUG    | node.py              (1152) | msgHasAcceptableViewNo | Node6 stashing a message for a future view: VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}
2017-08-28 15:28:10,929 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} from Node3
2017-08-28 15:28:10,929 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:10,930 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:10,930 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'txnSeqNo': 4} to 1 recipients: ['Node3']
2017-08-28 15:28:10,934 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node3
2017-08-28 15:28:10,948 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node3
2017-08-28 15:28:10,948 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node3
2017-08-28 15:28:10,952 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node3
2017-08-28 15:28:10,958 | DEBUG    | node.py              ( 889) | process_current_state_message | Node6 processing current state CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]} from Node2
2017-08-28 15:28:10,959 | DEBUG    | node.py              (1152) | msgHasAcceptableViewNo | Node6 stashing a message for a future view: VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}
2017-08-28 15:28:10,963 | INFO     | node.py              (1948) | _start_view_change_if_possible | VIEW CHANGE: Node6 starting view change for 2 after 2 view change indications from other nodes
2017-08-28 15:28:10,964 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action _check_view_change_completed with id 81 to run in 60 seconds
2017-08-28 15:28:10,964 | DEBUG    | replica.py           ( 453) | on_view_change_start | Node6:0 setting last prepared for master to None
2017-08-28 15:28:10,964 | DEBUG    | node.py              (2127) | startViewChange | Node6 resetting monitor stats after view change
2017-08-28 15:28:10,964 | DEBUG    | monitor.py           ( 192) | reset | Node6's Monitor being reset
2017-08-28 15:28:10,965 | DEBUG    | node.py              (1187) | sendToElector | Node6 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}, 'Node3')
2017-08-28 15:28:10,965 | DEBUG    | node.py              (1187) | sendToElector | Node6 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}, 'Node2')
2017-08-28 15:28:10,965 | DEBUG    | node.py              ( 978) | processStashedMsgsForView | Node6 processed 2 stashed msgs for view no 2
2017-08-28 15:28:10,982 | INFO     | node.py              (2140) | startViewChange | VIEW CHANGE: Node6 changed to view 2, will start catchup now
2017-08-28 15:28:10,985 | DEBUG    | node.py              (2166) | start_catchup | Node6 does not start the catchup procedure because it is already in this state
2017-08-28 15:28:10,986 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} from Node2
2017-08-28 15:28:10,988 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:10,990 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,007 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'txnSeqNo': 4} to 1 recipients: ['Node2']
2017-08-28 15:28:11,008 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node2
2017-08-28 15:28:11,008 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node2
2017-08-28 15:28:11,010 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node2
2017-08-28 15:28:11,013 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node2
2017-08-28 15:28:11,017 | DEBUG    | node.py              ( 889) | process_current_state_message | Node6 processing current state CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]} from Node1
2017-08-28 15:28:11,017 | DEBUG    | node.py              (1187) | sendToElector | Node6 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}, 'Node1')
2017-08-28 15:28:11,018 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} from Node1
2017-08-28 15:28:11,018 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,018 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,019 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'txnSeqNo': 4} to 1 recipients: ['Node1']
2017-08-28 15:28:11,019 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node1
2017-08-28 15:28:11,020 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node1
2017-08-28 15:28:11,020 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node1
2017-08-28 15:28:11,021 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node1
2017-08-28 15:28:11,022 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6C got 1 messages through listener
2017-08-28 15:28:11,022 | TRACE    | node.py              (1370) | validateClientMsg | Node6C received CLIENT message: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:11,023 | DEBUG    | node.py              (1416) | processClientInBox | Node6C processing b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' request LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:11,046 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} from b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:28:11,048 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,048 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,048 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,049 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:28:11,052 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node1
2017-08-28 15:28:11,052 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node1 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:11,052 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node1
2017-08-28 15:28:11,074 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node3
2017-08-28 15:28:11,074 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node3 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:11,075 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node3
2017-08-28 15:28:11,076 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node2
2017-08-28 15:28:11,077 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node2 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:11,077 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node2
2017-08-28 15:28:11,092 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:11,093 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: BATCH{'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2,"op":"CURRENT_STATE"}', '{"txnSeqNo":7,"ledgerId":0,"ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","viewNo":2,"op":"LEDGER_STATUS"}', '{"txnSeqNo":15,"ledgerId":1,"ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","viewNo":0,"op":"LEDGER_STATUS"}', '{"txnSeqNo":0,"ledgerId":2,"ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","viewNo":null,"op":"LEDGER_STATUS"}']}
2017-08-28 15:28:11,104 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2,"op":"CURRENT_STATE"}', '{"txnSeqNo":7,"ledgerId":0,"ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","viewNo":2,"op":"LEDGER_STATUS"}', '{"txnSeqNo":15,"ledgerId":1,"ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","viewNo":0,"op":"LEDGER_STATUS"}', '{"txnSeqNo":0,"ledgerId":2,"ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","viewNo":null,"op":"LEDGER_STATUS"}'], 'op': 'BATCH'}, 'Node4')
2017-08-28 15:28:11,104 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"viewNo":2,"op":"CURRENT_STATE"}', '{"txnSeqNo":7,"ledgerId":0,"ppSeqNo":6,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","viewNo":2,"op":"LEDGER_STATUS"}', '{"txnSeqNo":15,"ledgerId":1,"ppSeqNo":6,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","viewNo":0,"op":"LEDGER_STATUS"}', '{"txnSeqNo":0,"ledgerId":2,"ppSeqNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","viewNo":null,"op":"LEDGER_STATUS"}']}
2017-08-28 15:28:11,115 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:11,116 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}], 'op': 'CURRENT_STATE'}, 'Node4')
2017-08-28 15:28:11,123 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:11,124 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:11,124 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'txnSeqNo': 7, 'viewNo': 2, 'ppSeqNo': 6, 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node4')
2017-08-28 15:28:11,127 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:11,132 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:11,132 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'txnSeqNo': 15, 'viewNo': 0, 'ppSeqNo': 6, 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node4')
2017-08-28 15:28:11,132 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:11,141 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:11,142 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'txnSeqNo': 0, 'viewNo': None, 'ppSeqNo': None, 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node4')
2017-08-28 15:28:11,142 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:11,156 | DEBUG    | node.py              ( 889) | process_current_state_message | Node6 processing current state CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]} from Node4
2017-08-28 15:28:11,159 | DEBUG    | node.py              (1187) | sendToElector | Node6 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}, 'Node4')
2017-08-28 15:28:11,160 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} from Node4
2017-08-28 15:28:11,164 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,164 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 4 with 7
2017-08-28 15:28:11,167 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'txnSeqNo': 4} to 1 recipients: ['Node4']
2017-08-28 15:28:11,171 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node4
2017-08-28 15:28:11,172 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node4
2017-08-28 15:28:11,172 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node4
2017-08-28 15:28:11,176 | DEBUG    | ledger_manager.py    (1016) | stashLedgerStatus | Node6 stashing ledger status LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node4
2017-08-28 15:28:11,190 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node4
2017-08-28 15:28:11,191 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node4 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:11,200 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"merkleRoot":"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3","txnSeqNo":4,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}' to Node4
2017-08-28 15:28:11,202 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:28:11,216 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,219 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'op': 'CONSISTENCY_PROOF'}, 'Node2')
2017-08-28 15:28:11,224 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,225 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,225 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'op': 'CONSISTENCY_PROOF'}, 'Node3')
2017-08-28 15:28:11,227 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,236 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,236 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'op': 'CONSISTENCY_PROOF'}, 'Node1')
2017-08-28 15:28:11,243 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,248 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0} from Node2
2017-08-28 15:28:11,249 | DEBUG    | ledger_manager.py    ( 706) | canStartCatchUpProcess | Node6 cannot start catchup since received only 1 consistency proofs but need at least 2
2017-08-28 15:28:11,253 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0} from Node3
2017-08-28 15:28:11,254 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action partial(request_CPs_if_needed) with id 1 to run in 20 seconds
2017-08-28 15:28:11,268 | DEBUG    | ledger_manager.py    ( 688) | canStartCatchUpProcess | Node6 deciding on the basis of CPs {'Node3': CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}, 'Node2': CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}} and f 1
2017-08-28 15:28:11,269 | DEBUG    | node.py              (1461) | preLedgerCatchUp | Node6 going to process any ordered requests before starting catchup.
2017-08-28 15:28:11,269 | DEBUG    | replica.py           (2199) | _remove_ordered_from_queue | Node6:0 going to remove 0 Ordered messages from outbox
2017-08-28 15:28:11,270 | DEBUG    | node.py              (1869) | force_process_ordered | Node6 processed 0 Ordered batches for instance 0 before starting catch up
2017-08-28 15:28:11,279 | DEBUG    | replica.py           (2199) | _remove_ordered_from_queue | Node6:1 going to remove 0 Ordered messages from outbox
2017-08-28 15:28:11,280 | DEBUG    | node.py              (1869) | force_process_ordered | Node6 processed 0 Ordered batches for instance 1 before starting catch up
2017-08-28 15:28:11,280 | DEBUG    | node.py              (2427) | processStashedOrderedReqs | Node6 processed 0 stashed ordered requests
2017-08-28 15:28:11,280 | DEBUG    | monitor.py           ( 192) | reset | Node6's Monitor being reset
2017-08-28 15:28:11,282 | INFO     | node.py              (1471) | preLedgerCatchUp | Node6 reverted 0 batches before starting catch up for ledger 0
2017-08-28 15:28:11,283 | DEBUG    | ledger_manager.py    ( 788) | startCatchUpProcess | Node6 started catching up with consistency proof CONSISTENCY_PROOF{'hashes': ('DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q',), 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,284 | DEBUG    | node.py              (2566) | send | Node6 sending message CATCHUP_REQ{'catchupTill': 7, 'seqNoStart': 5, 'ledgerId': 0, 'seqNoEnd': 5} to 1 recipients: ['Node1']
2017-08-28 15:28:11,287 | DEBUG    | node.py              (2566) | send | Node6 sending message CATCHUP_REQ{'catchupTill': 7, 'seqNoStart': 6, 'ledgerId': 0, 'seqNoEnd': 6} to 1 recipients: ['Node2']
2017-08-28 15:28:11,288 | DEBUG    | node.py              (2566) | send | Node6 sending message CATCHUP_REQ{'catchupTill': 7, 'seqNoStart': 7, 'ledgerId': 0, 'seqNoEnd': 7} to 1 recipients: ['Node3']
2017-08-28 15:28:11,303 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action partial(request_txns_if_needed) with id 2 to run in 15.299999999999999 seconds
2017-08-28 15:28:11,311 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0} from Node1
2017-08-28 15:28:11,312 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:11,323 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"catchupTill":7,"seqNoStart":5,"ledgerId":0,"op":"CATCHUP_REQ","seqNoEnd":5}' to Node1
2017-08-28 15:28:11,329 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node1 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:11,330 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"catchupTill":7,"seqNoStart":5,"ledgerId":0,"op":"CATCHUP_REQ","seqNoEnd":5}' to Node1
2017-08-28 15:28:11,330 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"catchupTill":7,"seqNoStart":7,"ledgerId":0,"op":"CATCHUP_REQ","seqNoEnd":7}' to Node3
2017-08-28 15:28:11,331 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node3 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:11,334 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"catchupTill":7,"seqNoStart":7,"ledgerId":0,"op":"CATCHUP_REQ","seqNoEnd":7}' to Node3
2017-08-28 15:28:11,338 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"catchupTill":7,"seqNoStart":6,"ledgerId":0,"op":"CATCHUP_REQ","seqNoEnd":6}' to Node2
2017-08-28 15:28:11,338 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node2 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:11,339 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"catchupTill":7,"seqNoStart":6,"ledgerId":0,"op":"CATCHUP_REQ","seqNoEnd":6}' to Node2
2017-08-28 15:28:11,352 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:11,353 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,368 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0, 'op': 'CONSISTENCY_PROOF'}, 'Node4')
2017-08-28 15:28:11,368 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0}
2017-08-28 15:28:11,369 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['DNjKvRRd7MQg4SM2yfQUrYwS6akgsWoerDTr6Rc4kF2q'], 'seqNoStart': 4, 'seqNoEnd': 7, 'viewNo': 2, 'newMerkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 6, 'oldMerkleRoot': '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3', 'ledgerId': 0} from Node4
2017-08-28 15:28:11,380 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:11,408 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.923 seconds
2017-08-28 15:28:11,424 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:28:11,426 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: CATCHUP_REP{'txns': {'6': {'signature': '4JursX6ywfiH2shqqBEXQX5r852y5szJUBZ8YWSeLeueC3jHK6pUGTqJ3mVrw76g1RzPxcJbtM2wMjnT9Ahp8VbT', 'reqId': 1503934008154939, 'type': '0', 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'txnTime': 1503934008}}, 'ledgerId': 0, 'consProof': ['5yYbh3Csn1bGm21gsiv6vuQBMZg8dedahhTCWvr7WPk2', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}
2017-08-28 15:28:11,441 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'txns': {'6': {'signature': '4JursX6ywfiH2shqqBEXQX5r852y5szJUBZ8YWSeLeueC3jHK6pUGTqJ3mVrw76g1RzPxcJbtM2wMjnT9Ahp8VbT', 'reqId': 1503934008154939, 'type': '0', 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'txnTime': 1503934008}}, 'ledgerId': 0, 'op': 'CATCHUP_REP', 'consProof': ['5yYbh3Csn1bGm21gsiv6vuQBMZg8dedahhTCWvr7WPk2', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}, 'Node2')
2017-08-28 15:28:11,441 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CATCHUP_REP{'txns': {'6': {'signature': '4JursX6ywfiH2shqqBEXQX5r852y5szJUBZ8YWSeLeueC3jHK6pUGTqJ3mVrw76g1RzPxcJbtM2wMjnT9Ahp8VbT', 'reqId': 1503934008154939, 'type': '0', 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'txnTime': 1503934008}}, 'ledgerId': 0, 'consProof': ['5yYbh3Csn1bGm21gsiv6vuQBMZg8dedahhTCWvr7WPk2', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}
2017-08-28 15:28:11,443 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: CATCHUP_REP{'txns': {'7': {'signature': 'zhfYef2NnAS7vCPvewudttE57nFzERPnjyxvcerkg6x5fXuqvy81kN3uLESCSJjuVNRqs8yhfq2wGbSJbWa2odQ', 'reqId': 1503934088807442, 'txnTime': 1503934089, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd', 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': ['VALIDATOR']}, 'type': '0'}}, 'ledgerId': 0, 'consProof': []}
2017-08-28 15:28:11,453 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'txns': {'7': {'signature': 'zhfYef2NnAS7vCPvewudttE57nFzERPnjyxvcerkg6x5fXuqvy81kN3uLESCSJjuVNRqs8yhfq2wGbSJbWa2odQ', 'reqId': 1503934088807442, 'txnTime': 1503934089, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd', 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': ['VALIDATOR']}, 'type': '0'}}, 'ledgerId': 0, 'op': 'CATCHUP_REP', 'consProof': []}, 'Node3')
2017-08-28 15:28:11,454 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CATCHUP_REP{'txns': {'7': {'signature': 'zhfYef2NnAS7vCPvewudttE57nFzERPnjyxvcerkg6x5fXuqvy81kN3uLESCSJjuVNRqs8yhfq2wGbSJbWa2odQ', 'reqId': 1503934088807442, 'txnTime': 1503934089, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd', 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': ['VALIDATOR']}, 'type': '0'}}, 'ledgerId': 0, 'consProof': []}
2017-08-28 15:28:11,455 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CATCHUP_REP{'txns': {'5': {'signature': '5odcUT8Bzxrau3HGCBQ71nGHPmZa7SJmWhzTWT4BHHxL6mxYgUfoH6mahDvRQwSE1gnGNsUgqci23WXwQLehiDAA', 'reqId': 1503933979830988, 'txnTime': 1503933980, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.105', 'node_ip': '10.0.0.105', 'node_port': 9701}, 'type': '0'}}, 'ledgerId': 0, 'consProof': ['Etup9NgC9tEUzHq8fzn8VAZ6YyTW7YyYvrvYYtRZbwdj', '4K3SsKB7U1ZgzjLhZubLH7qJmEKWBQdGmPXCEGAEHP1g', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}
2017-08-28 15:28:11,471 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'txns': {'5': {'signature': '5odcUT8Bzxrau3HGCBQ71nGHPmZa7SJmWhzTWT4BHHxL6mxYgUfoH6mahDvRQwSE1gnGNsUgqci23WXwQLehiDAA', 'reqId': 1503933979830988, 'txnTime': 1503933980, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.105', 'node_ip': '10.0.0.105', 'node_port': 9701}, 'type': '0'}}, 'ledgerId': 0, 'op': 'CATCHUP_REP', 'consProof': ['Etup9NgC9tEUzHq8fzn8VAZ6YyTW7YyYvrvYYtRZbwdj', '4K3SsKB7U1ZgzjLhZubLH7qJmEKWBQdGmPXCEGAEHP1g', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}, 'Node1')
2017-08-28 15:28:11,479 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CATCHUP_REP{'txns': {'5': {'signature': '5odcUT8Bzxrau3HGCBQ71nGHPmZa7SJmWhzTWT4BHHxL6mxYgUfoH6mahDvRQwSE1gnGNsUgqci23WXwQLehiDAA', 'reqId': 1503933979830988, 'txnTime': 1503933980, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.105', 'node_ip': '10.0.0.105', 'node_port': 9701}, 'type': '0'}}, 'ledgerId': 0, 'consProof': ['Etup9NgC9tEUzHq8fzn8VAZ6YyTW7YyYvrvYYtRZbwdj', '4K3SsKB7U1ZgzjLhZubLH7qJmEKWBQdGmPXCEGAEHP1g', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}
2017-08-28 15:28:11,481 | DEBUG    | ledger_manager.py    ( 455) | processCatchupRep | Node6 received catchup reply from Node2: CATCHUP_REP{'txns': {'6': {'signature': '4JursX6ywfiH2shqqBEXQX5r852y5szJUBZ8YWSeLeueC3jHK6pUGTqJ3mVrw76g1RzPxcJbtM2wMjnT9Ahp8VbT', 'reqId': 1503934008154939, 'type': '0', 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'txnTime': 1503934008}}, 'ledgerId': 0, 'consProof': ['5yYbh3Csn1bGm21gsiv6vuQBMZg8dedahhTCWvr7WPk2', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}
2017-08-28 15:28:11,487 | DEBUG    | ledger_manager.py    ( 460) | processCatchupRep | Node6 found 1 transactions in the catchup from Node2
2017-08-28 15:28:11,495 | DEBUG    | ledger_manager.py    ( 477) | processCatchupRep | Node6 merging all received catchups
2017-08-28 15:28:11,496 | DEBUG    | ledger_manager.py    ( 485) | processCatchupRep | Node6 merged catchups, there are 1 of them now, from 6 to 6
2017-08-28 15:28:11,496 | DEBUG    | ledger_manager.py    ( 493) | processCatchupRep | Node6 processed 0 catchup replies with sequence numbers []
2017-08-28 15:28:11,499 | DEBUG    | ledger_manager.py    ( 455) | processCatchupRep | Node6 received catchup reply from Node3: CATCHUP_REP{'txns': {'7': {'signature': 'zhfYef2NnAS7vCPvewudttE57nFzERPnjyxvcerkg6x5fXuqvy81kN3uLESCSJjuVNRqs8yhfq2wGbSJbWa2odQ', 'reqId': 1503934088807442, 'txnTime': 1503934089, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd', 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': ['VALIDATOR']}, 'type': '0'}}, 'ledgerId': 0, 'consProof': []}
2017-08-28 15:28:11,505 | DEBUG    | ledger_manager.py    ( 460) | processCatchupRep | Node6 found 1 transactions in the catchup from Node3
2017-08-28 15:28:11,507 | DEBUG    | ledger_manager.py    ( 477) | processCatchupRep | Node6 merging all received catchups
2017-08-28 15:28:11,518 | DEBUG    | ledger_manager.py    ( 485) | processCatchupRep | Node6 merged catchups, there are 2 of them now, from 6 to 7
2017-08-28 15:28:11,519 | DEBUG    | ledger_manager.py    ( 493) | processCatchupRep | Node6 processed 0 catchup replies with sequence numbers []
2017-08-28 15:28:11,525 | DEBUG    | ledger_manager.py    ( 455) | processCatchupRep | Node6 received catchup reply from Node1: CATCHUP_REP{'txns': {'5': {'signature': '5odcUT8Bzxrau3HGCBQ71nGHPmZa7SJmWhzTWT4BHHxL6mxYgUfoH6mahDvRQwSE1gnGNsUgqci23WXwQLehiDAA', 'reqId': 1503933979830988, 'txnTime': 1503933980, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.105', 'node_ip': '10.0.0.105', 'node_port': 9701}, 'type': '0'}}, 'ledgerId': 0, 'consProof': ['Etup9NgC9tEUzHq8fzn8VAZ6YyTW7YyYvrvYYtRZbwdj', '4K3SsKB7U1ZgzjLhZubLH7qJmEKWBQdGmPXCEGAEHP1g', 'GMnnuWPjRegPoQDBTWCYX5rJEvo6y4odvZmf3yabHbLM', '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3']}
2017-08-28 15:28:11,526 | DEBUG    | ledger_manager.py    ( 460) | processCatchupRep | Node6 found 1 transactions in the catchup from Node1
2017-08-28 15:28:11,527 | DEBUG    | ledger_manager.py    ( 477) | processCatchupRep | Node6 merging all received catchups
2017-08-28 15:28:11,527 | DEBUG    | ledger_manager.py    ( 485) | processCatchupRep | Node6 merged catchups, there are 3 of them now, from 5 to 7
2017-08-28 15:28:11,528 | DEBUG    | ledger_manager.py    ( 595) | hasValidCatchupReplies | Node6 verifying proof for 5, 7, b'v\xe7n\x85\xea\xa0{[t\x194T\xb6h\x01S\xcf\xe7\xc5Z\xc3N5U\x15\xee\xf0\xf7#\xc2\xc3\xd9', b'K\xfc\xa00\x88\x15-\x1f\xe3O\xd4?Z\x86\xa7\x9a\xd0L\xa5=6\x06`\xae[67a\xb8v\x00\xd3', [b'\xcetDW\xf6\xfe}\xf0\xfe\x0c\x93\xfd7\x11\xc3\xd3\xedv\xa6\xa1\x18!\xddYO\xe9\xa7&\x1e\x99\x11z', b'12G\xf1\xc9i^\xb0Z\xcayG`f\xea\xf5\x07<0@\xd6m\xec\x16\xcb\xfc\x8b\xdd)w\xb5o', b'\xe42\xb2X&\x1f\x1f\\\xf9\x83\xba%\xddj,\x142\xb7A\xa5A\xe5\xc4\xfbz\xbb\x84"\xd7\x81dj', b"I\xb5*\x8a&f\x9e\x95\x8a\x93\xcc8\x0b\xfc\xb6\xfa\x9b\xd3R\x0e\xef\x87'\xba\x99\x89\xb2\x00H\x99m\xbe"]
2017-08-28 15:28:11,563 | DEBUG    | stack_manager.py     ( 160) | connectNewRemote | Node6 adding new node Node5 with HA ('10.0.0.105', 9701)
2017-08-28 15:28:11,563 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:11,564 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:11,565 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:11,565 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:11,567 | DEBUG    | kit_zstack.py        (  97) | connectToMissing | CONNECTION: Node6 found the following missing connections: Node5
2017-08-28 15:28:11,583 | TRACE    | remote.py            (  84) | connect | connecting socket 93 45716448 to remote Node5:HA(host='10.0.0.105', port=9701)
2017-08-28 15:28:11,583 | INFO     | zstack.py            ( 588) | connect | CONNECTION: Node6 looking for Node5 at 10.0.0.105:9701
2017-08-28 15:28:11,584 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node5
2017-08-28 15:28:11,584 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:28:11,586 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:28:11,587 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:28:11,591 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:28:11,591 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:28:11,593 | DEBUG    | node.py              ( 861) | sendPoolInfoToClients | Node6 sending new node info {'signature': '5odcUT8Bzxrau3HGCBQ71nGHPmZa7SJmWhzTWT4BHHxL6mxYgUfoH6mahDvRQwSE1gnGNsUgqci23WXwQLehiDAA', 'reqId': 1503933979830988, 'txnTime': 1503933980, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.105', 'node_ip': '10.0.0.105', 'node_port': 9701}, 'type': '0', 'seqNo': 5} to all clients
2017-08-28 15:28:11,594 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"txn":{"signature":"5odcUT8Bzxrau3HGCBQ71nGHPmZa7SJmWhzTWT4BHHxL6mxYgUfoH6mahDvRQwSE1gnGNsUgqci23WXwQLehiDAA","reqId":1503933979830988,"txnTime":1503933980,"identifier":"XhYtvJqezMUKfF6KVNaGmT","dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc","data":{"client_port":9702,"alias":"Node5","services":["VALIDATOR"],"client_ip":"10.0.0.105","node_ip":"10.0.0.105","node_port":9701},"type":"0","seqNo":5},"op":"POOL_LEDGER_TXNS"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:28:11,617 | DEBUG    | ledger_manager.py    ( 595) | hasValidCatchupReplies | Node6 verifying proof for 6, 7, b'_G\x046\x82\x17\x10\x98~\xc0b\x0b\xa3n\xb0\xe6\xd2\xb8\xdd\xe5\xf5\xe5\xb7s\xc2\xfe\xcaF>\x08\xfd\xa7', b'K\xfc\xa00\x88\x15-\x1f\xe3O\xd4?Z\x86\xa7\x9a\xd0L\xa5=6\x06`\xae[67a\xb8v\x00\xd3', [b'I\xeb\x00H\xb0@\x89Vy\xc4\xe4\x87n\xf2m_N\x18\xc6\x0e\xf7\xae\xd4Yv"ey\xcaE\x0e\x1f', b'\xe42\xb2X&\x1f\x1f\\\xf9\x83\xba%\xddj,\x142\xb7A\xa5A\xe5\xc4\xfbz\xbb\x84"\xd7\x81dj', b"I\xb5*\x8a&f\x9e\x95\x8a\x93\xcc8\x0b\xfc\xb6\xfa\x9b\xd3R\x0e\xef\x87'\xba\x99\x89\xb2\x00H\x99m\xbe"]
2017-08-28 15:28:11,643 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node5:HA(host='10.0.0.105', port=9701)
2017-08-28 15:28:11,645 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 93 45716448
2017-08-28 15:28:11,646 | DEBUG    | stack_manager.py     ( 212) | removeRemote | Node6 removed remote Node5
2017-08-28 15:28:11,646 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:11,646 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:11,646 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:11,647 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:11,663 | DEBUG    | kit_zstack.py        (  97) | connectToMissing | CONNECTION: Node6 found the following missing connections: Node5
2017-08-28 15:28:11,664 | TRACE    | remote.py            (  84) | connect | connecting socket 93 45716448 to remote Node5:HA(host='10.0.0.6', port=9701)
2017-08-28 15:28:11,664 | INFO     | zstack.py            ( 588) | connect | CONNECTION: Node6 looking for Node5 at 10.0.0.6:9701
2017-08-28 15:28:11,667 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node5
2017-08-28 15:28:11,667 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:11,668 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:11,668 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 78 45623280
2017-08-28 15:28:11,673 | TRACE    | remote.py            (  84) | connect | connecting socket 78 45623280 to remote Node1:HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:11,674 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node1
2017-08-28 15:28:11,709 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:11,710 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:11,710 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 81 45409504
2017-08-28 15:28:11,711 | TRACE    | remote.py            (  84) | connect | connecting socket 95 45765984 to remote Node3:HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:11,712 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node3
2017-08-28 15:28:11,712 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:11,716 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:11,716 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 85 45630352
2017-08-28 15:28:11,717 | TRACE    | remote.py            (  84) | connect | connecting socket 96 45800464 to remote Node2:HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:11,742 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node2
2017-08-28 15:28:11,743 | DEBUG    | zstack.py            ( 603) | reconnectRemote | Node6 reconnecting to Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:11,751 | DEBUG    | remote.py            (  87) | disconnect | disconnecting remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:11,755 | TRACE    | remote.py            (  90) | disconnect | disconnecting socket 86 45632000
2017-08-28 15:28:11,756 | TRACE    | remote.py            (  84) | connect | connecting socket 97 45552320 to remote Node4:HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:11,759 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node4
2017-08-28 15:28:11,759 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:28:11,769 | DEBUG    | pool_manager.py      ( 298) | nodeServicesChanged | Node 4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc not changing services since it is same as existing
2017-08-28 15:28:11,770 | DEBUG    | node.py              ( 861) | sendPoolInfoToClients | Node6 sending new node info {'signature': '4JursX6ywfiH2shqqBEXQX5r852y5szJUBZ8YWSeLeueC3jHK6pUGTqJ3mVrw76g1RzPxcJbtM2wMjnT9Ahp8VbT', 'reqId': 1503934008154939, 'type': '0', 'identifier': 'XhYtvJqezMUKfF6KVNaGmT', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': ['VALIDATOR'], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc', 'txnTime': 1503934008, 'seqNo': 6} to all clients
2017-08-28 15:28:11,785 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"txn":{"signature":"4JursX6ywfiH2shqqBEXQX5r852y5szJUBZ8YWSeLeueC3jHK6pUGTqJ3mVrw76g1RzPxcJbtM2wMjnT9Ahp8VbT","reqId":1503934008154939,"type":"0","identifier":"XhYtvJqezMUKfF6KVNaGmT","data":{"client_port":9702,"node_ip":"10.0.0.6","services":["VALIDATOR"],"client_ip":"10.0.0.6","alias":"Node5","node_port":9701},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc","txnTime":1503934008,"seqNo":6},"op":"POOL_LEDGER_TXNS"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:28:11,793 | DEBUG    | ledger_manager.py    ( 595) | hasValidCatchupReplies | Node6 verifying proof for 7, 7, b'K\xfc\xa00\x88\x15-\x1f\xe3O\xd4?Z\x86\xa7\x9a\xd0L\xa5=6\x06`\xae[67a\xb8v\x00\xd3', b'K\xfc\xa00\x88\x15-\x1f\xe3O\xd4?Z\x86\xa7\x9a\xd0L\xa5=6\x06`\xae[67a\xb8v\x00\xd3', []
2017-08-28 15:28:11,832 | DEBUG    | pool_manager.py      ( 298) | nodeServicesChanged | Node 6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G not changing services since it is same as existing
2017-08-28 15:28:11,843 | DEBUG    | node.py              ( 861) | sendPoolInfoToClients | Node6 sending new node info {'signature': 'zhfYef2NnAS7vCPvewudttE57nFzERPnjyxvcerkg6x5fXuqvy81kN3uLESCSJjuVNRqs8yhfq2wGbSJbWa2odQ', 'reqId': 1503934088807442, 'txnTime': 1503934089, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd', 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': ['VALIDATOR']}, 'type': '0', 'seqNo': 7} to all clients
2017-08-28 15:28:11,844 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"txn":{"signature":"zhfYef2NnAS7vCPvewudttE57nFzERPnjyxvcerkg6x5fXuqvy81kN3uLESCSJjuVNRqs8yhfq2wGbSJbWa2odQ","reqId":1503934088807442,"txnTime":1503934089,"identifier":"FBQ5W7LxRp8bLeVNEafjpd","dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G","data":{"client_port":9702,"alias":"Node6","node_port":9701,"client_ip":"10.0.0.7","node_ip":"10.0.0.7","services":["VALIDATOR"]},"type":"0","seqNo":7},"op":"POOL_LEDGER_TXNS"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:28:11,876 | DEBUG    | ledger_manager.py    ( 493) | processCatchupRep | Node6 processed 3 catchup replies with sequence numbers [5, 6, 7]
2017-08-28 15:28:11,878 | DEBUG    | node.py              ( 923) | checkInstances | Node6 choosing to start election on the basis of count 1 and nodes set()
2017-08-28 15:28:11,881 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node1']
2017-08-28 15:28:11,881 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node1 for ledger status of ledger 2
2017-08-28 15:28:11,881 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node2']
2017-08-28 15:28:11,882 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node2 for ledger status of ledger 2
2017-08-28 15:28:11,921 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node3']
2017-08-28 15:28:11,921 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node3 for ledger status of ledger 2
2017-08-28 15:28:11,922 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node4']
2017-08-28 15:28:11,928 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node4 for ledger status of ledger 2
2017-08-28 15:28:11,929 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:28:11,929 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node6 for ledger status of ledger 2
2017-08-28 15:28:11,930 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}} to 1 recipients: ['Node5']
2017-08-28 15:28:11,939 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node5 for ledger status of ledger 2
2017-08-28 15:28:11,939 | DEBUG    | ledger_manager.py    (1027) | processStashedLedgerStatuses | Node6 going to process 4 stashed ledger statuses for ledger 2
2017-08-28 15:28:11,943 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node3
2017-08-28 15:28:11,944 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:11,961 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:11,962 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node2
2017-08-28 15:28:11,962 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:11,964 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:11,964 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node1
2017-08-28 15:28:11,980 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:11,996 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,000 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node4
2017-08-28 15:28:12,006 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,014 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,015 | DEBUG    | ledger_manager.py    ( 312) | processLedgerStatus | Node6 found out from {'Node1', 'Node3', 'Node2', 'Node4'} that its ledger of type 2 is latest
2017-08-28 15:28:12,015 | DEBUG    | ledger_manager.py    ( 315) | processLedgerStatus | Node6 found from ledger status LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} that it does not need catchup
2017-08-28 15:28:12,023 | DEBUG    | node.py              (1461) | preLedgerCatchUp | Node6 going to process any ordered requests before starting catchup.
2017-08-28 15:28:12,027 | DEBUG    | replica.py           (2199) | _remove_ordered_from_queue | Node6:0 going to remove 0 Ordered messages from outbox
2017-08-28 15:28:12,028 | DEBUG    | node.py              (1869) | force_process_ordered | Node6 processed 0 Ordered batches for instance 0 before starting catch up
2017-08-28 15:28:12,028 | DEBUG    | replica.py           (2199) | _remove_ordered_from_queue | Node6:1 going to remove 0 Ordered messages from outbox
2017-08-28 15:28:12,028 | DEBUG    | node.py              (1869) | force_process_ordered | Node6 processed 0 Ordered batches for instance 1 before starting catch up
2017-08-28 15:28:12,029 | DEBUG    | node.py              (2427) | processStashedOrderedReqs | Node6 processed 0 stashed ordered requests
2017-08-28 15:28:12,029 | DEBUG    | monitor.py           ( 192) | reset | Node6's Monitor being reset
2017-08-28 15:28:12,031 | INFO     | node.py              (1471) | preLedgerCatchUp | Node6 reverted 0 batches before starting catch up for ledger 2
2017-08-28 15:28:12,037 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,038 | DEBUG    | pool_config.py       (  35) | processLedger | <sovrin_node.server.pool_config.PoolConfig object at 0x7f1e70ebf898> processing config ledger for any POOL_CONFIGs
2017-08-28 15:28:12,038 | DEBUG    | upgrader.py          ( 208) | processLedger | 6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G processing config ledger for any upgrades
2017-08-28 15:28:12,038 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node1']
2017-08-28 15:28:12,038 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node1 for ledger status of ledger 1
2017-08-28 15:28:12,039 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node2']
2017-08-28 15:28:12,039 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node2 for ledger status of ledger 1
2017-08-28 15:28:12,039 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node3']
2017-08-28 15:28:12,039 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node3 for ledger status of ledger 1
2017-08-28 15:28:12,099 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node4']
2017-08-28 15:28:12,101 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node4 for ledger status of ledger 1
2017-08-28 15:28:12,101 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:28:12,102 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node6 for ledger status of ledger 1
2017-08-28 15:28:12,108 | DEBUG    | node.py              (2566) | send | Node6 sending message MESSAGE_REQUEST{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}} to 1 recipients: ['Node5']
2017-08-28 15:28:12,108 | DEBUG    | node.py              ( 837) | _ask_for_ledger_status | Node6 asking Node5 for ledger status of ledger 1
2017-08-28 15:28:12,108 | DEBUG    | ledger_manager.py    (1027) | processStashedLedgerStatuses | Node6 going to process 4 stashed ledger statuses for ledger 1
2017-08-28 15:28:12,108 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node3
2017-08-28 15:28:12,112 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,114 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,120 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node3']
2017-08-28 15:28:12,120 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node2
2017-08-28 15:28:12,128 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,128 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,129 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node2']
2017-08-28 15:28:12,144 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node1
2017-08-28 15:28:12,145 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,145 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,166 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node1']
2017-08-28 15:28:12,167 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node4
2017-08-28 15:28:12,167 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,172 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,172 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node4']
2017-08-28 15:28:12,196 | INFO     | upgrader.py          ( 150) | should_notify_about_upgrade_result | Node's 'Node6' last upgrade txn is None
2017-08-28 15:28:12,196 | INFO     | ledger_manager.py    ( 848) | catchupCompleted | CATCH-UP: Node6 completed catching up ledger 2, caught up 0 in total
2017-08-28 15:28:12,200 | INFO     | ledger_manager.py    ( 848) | catchupCompleted | CATCH-UP: Node6 completed catching up ledger 0, caught up 3 in total
2017-08-28 15:28:12,202 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 4 msgs to Node5 into one transmission
2017-08-28 15:28:12,205 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}'])
2017-08-28 15:28:12,206 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node5: b'{"signature":null,"messages":["{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}"],"op":"BATCH"}'
2017-08-28 15:28:12,207 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}"],"op":"BATCH"}' to Node5
2017-08-28 15:28:12,222 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node5 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:12,222 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 7 msgs to Node4 into one transmission
2017-08-28 15:28:12,223 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'pi', b'pi', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:12,232 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node4: b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:12,232 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node4
2017-08-28 15:28:12,233 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node4 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:12,238 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 7 msgs to Node1 into one transmission
2017-08-28 15:28:12,238 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'pi', b'pi', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:12,244 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node1: b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:12,255 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node1
2017-08-28 15:28:12,256 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node1 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:12,256 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 7 msgs to Node3 into one transmission
2017-08-28 15:28:12,259 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'pi', b'pi', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:12,260 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node3: b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:12,274 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node3
2017-08-28 15:28:12,280 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node3 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:12,281 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 7 msgs to Node2 into one transmission
2017-08-28 15:28:12,281 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'pi', b'pi', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","op":"MESSAGE_REQUEST"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:12,289 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node2: b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:12,297 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["pi","pi","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":2},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"params\\":{\\"ledgerId\\":1},\\"msg_type\\":\\"LEDGER_STATUS\\",\\"op\\":\\"MESSAGE_REQUEST\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node2
2017-08-28 15:28:12,301 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node2 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:12,301 | DEBUG    | looper.py            ( 212) | runOnceNicely | it took 0.882 seconds to run once nicely
2017-08-28 15:28:12,318 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.899 seconds
2017-08-28 15:28:12,367 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:12,368 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":null,"txnSeqNo":15,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":null,"txnSeqNo":15,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}']}
2017-08-28 15:28:12,368 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":null,"txnSeqNo":15,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":null,"txnSeqNo":15,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:28:12,369 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":2},"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":null,"txnSeqNo":15,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":null,"txnSeqNo":15,"viewNo":null},"msg_type":"LEDGER_STATUS","op":"MESSAGE_RESPONSE"}']}
2017-08-28 15:28:12,369 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,369 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node5')
2017-08-28 15:28:12,369 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,370 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,370 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'params': {'ledgerId': 2}, 'msg_type': 'LEDGER_STATUS'}, 'Node5')
2017-08-28 15:28:12,370 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,370 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,370 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node5')
2017-08-28 15:28:12,370 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,371 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,371 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'params': {'ledgerId': 1}, 'msg_type': 'LEDGER_STATUS'}, 'Node5')
2017-08-28 15:28:12,396 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,416 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node5
2017-08-28 15:28:12,417 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,428 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,428 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node5
2017-08-28 15:28:12,429 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,436 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,436 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node5
2017-08-28 15:28:12,444 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,445 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,445 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node5']
2017-08-28 15:28:12,456 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node5
2017-08-28 15:28:12,456 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,456 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,458 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node5']
2017-08-28 15:28:12,470 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node5 into one transmission
2017-08-28 15:28:12,470 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:12,471 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node5: b'{"signature":null,"messages":["{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:12,474 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node5
2017-08-28 15:28:12,488 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node5 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:12,489 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 4 messages through listener
2017-08-28 15:28:12,502 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node4
2017-08-28 15:28:12,503 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node4
2017-08-28 15:28:12,505 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: BATCH{'signature': None, 'messages': ['{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE"}', '{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}']}
2017-08-28 15:28:12,507 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE"}', '{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}'], 'op': 'BATCH'}, 'Node4')
2017-08-28 15:28:12,519 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE"}', '{"params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE"}', '{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}']}
2017-08-28 15:28:12,520 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,532 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}, 'Node4')
2017-08-28 15:28:12,532 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,532 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,539 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}, 'Node4')
2017-08-28 15:28:12,539 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,540 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,540 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}, 'Node4')
2017-08-28 15:28:12,540 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,550 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,551 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}, 'Node4')
2017-08-28 15:28:12,562 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,562 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,563 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node4')
2017-08-28 15:28:12,568 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,568 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node1
2017-08-28 15:28:12,581 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node1
2017-08-28 15:28:12,582 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: BATCH{'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}']}
2017-08-28 15:28:12,582 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}'], 'op': 'BATCH'}, 'Node1')
2017-08-28 15:28:12,598 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}']}
2017-08-28 15:28:12,599 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,602 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'msg_type': 'LEDGER_STATUS'}, 'Node1')
2017-08-28 15:28:12,603 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,604 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,604 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'msg_type': 'LEDGER_STATUS'}, 'Node1')
2017-08-28 15:28:12,605 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,605 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,605 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'msg_type': 'LEDGER_STATUS'}, 'Node1')
2017-08-28 15:28:12,621 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,621 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,622 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'msg_type': 'LEDGER_STATUS'}, 'Node1')
2017-08-28 15:28:12,622 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,622 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,626 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node1')
2017-08-28 15:28:12,627 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,628 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node3
2017-08-28 15:28:12,628 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node3
2017-08-28 15:28:12,629 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0}}', '{"seqNoEnd":15,"seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","ledgerId":1,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6}']}
2017-08-28 15:28:12,651 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0}}', '{"seqNoEnd":15,"seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","ledgerId":1,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6}'], 'op': 'BATCH'}, 'Node3')
2017-08-28 15:28:12,654 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"MESSAGE_RESPONSE","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":2},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0}}', '{"op":"MESSAGE_RESPONSE","params":{"ledgerId":1},"msg_type":"LEDGER_STATUS","msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0}}', '{"seqNoEnd":15,"seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","ledgerId":1,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6}']}
2017-08-28 15:28:12,655 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,655 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS'}, 'Node3')
2017-08-28 15:28:12,666 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,670 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,670 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS'}, 'Node3')
2017-08-28 15:28:12,671 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,680 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,681 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS'}, 'Node3')
2017-08-28 15:28:12,681 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,682 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,683 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg_type': 'LEDGER_STATUS'}, 'Node3')
2017-08-28 15:28:12,683 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,688 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,689 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node3')
2017-08-28 15:28:12,707 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,708 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node2
2017-08-28 15:28:12,708 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node2
2017-08-28 15:28:12,709 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: BATCH{'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"op":"CONSISTENCY_PROOF","viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"ppSeqNo":6,"seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ledgerId":1}']}
2017-08-28 15:28:12,723 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"op":"CONSISTENCY_PROOF","viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"ppSeqNo":6,"seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ledgerId":1}'], 'op': 'BATCH'}, 'Node2')
2017-08-28 15:28:12,726 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":2,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","ppSeqNo":null,"txnSeqNo":0,"viewNo":null},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":2}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"msg":{"ledgerId":1,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ppSeqNo":6,"txnSeqNo":15,"viewNo":0},"op":"MESSAGE_RESPONSE","msg_type":"LEDGER_STATUS","params":{"ledgerId":1}}', '{"op":"CONSISTENCY_PROOF","viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"ppSeqNo":6,"seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","ledgerId":1}']}
2017-08-28 15:28:12,734 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,735 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'msg_type': 'LEDGER_STATUS'}, 'Node2')
2017-08-28 15:28:12,735 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,743 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,744 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 2}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}, 'msg_type': 'LEDGER_STATUS'}, 'Node2')
2017-08-28 15:28:12,744 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 2}, 'msg': {'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}}
2017-08-28 15:28:12,752 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,752 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'msg_type': 'LEDGER_STATUS'}, 'Node2')
2017-08-28 15:28:12,752 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,753 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,753 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'params': {'ledgerId': 1}, 'op': 'MESSAGE_RESPONSE', 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}, 'msg_type': 'LEDGER_STATUS'}, 'Node2')
2017-08-28 15:28:12,755 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox MESSAGE_RESPONSE{'msg_type': 'LEDGER_STATUS', 'params': {'ledgerId': 1}, 'msg': {'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}}
2017-08-28 15:28:12,760 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,763 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node2')
2017-08-28 15:28:12,775 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,787 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node4
2017-08-28 15:28:12,788 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,788 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,797 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node4
2017-08-28 15:28:12,797 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,797 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,798 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node4
2017-08-28 15:28:12,800 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,820 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,820 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node4']
2017-08-28 15:28:12,821 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node4
2017-08-28 15:28:12,821 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,822 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,823 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node4']
2017-08-28 15:28:12,832 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node4
2017-08-28 15:28:12,839 | DEBUG    | ledger_manager.py    ( 706) | canStartCatchUpProcess | Node6 cannot start catchup since received only 1 consistency proofs but need at least 2
2017-08-28 15:28:12,850 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node1
2017-08-28 15:28:12,850 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,851 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,852 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node1
2017-08-28 15:28:12,854 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,854 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,855 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node1
2017-08-28 15:28:12,855 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,856 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,880 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node1']
2017-08-28 15:28:12,880 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node1
2017-08-28 15:28:12,881 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,881 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,884 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node1']
2017-08-28 15:28:12,885 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node1
2017-08-28 15:28:12,904 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action partial(request_CPs_if_needed) with id 3 to run in 25 seconds
2017-08-28 15:28:12,905 | DEBUG    | ledger_manager.py    ( 688) | canStartCatchUpProcess | Node6 deciding on the basis of CPs {'Node1': CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}, 'Node4': CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}} and f 1
2017-08-28 15:28:12,908 | DEBUG    | node.py              (1461) | preLedgerCatchUp | Node6 going to process any ordered requests before starting catchup.
2017-08-28 15:28:12,911 | DEBUG    | replica.py           (2199) | _remove_ordered_from_queue | Node6:0 going to remove 0 Ordered messages from outbox
2017-08-28 15:28:12,912 | DEBUG    | node.py              (1869) | force_process_ordered | Node6 processed 0 Ordered batches for instance 0 before starting catch up
2017-08-28 15:28:12,913 | DEBUG    | replica.py           (2199) | _remove_ordered_from_queue | Node6:1 going to remove 0 Ordered messages from outbox
2017-08-28 15:28:12,927 | DEBUG    | node.py              (1869) | force_process_ordered | Node6 processed 0 Ordered batches for instance 1 before starting catch up
2017-08-28 15:28:12,931 | DEBUG    | node.py              (2427) | processStashedOrderedReqs | Node6 processed 0 stashed ordered requests
2017-08-28 15:28:12,931 | DEBUG    | monitor.py           ( 192) | reset | Node6's Monitor being reset
2017-08-28 15:28:12,932 | INFO     | node.py              (1471) | preLedgerCatchUp | Node6 reverted 0 batches before starting catch up for ledger 1
2017-08-28 15:28:12,932 | DEBUG    | ledger_manager.py    ( 788) | startCatchUpProcess | Node6 started catching up with consistency proof CONSISTENCY_PROOF{'hashes': ('jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'), 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:12,934 | DEBUG    | node.py              (2566) | send | Node6 sending message CATCHUP_REQ{'catchupTill': 15, 'seqNoStart': 10, 'ledgerId': 1, 'seqNoEnd': 11} to 1 recipients: ['Node1']
2017-08-28 15:28:12,937 | DEBUG    | node.py              (2566) | send | Node6 sending message CATCHUP_REQ{'catchupTill': 15, 'seqNoStart': 12, 'ledgerId': 1, 'seqNoEnd': 13} to 1 recipients: ['Node2']
2017-08-28 15:28:12,938 | DEBUG    | node.py              (2566) | send | Node6 sending message CATCHUP_REQ{'catchupTill': 15, 'seqNoStart': 14, 'ledgerId': 1, 'seqNoEnd': 15} to 1 recipients: ['Node3']
2017-08-28 15:28:12,938 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action partial(request_txns_if_needed) with id 4 to run in 15.600000000000001 seconds
2017-08-28 15:28:12,939 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node3
2017-08-28 15:28:12,942 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,942 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,943 | DEBUG    | ledger_manager.py    ( 312) | processLedgerStatus | Node6 found out from {'Node5', 'Node4', 'Node3', 'Node1'} that its ledger of type 2 is latest
2017-08-28 15:28:12,944 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node3
2017-08-28 15:28:12,968 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,973 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:12,973 | DEBUG    | ledger_manager.py    ( 312) | processLedgerStatus | Node6 found out from {'Node5', 'Node4', 'Node3', 'Node1'} that its ledger of type 2 is latest
2017-08-28 15:28:12,974 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node3
2017-08-28 15:28:12,976 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,990 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,991 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node3
2017-08-28 15:28:12,995 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,995 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:12,996 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node3
2017-08-28 15:28:12,997 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,015 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node2
2017-08-28 15:28:13,019 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:13,019 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:13,020 | DEBUG    | ledger_manager.py    ( 312) | processLedgerStatus | Node6 found out from {'Node5', 'Node4', 'Node3', 'Node1', 'Node2'} that its ledger of type 2 is latest
2017-08-28 15:28:13,020 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node2
2017-08-28 15:28:13,024 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:13,039 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:13,039 | DEBUG    | ledger_manager.py    ( 312) | processLedgerStatus | Node6 found out from {'Node5', 'Node4', 'Node3', 'Node1', 'Node2'} that its ledger of type 2 is latest
2017-08-28 15:28:13,040 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node2
2017-08-28 15:28:13,041 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:13,042 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:13,057 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 0, 'ppSeqNo': 6, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node2
2017-08-28 15:28:13,057 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:13,058 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 9 with 15
2017-08-28 15:28:13,058 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node2
2017-08-28 15:28:13,075 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,081 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node4 into one transmission
2017-08-28 15:28:13,081 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:13,082 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node4: b'{"signature":null,"messages":["{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:13,083 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node4
2017-08-28 15:28:13,084 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 3 msgs to Node1 into one transmission
2017-08-28 15:28:13,084 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}', b'{"catchupTill":15,"seqNoStart":10,"ledgerId":1,"op":"CATCHUP_REQ","seqNoEnd":11}'])
2017-08-28 15:28:13,084 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node1: b'{"signature":null,"messages":["{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"catchupTill\\":15,\\"seqNoStart\\":10,\\"ledgerId\\":1,\\"op\\":\\"CATCHUP_REQ\\",\\"seqNoEnd\\":11}"],"op":"BATCH"}'
2017-08-28 15:28:13,109 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"catchupTill\\":15,\\"seqNoStart\\":10,\\"ledgerId\\":1,\\"op\\":\\"CATCHUP_REQ\\",\\"seqNoEnd\\":11}"],"op":"BATCH"}' to Node1
2017-08-28 15:28:13,110 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"catchupTill":15,"seqNoStart":14,"ledgerId":1,"op":"CATCHUP_REQ","seqNoEnd":15}' to Node3
2017-08-28 15:28:13,117 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"catchupTill":15,"seqNoStart":14,"ledgerId":1,"op":"CATCHUP_REQ","seqNoEnd":15}' to Node3
2017-08-28 15:28:13,118 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"catchupTill":15,"seqNoStart":12,"ledgerId":1,"op":"CATCHUP_REQ","seqNoEnd":13}' to Node2
2017-08-28 15:28:13,118 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"catchupTill":15,"seqNoStart":12,"ledgerId":1,"op":"CATCHUP_REQ","seqNoEnd":13}' to Node2
2017-08-28 15:28:13,119 | DEBUG    | keep_in_touch.py     (  68) | conns | Node6's connections changed from set() to {'Node1', 'Node3', 'Node2', 'Node4'}
2017-08-28 15:28:13,133 | INFO     | keep_in_touch.py     (  98) | _connsChanged | CONNECTION: Node6 now connected to Node1
2017-08-28 15:28:13,135 | INFO     | keep_in_touch.py     (  98) | _connsChanged | CONNECTION: Node6 now connected to Node3
2017-08-28 15:28:13,135 | INFO     | keep_in_touch.py     (  98) | _connsChanged | CONNECTION: Node6 now connected to Node2
2017-08-28 15:28:13,139 | INFO     | keep_in_touch.py     (  98) | _connsChanged | CONNECTION: Node6 now connected to Node4
2017-08-28 15:28:13,140 | DEBUG    | motor.py             (  34) | set_status | Node6 changing status from starting to started_hungry
2017-08-28 15:28:13,140 | DEBUG    | node.py              ( 923) | checkInstances | Node6 choosing to start election on the basis of count 5 and nodes {'Node1', 'Node3', 'Node2', 'Node4'}
2017-08-28 15:28:13,147 | DEBUG    | primary_selector.py  (  77) | get_msgs_for_lagged_nodes | Node6 has no ViewChangeDone message to send for view 2
2017-08-28 15:28:13,147 | DEBUG    | node.py              ( 884) | send_current_state_to_lagging_node | Node6 sending current state CURRENT_STATE{'viewNo': 2, 'primary': []} to lagged node Node1
2017-08-28 15:28:13,148 | DEBUG    | node.py              (2566) | send | Node6 sending message CURRENT_STATE{'viewNo': 2, 'primary': []} to 1 recipients: ['Node1']
2017-08-28 15:28:13,148 | DEBUG    | primary_selector.py  (  77) | get_msgs_for_lagged_nodes | Node6 has no ViewChangeDone message to send for view 2
2017-08-28 15:28:13,157 | DEBUG    | node.py              ( 884) | send_current_state_to_lagging_node | Node6 sending current state CURRENT_STATE{'viewNo': 2, 'primary': []} to lagged node Node3
2017-08-28 15:28:13,157 | DEBUG    | node.py              (2566) | send | Node6 sending message CURRENT_STATE{'viewNo': 2, 'primary': []} to 1 recipients: ['Node3']
2017-08-28 15:28:13,158 | DEBUG    | primary_selector.py  (  77) | get_msgs_for_lagged_nodes | Node6 has no ViewChangeDone message to send for view 2
2017-08-28 15:28:13,163 | DEBUG    | node.py              ( 884) | send_current_state_to_lagging_node | Node6 sending current state CURRENT_STATE{'viewNo': 2, 'primary': []} to lagged node Node2
2017-08-28 15:28:13,164 | DEBUG    | node.py              (2566) | send | Node6 sending message CURRENT_STATE{'viewNo': 2, 'primary': []} to 1 recipients: ['Node2']
2017-08-28 15:28:13,164 | DEBUG    | primary_selector.py  (  77) | get_msgs_for_lagged_nodes | Node6 has no ViewChangeDone message to send for view 2
2017-08-28 15:28:13,171 | DEBUG    | node.py              ( 884) | send_current_state_to_lagging_node | Node6 sending current state CURRENT_STATE{'viewNo': 2, 'primary': []} to lagged node Node4
2017-08-28 15:28:13,171 | DEBUG    | node.py              (2566) | send | Node6 sending message CURRENT_STATE{'viewNo': 2, 'primary': []} to 1 recipients: ['Node4']
2017-08-28 15:28:13,180 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} to 1 recipients: ['Node1']
2017-08-28 15:28:13,180 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node1']
2017-08-28 15:28:13,188 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} to 1 recipients: ['Node3']
2017-08-28 15:28:13,189 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node3']
2017-08-28 15:28:13,189 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} to 1 recipients: ['Node2']
2017-08-28 15:28:13,200 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node2']
2017-08-28 15:28:13,204 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} to 1 recipients: ['Node4']
2017-08-28 15:28:13,205 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'txnSeqNo': 9} to 1 recipients: ['Node4']
2017-08-28 15:28:13,211 | DEBUG    | looper.py            ( 212) | runOnceNicely | it took 0.723 seconds to run once nicely
2017-08-28 15:28:13,220 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 4 messages through listener
2017-08-28 15:28:13,231 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"seqNoEnd":15,"ledgerId":1,"ppSeqNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}', '{"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"seqNoEnd":15,"ledgerId":1,"ppSeqNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}']}
2017-08-28 15:28:13,238 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"seqNoEnd":15,"ledgerId":1,"ppSeqNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}', '{"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"seqNoEnd":15,"ledgerId":1,"ppSeqNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:28:13,251 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"seqNoEnd":15,"ledgerId":1,"ppSeqNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}', '{"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"seqNoEnd":15,"ledgerId":1,"ppSeqNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","seqNoStart":9,"viewNo":0,"op":"CONSISTENCY_PROOF","newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"}']}
2017-08-28 15:28:13,255 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,256 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node5')
2017-08-28 15:28:13,259 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,279 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,281 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node5')
2017-08-28 15:28:13,282 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,289 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: CATCHUP_REP{'txns': {'15': {'signature': '4bFhS6WMQ2uMTKhy5Np98naJNhtiQXFWbizbUoDKML7QK8HZs9RTSnu7uEVCSovQfEzNtT1QCbuu3BoQgY1hRW2x', 'reqId': 1503933484172953, 'txnTime': 1503933484, 'dest': '5qhSKMkpqui6taGLTecwqf', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~UWK9xwiKsqMQSBmymYpR5v'}, '14': {'signature': '5QCYSHuA6UfU5wYZ86osbmhHADHiXfD5mUqjR7syPBnbqhJzJuzrzHtL3L3VeeoiTR4ybMJGM6RYnPG2JKjtSpmW', 'reqId': 1503933478716086, 'txnTime': 1503933479, 'role': '101', 'dest': 'Rhx2qwaeiGyhU9vn4ynHSS', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RTJS62PRHurqHJnbnT1Hgi'}}, 'ledgerId': 1, 'consProof': []}
2017-08-28 15:28:13,292 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'txns': {'15': {'signature': '4bFhS6WMQ2uMTKhy5Np98naJNhtiQXFWbizbUoDKML7QK8HZs9RTSnu7uEVCSovQfEzNtT1QCbuu3BoQgY1hRW2x', 'reqId': 1503933484172953, 'txnTime': 1503933484, 'dest': '5qhSKMkpqui6taGLTecwqf', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~UWK9xwiKsqMQSBmymYpR5v'}, '14': {'signature': '5QCYSHuA6UfU5wYZ86osbmhHADHiXfD5mUqjR7syPBnbqhJzJuzrzHtL3L3VeeoiTR4ybMJGM6RYnPG2JKjtSpmW', 'reqId': 1503933478716086, 'txnTime': 1503933479, 'role': '101', 'dest': 'Rhx2qwaeiGyhU9vn4ynHSS', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RTJS62PRHurqHJnbnT1Hgi'}}, 'ledgerId': 1, 'op': 'CATCHUP_REP', 'consProof': []}, 'Node3')
2017-08-28 15:28:13,306 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CATCHUP_REP{'txns': {'15': {'signature': '4bFhS6WMQ2uMTKhy5Np98naJNhtiQXFWbizbUoDKML7QK8HZs9RTSnu7uEVCSovQfEzNtT1QCbuu3BoQgY1hRW2x', 'reqId': 1503933484172953, 'txnTime': 1503933484, 'dest': '5qhSKMkpqui6taGLTecwqf', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~UWK9xwiKsqMQSBmymYpR5v'}, '14': {'signature': '5QCYSHuA6UfU5wYZ86osbmhHADHiXfD5mUqjR7syPBnbqhJzJuzrzHtL3L3VeeoiTR4ybMJGM6RYnPG2JKjtSpmW', 'reqId': 1503933478716086, 'txnTime': 1503933479, 'role': '101', 'dest': 'Rhx2qwaeiGyhU9vn4ynHSS', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RTJS62PRHurqHJnbnT1Hgi'}}, 'ledgerId': 1, 'consProof': []}
2017-08-28 15:28:13,308 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: CATCHUP_REP{'txns': {'13': {'signature': '4nubSjtZvYdVxRdaGWGMLKruPpWZMJ9JTJK94QstXMKMt3Fe13qmUNzzKacSD1VZVk7L89caYvYjPdwfAYXsf1ku', 'reqId': 1503933471684615, 'type': '1', 'role': '100', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': '6XiqM9DwYDgFnmyQQBFr72', 'txnTime': 1503933471, 'verkey': '~YHDYsnaN59TzUsLqvPYdRr'}, '12': {'signature': 'dXFex68iuW2sNXJPJaAq1zxyYfxk7s4ss89ydcmfe2NS7L5RndxCJhc6VGvRWH6cEArg5SowGE4JuPLCD87fgQi', 'reqId': 1503933466149129, 'type': '1', 'role': '2', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': 'FBQ5W7LxRp8bLeVNEafjpd', 'txnTime': 1503933466, 'verkey': '~2a4KepmnVVAPGo5gNA3P3p'}}, 'ledgerId': 1, 'consProof': ['HrVYnCQ4sRNzhQigrJsjEPBmrAWsPa9H1xsgfef3LEMw', '6eiB4VZiNXZ35Z1SPBMg8HQ9QuBNLozoWqhLHcGgYAE4', 'AbFuZ9GiDG7iwYdCm8cC6UcKVLH4i1C3RH4jXb8msWs9', '4DUBvjuKvdvzPKForxX5ArBcFEbXEKk9NrMQy26SkxLK', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}
2017-08-28 15:28:13,315 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'txns': {'13': {'signature': '4nubSjtZvYdVxRdaGWGMLKruPpWZMJ9JTJK94QstXMKMt3Fe13qmUNzzKacSD1VZVk7L89caYvYjPdwfAYXsf1ku', 'reqId': 1503933471684615, 'type': '1', 'role': '100', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': '6XiqM9DwYDgFnmyQQBFr72', 'txnTime': 1503933471, 'verkey': '~YHDYsnaN59TzUsLqvPYdRr'}, '12': {'signature': 'dXFex68iuW2sNXJPJaAq1zxyYfxk7s4ss89ydcmfe2NS7L5RndxCJhc6VGvRWH6cEArg5SowGE4JuPLCD87fgQi', 'reqId': 1503933466149129, 'type': '1', 'role': '2', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': 'FBQ5W7LxRp8bLeVNEafjpd', 'txnTime': 1503933466, 'verkey': '~2a4KepmnVVAPGo5gNA3P3p'}}, 'ledgerId': 1, 'op': 'CATCHUP_REP', 'consProof': ['HrVYnCQ4sRNzhQigrJsjEPBmrAWsPa9H1xsgfef3LEMw', '6eiB4VZiNXZ35Z1SPBMg8HQ9QuBNLozoWqhLHcGgYAE4', 'AbFuZ9GiDG7iwYdCm8cC6UcKVLH4i1C3RH4jXb8msWs9', '4DUBvjuKvdvzPKForxX5ArBcFEbXEKk9NrMQy26SkxLK', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}, 'Node2')
2017-08-28 15:28:13,316 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CATCHUP_REP{'txns': {'13': {'signature': '4nubSjtZvYdVxRdaGWGMLKruPpWZMJ9JTJK94QstXMKMt3Fe13qmUNzzKacSD1VZVk7L89caYvYjPdwfAYXsf1ku', 'reqId': 1503933471684615, 'type': '1', 'role': '100', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': '6XiqM9DwYDgFnmyQQBFr72', 'txnTime': 1503933471, 'verkey': '~YHDYsnaN59TzUsLqvPYdRr'}, '12': {'signature': 'dXFex68iuW2sNXJPJaAq1zxyYfxk7s4ss89ydcmfe2NS7L5RndxCJhc6VGvRWH6cEArg5SowGE4JuPLCD87fgQi', 'reqId': 1503933466149129, 'type': '1', 'role': '2', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': 'FBQ5W7LxRp8bLeVNEafjpd', 'txnTime': 1503933466, 'verkey': '~2a4KepmnVVAPGo5gNA3P3p'}}, 'ledgerId': 1, 'consProof': ['HrVYnCQ4sRNzhQigrJsjEPBmrAWsPa9H1xsgfef3LEMw', '6eiB4VZiNXZ35Z1SPBMg8HQ9QuBNLozoWqhLHcGgYAE4', 'AbFuZ9GiDG7iwYdCm8cC6UcKVLH4i1C3RH4jXb8msWs9', '4DUBvjuKvdvzPKForxX5ArBcFEbXEKk9NrMQy26SkxLK', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}
2017-08-28 15:28:13,320 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: BATCH{'signature': None, 'messages': ['{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}', '{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}']}
2017-08-28 15:28:13,321 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}', '{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}'], 'op': 'BATCH'}, 'Node4')
2017-08-28 15:28:13,340 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}', '{"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","op":"CONSISTENCY_PROOF","seqNoStart":9,"ledgerId":1,"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","seqNoEnd":15,"hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"viewNo":0,"ppSeqNo":6}']}
2017-08-28 15:28:13,342 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,346 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node4')
2017-08-28 15:28:13,364 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,365 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,370 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node4')
2017-08-28 15:28:13,373 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,374 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node5
2017-08-28 15:28:13,374 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,375 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 0, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node5
2017-08-28 15:28:13,376 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,376 | DEBUG    | ledger_manager.py    ( 455) | processCatchupRep | Node6 received catchup reply from Node3: CATCHUP_REP{'txns': {'15': {'signature': '4bFhS6WMQ2uMTKhy5Np98naJNhtiQXFWbizbUoDKML7QK8HZs9RTSnu7uEVCSovQfEzNtT1QCbuu3BoQgY1hRW2x', 'reqId': 1503933484172953, 'txnTime': 1503933484, 'dest': '5qhSKMkpqui6taGLTecwqf', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~UWK9xwiKsqMQSBmymYpR5v'}, '14': {'signature': '5QCYSHuA6UfU5wYZ86osbmhHADHiXfD5mUqjR7syPBnbqhJzJuzrzHtL3L3VeeoiTR4ybMJGM6RYnPG2JKjtSpmW', 'reqId': 1503933478716086, 'txnTime': 1503933479, 'role': '101', 'dest': 'Rhx2qwaeiGyhU9vn4ynHSS', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RTJS62PRHurqHJnbnT1Hgi'}}, 'ledgerId': 1, 'consProof': []}
2017-08-28 15:28:13,377 | DEBUG    | ledger_manager.py    ( 460) | processCatchupRep | Node6 found 2 transactions in the catchup from Node3
2017-08-28 15:28:13,377 | DEBUG    | ledger_manager.py    ( 477) | processCatchupRep | Node6 merging all received catchups
2017-08-28 15:28:13,400 | DEBUG    | ledger_manager.py    ( 485) | processCatchupRep | Node6 merged catchups, there are 2 of them now, from 14 to 15
2017-08-28 15:28:13,401 | DEBUG    | ledger_manager.py    ( 493) | processCatchupRep | Node6 processed 0 catchup replies with sequence numbers []
2017-08-28 15:28:13,401 | DEBUG    | ledger_manager.py    ( 455) | processCatchupRep | Node6 received catchup reply from Node2: CATCHUP_REP{'txns': {'13': {'signature': '4nubSjtZvYdVxRdaGWGMLKruPpWZMJ9JTJK94QstXMKMt3Fe13qmUNzzKacSD1VZVk7L89caYvYjPdwfAYXsf1ku', 'reqId': 1503933471684615, 'type': '1', 'role': '100', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': '6XiqM9DwYDgFnmyQQBFr72', 'txnTime': 1503933471, 'verkey': '~YHDYsnaN59TzUsLqvPYdRr'}, '12': {'signature': 'dXFex68iuW2sNXJPJaAq1zxyYfxk7s4ss89ydcmfe2NS7L5RndxCJhc6VGvRWH6cEArg5SowGE4JuPLCD87fgQi', 'reqId': 1503933466149129, 'type': '1', 'role': '2', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'dest': 'FBQ5W7LxRp8bLeVNEafjpd', 'txnTime': 1503933466, 'verkey': '~2a4KepmnVVAPGo5gNA3P3p'}}, 'ledgerId': 1, 'consProof': ['HrVYnCQ4sRNzhQigrJsjEPBmrAWsPa9H1xsgfef3LEMw', '6eiB4VZiNXZ35Z1SPBMg8HQ9QuBNLozoWqhLHcGgYAE4', 'AbFuZ9GiDG7iwYdCm8cC6UcKVLH4i1C3RH4jXb8msWs9', '4DUBvjuKvdvzPKForxX5ArBcFEbXEKk9NrMQy26SkxLK', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}
2017-08-28 15:28:13,413 | DEBUG    | ledger_manager.py    ( 460) | processCatchupRep | Node6 found 2 transactions in the catchup from Node2
2017-08-28 15:28:13,413 | DEBUG    | ledger_manager.py    ( 477) | processCatchupRep | Node6 merging all received catchups
2017-08-28 15:28:13,422 | DEBUG    | ledger_manager.py    ( 485) | processCatchupRep | Node6 merged catchups, there are 4 of them now, from 12 to 15
2017-08-28 15:28:13,423 | DEBUG    | ledger_manager.py    ( 493) | processCatchupRep | Node6 processed 0 catchup replies with sequence numbers []
2017-08-28 15:28:13,426 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node4
2017-08-28 15:28:13,427 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,429 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node4
2017-08-28 15:28:13,436 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,455 | DEBUG    | primary_selector.py  ( 106) | _processViewChangeDoneMessage | Node6's primary selector started processing of ViewChangeDone msg from Node3 : VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}
2017-08-28 15:28:13,456 | DEBUG    | primary_selector.py  ( 190) | _hasViewChangeQuorum | Node6 needs 1 ViewChangeDone messages
2017-08-28 15:28:13,456 | DEBUG    | primary_selector.py  ( 265) | _startSelection | Node6 cannot start primary selection found failure in primary verification. This can happen due to lack of appropriate ViewChangeDone messages
2017-08-28 15:28:13,458 | DEBUG    | primary_selector.py  ( 106) | _processViewChangeDoneMessage | Node6's primary selector started processing of ViewChangeDone msg from Node2 : VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}
2017-08-28 15:28:13,462 | DEBUG    | primary_selector.py  ( 196) | _hasViewChangeQuorum | Node6 got view change quorum (2 >= 2)
2017-08-28 15:28:13,462 | DEBUG    | primary_selector.py  ( 214) | has_view_change_from_primary | Node6 received ViewChangeDone from primary Node3
2017-08-28 15:28:13,463 | DEBUG    | primary_selector.py  ( 237) | has_sufficient_same_view_change_done_messages | Node6 found acceptable primary Node3 and ledger info ((0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn'))
2017-08-28 15:28:13,482 | DEBUG    | primary_selector.py  ( 270) | _startSelection | Node6 cannot start primary selection since mode is 400
2017-08-28 15:28:13,486 | DEBUG    | primary_selector.py  ( 106) | _processViewChangeDoneMessage | Node6's primary selector started processing of ViewChangeDone msg from Node1 : VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}
2017-08-28 15:28:13,487 | DEBUG    | primary_selector.py  ( 270) | _startSelection | Node6 cannot start primary selection since mode is 400
2017-08-28 15:28:13,488 | DEBUG    | primary_selector.py  ( 106) | _processViewChangeDoneMessage | Node6's primary selector started processing of ViewChangeDone msg from Node4 : VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}
2017-08-28 15:28:13,491 | DEBUG    | primary_selector.py  ( 270) | _startSelection | Node6 cannot start primary selection since mode is 400
2017-08-28 15:28:13,506 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 3 msgs to Node4 into one transmission
2017-08-28 15:28:13,507 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"viewNo":2,"primary":[],"op":"CURRENT_STATE"}', b'{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:13,510 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node4: b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:13,511 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node4
2017-08-28 15:28:13,515 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 3 msgs to Node1 into one transmission
2017-08-28 15:28:13,516 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"viewNo":2,"primary":[],"op":"CURRENT_STATE"}', b'{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:13,516 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node1: b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:13,519 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node1
2017-08-28 15:28:13,534 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 3 msgs to Node3 into one transmission
2017-08-28 15:28:13,534 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"viewNo":2,"primary":[],"op":"CURRENT_STATE"}', b'{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:13,535 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node3: b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:13,543 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node3
2017-08-28 15:28:13,548 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 3 msgs to Node2 into one transmission
2017-08-28 15:28:13,549 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"viewNo":2,"primary":[],"op":"CURRENT_STATE"}', b'{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","txnSeqNo":9,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:13,549 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node2: b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:13,558 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t\\",\\"txnSeqNo\\":9,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node2
2017-08-28 15:28:13,576 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:13,577 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: BATCH{'signature': None, 'messages': ['{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}', '{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}', '{"consProof":["8Bzurgoq64vp4oTqg9QmtZkGLuK4QzErdQJfLftvozvY","5BCQoW4vvq573NaYeKU6uTgZVrdKxzuz7J1QdwN5Njna","A7qtYTZtZNYspt397HsDzan5USLKgejsseghD1sboLRw","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"op":"CATCHUP_REP","txns":{"10":{"signature":"4GMeDHCT3dxL3amjDEFBWCZrAmCS3foqhnpwdsML4CXuHYFNm5yKiRBZn9dqoyLKV8CQpCc7mwAyVZdKsxyUp2zn","txnTime":1503933200,"identifier":"V4SGRU86Z58d6TV7PBUe6f","dest":"V4SGRU86Z58d6TV7PBUe61","type":"1","reqId":1503933200605368},"11":{"signature":"33DEkGqHn1PQVocRBgvEz91rSXKgw8GV68L71GJSn4NGKCnXy2UdP3amXysxv8UAhuxTG5iuAUzKcqtLjfa4CWXe","txnTime":1503933459,"verkey":"~RmCt3RtDvKDfQBSKxo4qvy","identifier":"V4SGRU86Z58d6TV7PBUe6f","dest":"XhYtvJqezMUKfF6KVNaGmT","type":"1","role":"2","reqId":1503933459424488}},"ledgerId":1}']}
2017-08-28 15:28:13,591 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}', '{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}', '{"consProof":["8Bzurgoq64vp4oTqg9QmtZkGLuK4QzErdQJfLftvozvY","5BCQoW4vvq573NaYeKU6uTgZVrdKxzuz7J1QdwN5Njna","A7qtYTZtZNYspt397HsDzan5USLKgejsseghD1sboLRw","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"op":"CATCHUP_REP","txns":{"10":{"signature":"4GMeDHCT3dxL3amjDEFBWCZrAmCS3foqhnpwdsML4CXuHYFNm5yKiRBZn9dqoyLKV8CQpCc7mwAyVZdKsxyUp2zn","txnTime":1503933200,"identifier":"V4SGRU86Z58d6TV7PBUe6f","dest":"V4SGRU86Z58d6TV7PBUe61","type":"1","reqId":1503933200605368},"11":{"signature":"33DEkGqHn1PQVocRBgvEz91rSXKgw8GV68L71GJSn4NGKCnXy2UdP3amXysxv8UAhuxTG5iuAUzKcqtLjfa4CWXe","txnTime":1503933459,"verkey":"~RmCt3RtDvKDfQBSKxo4qvy","identifier":"V4SGRU86Z58d6TV7PBUe6f","dest":"XhYtvJqezMUKfF6KVNaGmT","type":"1","role":"2","reqId":1503933459424488}},"ledgerId":1}'], 'op': 'BATCH'}, 'Node1')
2017-08-28 15:28:13,599 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}', '{"newMerkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"CONSISTENCY_PROOF","seqNoEnd":15,"ledgerId":1,"ppSeqNo":6,"seqNoStart":9,"viewNo":0,"oldMerkleRoot":"42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t","hashes":["jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb","dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC","FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"]}', '{"consProof":["8Bzurgoq64vp4oTqg9QmtZkGLuK4QzErdQJfLftvozvY","5BCQoW4vvq573NaYeKU6uTgZVrdKxzuz7J1QdwN5Njna","A7qtYTZtZNYspt397HsDzan5USLKgejsseghD1sboLRw","GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN","FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz"],"op":"CATCHUP_REP","txns":{"10":{"signature":"4GMeDHCT3dxL3amjDEFBWCZrAmCS3foqhnpwdsML4CXuHYFNm5yKiRBZn9dqoyLKV8CQpCc7mwAyVZdKsxyUp2zn","txnTime":1503933200,"identifier":"V4SGRU86Z58d6TV7PBUe6f","dest":"V4SGRU86Z58d6TV7PBUe61","type":"1","reqId":1503933200605368},"11":{"signature":"33DEkGqHn1PQVocRBgvEz91rSXKgw8GV68L71GJSn4NGKCnXy2UdP3amXysxv8UAhuxTG5iuAUzKcqtLjfa4CWXe","txnTime":1503933459,"verkey":"~RmCt3RtDvKDfQBSKxo4qvy","identifier":"V4SGRU86Z58d6TV7PBUe6f","dest":"XhYtvJqezMUKfF6KVNaGmT","type":"1","role":"2","reqId":1503933459424488}},"ledgerId":1}']}
2017-08-28 15:28:13,604 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,605 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node1')
2017-08-28 15:28:13,611 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,623 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,631 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node1')
2017-08-28 15:28:13,639 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,640 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CATCHUP_REP{'txns': {'11': {'signature': '33DEkGqHn1PQVocRBgvEz91rSXKgw8GV68L71GJSn4NGKCnXy2UdP3amXysxv8UAhuxTG5iuAUzKcqtLjfa4CWXe', 'reqId': 1503933459424488, 'txnTime': 1503933459, 'role': '2', 'dest': 'XhYtvJqezMUKfF6KVNaGmT', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RmCt3RtDvKDfQBSKxo4qvy'}, '10': {'signature': '4GMeDHCT3dxL3amjDEFBWCZrAmCS3foqhnpwdsML4CXuHYFNm5yKiRBZn9dqoyLKV8CQpCc7mwAyVZdKsxyUp2zn', 'reqId': 1503933200605368, 'txnTime': 1503933200, 'dest': 'V4SGRU86Z58d6TV7PBUe61', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1'}}, 'ledgerId': 1, 'consProof': ['8Bzurgoq64vp4oTqg9QmtZkGLuK4QzErdQJfLftvozvY', '5BCQoW4vvq573NaYeKU6uTgZVrdKxzuz7J1QdwN5Njna', 'A7qtYTZtZNYspt397HsDzan5USLKgejsseghD1sboLRw', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}
2017-08-28 15:28:13,649 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'txns': {'11': {'signature': '33DEkGqHn1PQVocRBgvEz91rSXKgw8GV68L71GJSn4NGKCnXy2UdP3amXysxv8UAhuxTG5iuAUzKcqtLjfa4CWXe', 'reqId': 1503933459424488, 'txnTime': 1503933459, 'role': '2', 'dest': 'XhYtvJqezMUKfF6KVNaGmT', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RmCt3RtDvKDfQBSKxo4qvy'}, '10': {'signature': '4GMeDHCT3dxL3amjDEFBWCZrAmCS3foqhnpwdsML4CXuHYFNm5yKiRBZn9dqoyLKV8CQpCc7mwAyVZdKsxyUp2zn', 'reqId': 1503933200605368, 'txnTime': 1503933200, 'dest': 'V4SGRU86Z58d6TV7PBUe61', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1'}}, 'ledgerId': 1, 'op': 'CATCHUP_REP', 'consProof': ['8Bzurgoq64vp4oTqg9QmtZkGLuK4QzErdQJfLftvozvY', '5BCQoW4vvq573NaYeKU6uTgZVrdKxzuz7J1QdwN5Njna', 'A7qtYTZtZNYspt397HsDzan5USLKgejsseghD1sboLRw', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}, 'Node1')
2017-08-28 15:28:13,650 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CATCHUP_REP{'txns': {'11': {'signature': '33DEkGqHn1PQVocRBgvEz91rSXKgw8GV68L71GJSn4NGKCnXy2UdP3amXysxv8UAhuxTG5iuAUzKcqtLjfa4CWXe', 'reqId': 1503933459424488, 'txnTime': 1503933459, 'role': '2', 'dest': 'XhYtvJqezMUKfF6KVNaGmT', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RmCt3RtDvKDfQBSKxo4qvy'}, '10': {'signature': '4GMeDHCT3dxL3amjDEFBWCZrAmCS3foqhnpwdsML4CXuHYFNm5yKiRBZn9dqoyLKV8CQpCc7mwAyVZdKsxyUp2zn', 'reqId': 1503933200605368, 'txnTime': 1503933200, 'dest': 'V4SGRU86Z58d6TV7PBUe61', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1'}}, 'ledgerId': 1, 'consProof': ['8Bzurgoq64vp4oTqg9QmtZkGLuK4QzErdQJfLftvozvY', '5BCQoW4vvq573NaYeKU6uTgZVrdKxzuz7J1QdwN5Njna', 'A7qtYTZtZNYspt397HsDzan5USLKgejsseghD1sboLRw', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}
2017-08-28 15:28:13,660 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node1
2017-08-28 15:28:13,675 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,676 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node1
2017-08-28 15:28:13,682 | DEBUG    | ledger_manager.py    ( 356) | canProcessConsistencyProof | Node6 cannot process consistency proof since ledger state is 2
2017-08-28 15:28:13,683 | DEBUG    | ledger_manager.py    ( 455) | processCatchupRep | Node6 received catchup reply from Node1: CATCHUP_REP{'txns': {'11': {'signature': '33DEkGqHn1PQVocRBgvEz91rSXKgw8GV68L71GJSn4NGKCnXy2UdP3amXysxv8UAhuxTG5iuAUzKcqtLjfa4CWXe', 'reqId': 1503933459424488, 'txnTime': 1503933459, 'role': '2', 'dest': 'XhYtvJqezMUKfF6KVNaGmT', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1', 'verkey': '~RmCt3RtDvKDfQBSKxo4qvy'}, '10': {'signature': '4GMeDHCT3dxL3amjDEFBWCZrAmCS3foqhnpwdsML4CXuHYFNm5yKiRBZn9dqoyLKV8CQpCc7mwAyVZdKsxyUp2zn', 'reqId': 1503933200605368, 'txnTime': 1503933200, 'dest': 'V4SGRU86Z58d6TV7PBUe61', 'identifier': 'V4SGRU86Z58d6TV7PBUe6f', 'type': '1'}}, 'ledgerId': 1, 'consProof': ['8Bzurgoq64vp4oTqg9QmtZkGLuK4QzErdQJfLftvozvY', '5BCQoW4vvq573NaYeKU6uTgZVrdKxzuz7J1QdwN5Njna', 'A7qtYTZtZNYspt397HsDzan5USLKgejsseghD1sboLRw', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz']}
2017-08-28 15:28:13,694 | DEBUG    | ledger_manager.py    ( 460) | processCatchupRep | Node6 found 2 transactions in the catchup from Node1
2017-08-28 15:28:13,694 | DEBUG    | ledger_manager.py    ( 477) | processCatchupRep | Node6 merging all received catchups
2017-08-28 15:28:13,694 | DEBUG    | ledger_manager.py    ( 485) | processCatchupRep | Node6 merged catchups, there are 6 of them now, from 10 to 15
2017-08-28 15:28:13,707 | DEBUG    | ledger_manager.py    ( 595) | hasValidCatchupReplies | Node6 verifying proof for 11, 15, b"\x1e\t\xbc02\xb7\xf2'\xd8Z+\x97.\xf9u\xa0D\xb1o\x8dS\xc1\x9fo\xb3\x92(Y\x89\x87\x83\xe6", b'\xcc\xe8+\x8d\xa3R\xe0\x8fX\xdf\xa2\x91\x9ea\x9b\x10c\x12\xa8]+\xdbD~\xa7U\xf6]>\xce_F', [b'j\xd3\x18\xdc\x8f\x1e\xb1\xc7\xc7\x14\xd1\xc1\xf8/i\xd3\xad\xb2\x94)8`\xff\x95\xc2^Ct\x15{\x10\x85', b'>\x0bu!\xe0\xdaw[\x1b5a\x8a*y\x9fj\xe4R\xa0f\xe2\x8eN4C\x9d\xca\xa0_\x1a\xf8c', b'\x87y\xf2\x1e|:\xe7\x17\xf3\xd3\xc7;o8\xce\x05u\x01Qn\n\x95\xc4\xee\xd8\xc00\xf6\x9b\xfb\xc7\xe2', b"\xe4\x84|W\xe7\xa2\x88\xb8p\r\xe7\xf6/3\xfc\xc4\x1a\x11\xea\xce\x90dL\xc2m\xb9\xe4!\x17F\xc5'", b'\xd7\r\x82\x86t\xa38`\xaa\xbb0\xa9J\xccM\xd1\xbci\xb8_~^\x18\x1a\x92\x934\xcd\xcaB\x1f\xb9']
2017-08-28 15:28:13,711 | DEBUG    | ledger_manager.py    ( 595) | hasValidCatchupReplies | Node6 verifying proof for 13, 15, b'\xac\x8c\xc5\xa0\x8cw\x03\x88.P\x13]\xabb\xe9\x9d\x14X\x1196\xd27\x97DY\xdcZk\x8f\x08E', b'\xcc\xe8+\x8d\xa3R\xe0\x8fX\xdf\xa2\x91\x9ea\x9b\x10c\x12\xa8]+\xdbD~\xa7U\xf6]>\xce_F', [b'\xfah\xba\xcd\xa2P\xe0z\x03\xc7\x93\xc3B-_1<%\xa9\xfc`\xfb\x8e\rv\xc4,\x10\\3=\xaa', b'S\xf3z\xc3\x02\x9b\xfb\x14\xb8J\xf8\xbaHB\xebc5R\x98\xd9i\xe6\xe1G\xaa\x86u\x10\x04\t\xf3\xa1', b'\x8e\x7f\xc8\xc3\x92\\v\t\xff\xff\xc6\xf1\xd2O/\xedN][\xd2R\xff\xb2\x81\x00\xb5\x04N\xe2w\x01\xe0', b'/\xc4\xc5\xc2^\xc3\x85V\xb9\x83\x1a\x90\xcc\x18\xf1\xfa\xeeO\xcf\xbbD\x95 \xfa\xe8\xdb\xe6x\xa2\x8fK\xc4', b'\xd7\r\x82\x86t\xa38`\xaa\xbb0\xa9J\xccM\xd1\xbci\xb8_~^\x18\x1a\x92\x934\xcd\xcaB\x1f\xb9']
2017-08-28 15:28:13,728 | DEBUG    | ledger_manager.py    ( 595) | hasValidCatchupReplies | Node6 verifying proof for 15, 15, b'\xcc\xe8+\x8d\xa3R\xe0\x8fX\xdf\xa2\x91\x9ea\x9b\x10c\x12\xa8]+\xdbD~\xa7U\xf6]>\xce_F', b'\xcc\xe8+\x8d\xa3R\xe0\x8fX\xdf\xa2\x91\x9ea\x9b\x10c\x12\xa8]+\xdbD~\xa7U\xf6]>\xce_F', []
2017-08-28 15:28:13,754 | DEBUG    | ledger_manager.py    ( 493) | processCatchupRep | Node6 processed 6 catchup replies with sequence numbers [10, 11, 12, 13, 14, 15]
2017-08-28 15:28:13,771 | INFO     | ledger_manager.py    ( 848) | catchupCompleted | CATCH-UP: Node6 completed catching up ledger 1, caught up 6 in total
2017-08-28 15:28:13,775 | DEBUG    | node.py              (1591) | num_txns_caught_up_in_last_catchup | Node6 caught up to 9 txns in the last catchup
2017-08-28 15:28:13,776 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:0 set last ordered as (2, 6)
2017-08-28 15:28:13,776 | DEBUG    | replica.py           (2174) | _remove_till_caught_up_3pc | Node6:0 going to remove messages for 0 3PC keys
2017-08-28 15:28:13,778 | DEBUG    | replica.py           (2199) | _remove_ordered_from_queue | Node6:0 going to remove 0 Ordered messages from outbox
2017-08-28 15:28:13,779 | INFO     | node.py              (1507) | allLedgersCaughtUp | CATCH-UP: Node6 caught up till (2, 6)
2017-08-28 15:28:13,779 | DEBUG    | node.py              (2427) | processStashedOrderedReqs | Node6 processed 0 stashed ordered requests
2017-08-28 15:28:13,780 | DEBUG    | monitor.py           ( 192) | reset | Node6's Monitor being reset
2017-08-28 15:28:13,784 | DEBUG    | primary_selector.py  ( 196) | _hasViewChangeQuorum | Node6 got view change quorum (4 >= 2)
2017-08-28 15:28:13,798 | DEBUG    | node.py              (1531) | is_catchup_needed | Node6 is caught up for the current view 2
2017-08-28 15:28:13,798 | INFO     | node.py              (1519) | allLedgersCaughtUp | CATCH-UP: Node6 does not need any more catchups
2017-08-28 15:28:13,805 | DEBUG    | primary_decider.py   ( 131) | send | Node6's elector sending VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [(0, 7, '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'), (1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')], 'name': 'Node3'}
2017-08-28 15:28:13,805 | DEBUG    | primary_selector.py  ( 280) | _startSelection | Node6 starting selection
2017-08-28 15:28:13,808 | DISPLAY  | primary_selector.py  ( 290) | _startSelection | PRIMARY SELECTION: Node6:0 selected primary Node3:0 for instance 0 (view 2)
2017-08-28 15:28:13,809 | INFO     | node.py              ( 503) | start_participating | Node6 started participating
2017-08-28 15:28:13,809 | DEBUG    | replica.py           ( 406) | primaryName | Node6:0 setting primaryName for view no 2 to: Node3:0
2017-08-28 15:28:13,823 | DEBUG    | replica.py           (1678) | _gc | Node6:0 cleaning up till (2, 6)
2017-08-28 15:28:13,824 | DEBUG    | replica.py           (1693) | _gc | Node6:0 found 0 3-phase keys to clean
2017-08-28 15:28:13,824 | DEBUG    | replica.py           (1695) | _gc | Node6:0 found 0 request keys to clean
2017-08-28 15:28:13,824 | DEBUG    | replica.py           ( 311) | h | Node6:0 set watermarks as 0 300
2017-08-28 15:28:13,836 | DISPLAY  | primary_selector.py  ( 314) | _startSelection | VIEW CHANGE: Node6:0 declares view change 2 as completed for instance 0, new primary is Node3:0, ledger info is [(0, 7, '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'), (1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')]
2017-08-28 15:28:13,836 | DISPLAY  | primary_selector.py  ( 290) | _startSelection | PRIMARY SELECTION: Node6:1 selected primary Node4:1 for instance 1 (view 2)
2017-08-28 15:28:13,842 | DEBUG    | replica.py           ( 406) | primaryName | Node6:1 setting primaryName for view no 2 to: Node4:1
2017-08-28 15:28:13,843 | DEBUG    | replica.py           (1678) | _gc | Node6:1 cleaning up till (0, 0)
2017-08-28 15:28:13,847 | DEBUG    | replica.py           (1693) | _gc | Node6:1 found 0 3-phase keys to clean
2017-08-28 15:28:13,847 | DEBUG    | replica.py           (1695) | _gc | Node6:1 found 0 request keys to clean
2017-08-28 15:28:13,848 | DEBUG    | replica.py           ( 311) | h | Node6:1 set watermarks as 0 300
2017-08-28 15:28:13,857 | DEBUG    | replica.py           ( 512) | _setup_for_non_master | Node6:1 Setting last ordered for non-master as (0, 0)
2017-08-28 15:28:13,858 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 0)
2017-08-28 15:28:13,858 | DISPLAY  | primary_selector.py  ( 314) | _startSelection | VIEW CHANGE: Node6:1 declares view change 2 as completed for instance 1, new primary is Node4:1, ledger info is [(0, 7, '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'), (1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')]
2017-08-28 15:28:13,866 | DEBUG    | node.py              (2566) | send | Node6 sending message VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [(0, 7, '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'), (1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')], 'name': 'Node3'} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:28:13,870 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node5
2017-08-28 15:28:13,870 | DEBUG    | zstack.py            ( 726) | transmit | Remote Node5 is not connected - message will not be sent immediately.If this problem does not resolve itself - check your firewall settings
2017-08-28 15:28:13,870 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node5
2017-08-28 15:28:13,887 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node4
2017-08-28 15:28:13,892 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node4
2017-08-28 15:28:13,892 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node1
2017-08-28 15:28:13,892 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node1
2017-08-28 15:28:13,893 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node3
2017-08-28 15:28:13,893 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node3
2017-08-28 15:28:13,904 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node2
2017-08-28 15:28:13,905 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":2,"ledgerInfo":[[0,7,"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","op":"VIEW_CHANGE_DONE"}' to Node2
2017-08-28 15:28:13,907 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 4 messages through listener
2017-08-28 15:28:13,924 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,936 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node1')
2017-08-28 15:28:13,936 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,947 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,949 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node4')
2017-08-28 15:28:13,949 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,959 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,974 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node3')
2017-08-28 15:28:13,975 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,981 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,984 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1, 'op': 'CONSISTENCY_PROOF'}, 'Node2')
2017-08-28 15:28:13,989 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1}
2017-08-28 15:28:13,996 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node1
2017-08-28 15:28:13,997 | DEBUG    | ledger_manager.py    ( 351) | canProcessConsistencyProof | Node6 cannot process consistency proof since canSync is False
2017-08-28 15:28:14,005 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node4
2017-08-28 15:28:14,011 | DEBUG    | ledger_manager.py    ( 351) | canProcessConsistencyProof | Node6 cannot process consistency proof since canSync is False
2017-08-28 15:28:14,012 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node3
2017-08-28 15:28:14,013 | DEBUG    | ledger_manager.py    ( 351) | canProcessConsistencyProof | Node6 cannot process consistency proof since canSync is False
2017-08-28 15:28:14,013 | DEBUG    | ledger_manager.py    ( 335) | processConsistencyProof | Node6 received consistency proof: CONSISTENCY_PROOF{'hashes': ['jGLMZFGcgHUKKu3WgPHBiDkc9H9CVvA4gmrmS3PnmUb', 'dkwHWCFiwe2wbhEexYxaQWdtpxCuP3hfXcrEQS5DZFC', 'FnKGRvRo7xL57adPnN2aqCYoUpTEnoD9oTgy3BKedf73', 'GP38KJDvXgfPRRZYoabDS3yMBEprbtFzBWdPehWc7TSN', 'FUUbzChmnGjrGChBv3LZoKunodBPrVuMcg2vUrhkndmz'], 'seqNoStart': 9, 'seqNoEnd': 15, 'viewNo': 0, 'newMerkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ppSeqNo': 6, 'oldMerkleRoot': '42dQyZ5xFGCTdN3EEkEkAGKoyZv7NGsBF1ps6DZ1KM1t', 'ledgerId': 1} from Node2
2017-08-28 15:28:14,036 | DEBUG    | ledger_manager.py    ( 351) | canProcessConsistencyProof | Node6 cannot process consistency proof since canSync is False
2017-08-28 15:28:14,075 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 1.713 seconds
2017-08-28 15:28:18,518 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 80
2017-08-28 15:28:18,519 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:28:18,519 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:28:18,531 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:28:18,532 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:28:18,532 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:28:18,532 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:28:18,535 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 82 to run in 10 seconds
2017-08-28 15:28:25,211 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:25,224 | DEBUG    | zstack.py            ( 661) | handlePingPong | Node6 got ping from Node5
2017-08-28 15:28:25,224 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 ponged Node5
2017-08-28 15:28:25,239 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'po' to Node5
2017-08-28 15:28:25,252 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'po' to Node5
2017-08-28 15:28:25,377 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:25,378 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"viewNo":2,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"op":"CURRENT_STATE"}', '{"ledgerId":0,"viewNo":2,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","op":"LEDGER_STATUS","txnSeqNo":7,"ppSeqNo":6}', '{"ledgerId":1,"viewNo":null,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"LEDGER_STATUS","txnSeqNo":15,"ppSeqNo":null}', '{"ledgerId":2,"viewNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","op":"LEDGER_STATUS","txnSeqNo":0,"ppSeqNo":null}']}
2017-08-28 15:28:25,388 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"viewNo":2,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"op":"CURRENT_STATE"}', '{"ledgerId":0,"viewNo":2,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","op":"LEDGER_STATUS","txnSeqNo":7,"ppSeqNo":6}', '{"ledgerId":1,"viewNo":null,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"LEDGER_STATUS","txnSeqNo":15,"ppSeqNo":null}', '{"ledgerId":2,"viewNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","op":"LEDGER_STATUS","txnSeqNo":0,"ppSeqNo":null}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:28:25,403 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"viewNo":2,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"op":"CURRENT_STATE"}', '{"ledgerId":0,"viewNo":2,"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","op":"LEDGER_STATUS","txnSeqNo":7,"ppSeqNo":6}', '{"ledgerId":1,"viewNo":null,"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","op":"LEDGER_STATUS","txnSeqNo":15,"ppSeqNo":null}', '{"ledgerId":2,"viewNo":null,"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","op":"LEDGER_STATUS","txnSeqNo":0,"ppSeqNo":null}']}
2017-08-28 15:28:25,408 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:25,408 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}], 'op': 'CURRENT_STATE'}, 'Node5')
2017-08-28 15:28:25,409 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]}
2017-08-28 15:28:25,409 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:25,410 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'txnSeqNo': 7, 'viewNo': 2, 'ppSeqNo': 6, 'ledgerId': 0, 'op': 'LEDGER_STATUS'}, 'Node5')
2017-08-28 15:28:25,420 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7}
2017-08-28 15:28:25,428 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:25,428 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'txnSeqNo': 15, 'viewNo': None, 'ppSeqNo': None, 'ledgerId': 1, 'op': 'LEDGER_STATUS'}, 'Node5')
2017-08-28 15:28:25,428 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15}
2017-08-28 15:28:25,434 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:25,434 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'txnSeqNo': 0, 'viewNo': None, 'ppSeqNo': None, 'ledgerId': 2, 'op': 'LEDGER_STATUS'}, 'Node5')
2017-08-28 15:28:25,444 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0}
2017-08-28 15:28:25,445 | DEBUG    | node.py              ( 889) | process_current_state_message | Node6 processing current state CURRENT_STATE{'viewNo': 2, 'primary': [{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}]} from Node5
2017-08-28 15:28:25,463 | DEBUG    | node.py              (1187) | sendToElector | Node6 sending message to elector: (VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}, 'Node5')
2017-08-28 15:28:25,465 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': 2, 'ppSeqNo': 6, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} from Node5
2017-08-28 15:28:25,465 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 7 with 7
2017-08-28 15:28:25,466 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 0 of size 7 with 7
2017-08-28 15:28:25,466 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} from Node5
2017-08-28 15:28:25,479 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 15 with 15
2017-08-28 15:28:25,479 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 1 of size 15 with 15
2017-08-28 15:28:25,480 | DEBUG    | ledger_manager.py    ( 246) | processLedgerStatus | Node6 received ledger status: LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} from Node5
2017-08-28 15:28:25,486 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:25,503 | DEBUG    | ledger_manager.py    ( 984) | _compareLedger | Node6 comparing its ledger 2 of size 0 with 0
2017-08-28 15:28:25,506 | DEBUG    | ledger_manager.py    ( 312) | processLedgerStatus | Node6 found out from {'Node5', 'Node4', 'Node3', 'Node1', 'Node2'} that its ledger of type 2 is latest
2017-08-28 15:28:25,514 | DEBUG    | primary_selector.py  ( 106) | _processViewChangeDoneMessage | Node6's primary selector started processing of ViewChangeDone msg from Node5 : VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'}
2017-08-28 15:28:25,524 | DEBUG    | message_processor.py (  29) | discard | Node6 discarding message VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': [[0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'], [1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'], [2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn']], 'name': 'Node3'} because it already decided primary which is Node3:0
2017-08-28 15:28:25,553 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.178 seconds
2017-08-28 15:28:26,625 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action partial(request_txns_if_needed) with id 2
2017-08-28 15:28:26,626 | DEBUG    | ledger_manager.py    ( 136) | request_txns_if_needed | Node6 not missing any transactions for ledger 0
2017-08-28 15:28:26,652 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:26,675 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:28:26,676 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:26,676 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:26,676 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:26,677 | DEBUG    | zstack.py            ( 643) | sendPingPong | Node6 pinged Node5
2017-08-28 15:28:26,679 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:28:26,721 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'pi' to Node5
2017-08-28 15:28:26,721 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'pi' to Node5
2017-08-28 15:28:26,771 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:28:26,776 | DEBUG    | zstack.py            ( 667) | handlePingPong | Node6 got pong from Node5
2017-08-28 15:28:26,800 | DEBUG    | keep_in_touch.py     (  68) | conns | Node6's connections changed from {'Node1', 'Node3', 'Node2', 'Node4'} to {'Node1', 'Node3', 'Node2', 'Node4', 'Node5'}
2017-08-28 15:28:26,801 | INFO     | keep_in_touch.py     (  98) | _connsChanged | CONNECTION: Node6 now connected to Node5
2017-08-28 15:28:26,803 | DEBUG    | motor.py             (  34) | set_status | Node6 changing status from started_hungry to started
2017-08-28 15:28:26,804 | DEBUG    | node.py              ( 923) | checkInstances | Node6 choosing to start election on the basis of count 6 and nodes {'Node1', 'Node3', 'Node2', 'Node4', 'Node5'}
2017-08-28 15:28:26,805 | DEBUG    | node.py              ( 884) | send_current_state_to_lagging_node | Node6 sending current state CURRENT_STATE{'viewNo': 2, 'primary': [VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': ((0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')), 'name': 'Node3'}]} to lagged node Node5
2017-08-28 15:28:26,809 | DEBUG    | node.py              (2566) | send | Node6 sending message CURRENT_STATE{'viewNo': 2, 'primary': [VIEW_CHANGE_DONE{'viewNo': 2, 'ledgerInfo': ((0, 4, '5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3'), (1, 15, 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT'), (2, 0, 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn')), 'name': 'Node3'}]} to 1 recipients: ['Node5']
2017-08-28 15:28:26,810 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ledgerId': 0, 'txnSeqNo': 7} to 1 recipients: ['Node5']
2017-08-28 15:28:26,828 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT', 'ledgerId': 1, 'txnSeqNo': 15} to 1 recipients: ['Node5']
2017-08-28 15:28:26,828 | DEBUG    | node.py              (2566) | send | Node6 sending message LEDGER_STATUS{'viewNo': None, 'ppSeqNo': None, 'merkleRoot': 'GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn', 'ledgerId': 2, 'txnSeqNo': 0} to 1 recipients: ['Node5']
2017-08-28 15:28:26,839 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 4 msgs to Node5 into one transmission
2017-08-28 15:28:26,848 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"viewNo":2,"primary":[{"ledgerInfo":[[0,4,"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3"],[1,15,"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT"],[2,0,"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn"]],"name":"Node3","viewNo":2}],"op":"CURRENT_STATE"}', b'{"merkleRoot":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","txnSeqNo":7,"viewNo":null,"ppSeqNo":null,"ledgerId":0,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT","txnSeqNo":15,"viewNo":null,"ppSeqNo":null,"ledgerId":1,"op":"LEDGER_STATUS"}', b'{"merkleRoot":"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn","txnSeqNo":0,"viewNo":null,"ppSeqNo":null,"ledgerId":2,"op":"LEDGER_STATUS"}'])
2017-08-28 15:28:26,848 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node5: b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,15,\\"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT\\"],[2,0,\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\"]],\\"name\\":\\"Node3\\",\\"viewNo\\":2}],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT\\",\\"txnSeqNo\\":15,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"txnSeqNo\\":0,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":2,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}'
2017-08-28 15:28:26,871 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"viewNo\\":2,\\"primary\\":[{\\"ledgerInfo\\":[[0,4,\\"5xizCdcGJoYwSK5swMP4BDasTxDbULANZozqM2M2uRo3\\"],[1,15,\\"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT\\"],[2,0,\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\"]],\\"name\\":\\"Node3\\",\\"viewNo\\":2}],\\"op\\":\\"CURRENT_STATE\\"}","{\\"merkleRoot\\":\\"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ\\",\\"txnSeqNo\\":7,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":0,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"EnsWAN36gj3N5ye36x1B4f5W1jpritjXhxyAsDJgf1FT\\",\\"txnSeqNo\\":15,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":1,\\"op\\":\\"LEDGER_STATUS\\"}","{\\"merkleRoot\\":\\"GKot5hBsd81kMupNCXHaqbhv3huEbxAFMLnpcX2hniwn\\",\\"txnSeqNo\\":0,\\"viewNo\\":null,\\"ppSeqNo\\":null,\\"ledgerId\\":2,\\"op\\":\\"LEDGER_STATUS\\"}"],"op":"BATCH"}' to Node5
2017-08-28 15:28:26,873 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.103 seconds
2017-08-28 15:28:28,608 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 82
2017-08-28 15:28:28,616 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:28:28,616 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:28:28,616 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:28:28,634 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:28:28,634 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:28:28,635 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:28:28,635 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 83 to run in 10 seconds
2017-08-28 15:28:28,643 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action partial(request_txns_if_needed) with id 4
2017-08-28 15:28:28,644 | DEBUG    | ledger_manager.py    ( 136) | request_txns_if_needed | Node6 not missing any transactions for ledger 1
2017-08-28 15:28:28,719 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.130 seconds
2017-08-28 15:28:31,254 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action partial(request_CPs_if_needed) with id 1
2017-08-28 15:28:37,908 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action partial(request_CPs_if_needed) with id 3
2017-08-28 15:28:38,673 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 83
2017-08-28 15:28:38,673 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:28:38,674 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:28:38,674 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:28:38,682 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:28:38,682 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:28:38,695 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:28:38,695 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 84 to run in 10 seconds
2017-08-28 15:28:41,690 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:41,691 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:28:41,692 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:41,692 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:41,692 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:41,694 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:28:48,700 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 84
2017-08-28 15:28:48,700 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:28:48,701 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:28:48,715 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:28:48,716 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:28:48,716 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:28:48,718 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:28:48,728 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 85 to run in 10 seconds
2017-08-28 15:28:56,708 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:28:56,708 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:28:56,709 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:28:56,709 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:28:56,709 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:28:56,710 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:28:58,731 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 85
2017-08-28 15:28:58,747 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:28:58,748 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:28:58,749 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:28:58,749 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:28:58,750 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:28:58,767 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:28:58,768 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 86 to run in 10 seconds
2017-08-28 15:29:07,024 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 79
2017-08-28 15:29:07,027 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:29:07,027 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:29:07,030 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 87 to run in 60 seconds
2017-08-28 15:29:07,031 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 12
2017-08-28 15:29:07,035 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:29:07,036 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 13 to run in 60 seconds
2017-08-28 15:29:08,802 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 86
2017-08-28 15:29:08,818 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:29:08,819 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:29:08,822 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:29:08,823 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:29:08,823 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:29:08,827 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:29:08,827 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 88 to run in 10 seconds
2017-08-28 15:29:10,982 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action _check_view_change_completed with id 81
2017-08-28 15:29:10,987 | DEBUG    | node.py              ( 993) | _check_view_change_completed | Node6 running the scheduled check for view change completion
2017-08-28 15:29:10,995 | DEBUG    | node.py              ( 995) | _check_view_change_completed | Node6 already completion view change
2017-08-28 15:29:11,733 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:29:11,734 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:29:11,734 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:29:11,737 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:29:11,748 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:29:11,749 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:29:18,829 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 88
2017-08-28 15:29:18,830 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:29:18,830 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:29:18,831 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:29:18,835 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:29:18,839 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:29:18,840 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:29:18,840 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 89 to run in 10 seconds
2017-08-28 15:29:26,757 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:29:26,757 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:29:26,758 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:29:26,758 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:29:26,760 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:29:26,772 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:29:28,853 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 89
2017-08-28 15:29:28,854 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:29:28,854 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:29:28,856 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:29:28,856 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:29:28,856 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:29:28,859 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:29:28,861 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 90 to run in 10 seconds
2017-08-28 15:29:38,872 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 90
2017-08-28 15:29:38,875 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:29:38,878 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:29:38,882 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:29:38,882 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:29:38,882 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:29:38,883 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:29:38,883 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 91 to run in 10 seconds
2017-08-28 15:29:41,762 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:29:41,773 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:29:41,773 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:29:41,773 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:29:41,773 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:29:41,774 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:29:48,909 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 91
2017-08-28 15:29:48,914 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:29:48,923 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:29:48,927 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:29:48,928 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:29:48,928 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:29:48,928 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:29:48,929 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 92 to run in 10 seconds
2017-08-28 15:29:56,764 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:29:56,788 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:29:56,788 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:29:56,788 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:29:56,789 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:29:56,789 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:29:58,938 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 92
2017-08-28 15:29:58,938 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:29:58,939 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:29:58,947 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:29:58,952 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:29:58,955 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:29:58,956 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:29:58,956 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 93 to run in 10 seconds
2017-08-28 15:30:07,047 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 87
2017-08-28 15:30:07,050 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:30:07,054 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:30:07,054 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 94 to run in 60 seconds
2017-08-28 15:30:07,054 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 13
2017-08-28 15:30:07,054 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:30:07,055 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 14 to run in 60 seconds
2017-08-28 15:30:08,977 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 93
2017-08-28 15:30:08,978 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:30:08,978 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:30:08,978 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:30:08,995 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:30:08,996 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:30:08,996 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:30:08,996 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 95 to run in 10 seconds
2017-08-28 15:30:11,797 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:30:11,797 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:30:11,798 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:30:11,798 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:30:11,801 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:30:11,802 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:30:18,999 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 95
2017-08-28 15:30:19,000 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:30:19,003 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:30:19,003 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:30:19,010 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:30:19,011 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:30:19,014 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:30:19,015 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 96 to run in 10 seconds
2017-08-28 15:30:26,811 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:30:26,828 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:30:26,828 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:30:26,828 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:30:26,829 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:30:26,832 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:30:29,041 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 96
2017-08-28 15:30:29,042 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:30:29,042 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:30:29,042 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:30:29,042 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:30:29,042 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:30:29,043 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:30:29,043 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 97 to run in 10 seconds
2017-08-28 15:30:39,065 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 97
2017-08-28 15:30:39,066 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:30:39,067 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:30:39,067 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:30:39,088 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:30:39,089 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:30:39,089 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:30:39,089 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 98 to run in 10 seconds
2017-08-28 15:30:41,828 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:30:41,832 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:30:41,833 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:30:41,851 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:30:41,852 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:30:41,852 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:30:49,110 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 98
2017-08-28 15:30:49,121 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:30:49,124 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:30:49,124 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:30:49,124 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:30:49,128 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:30:49,130 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:30:49,130 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 99 to run in 10 seconds
2017-08-28 15:30:56,842 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:30:56,845 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:30:56,846 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:30:56,848 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:30:56,849 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:30:56,849 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:30:59,167 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 99
2017-08-28 15:30:59,168 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:30:59,168 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:30:59,168 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:30:59,169 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:30:59,171 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:30:59,173 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:30:59,183 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 100 to run in 10 seconds
2017-08-28 15:31:07,080 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 94
2017-08-28 15:31:07,086 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:31:07,086 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:31:07,087 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 101 to run in 60 seconds
2017-08-28 15:31:07,089 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 14
2017-08-28 15:31:07,090 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:31:07,090 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 15 to run in 60 seconds
2017-08-28 15:31:09,190 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 100
2017-08-28 15:31:09,193 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:31:09,195 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:31:09,196 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:31:09,196 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:31:09,196 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:31:09,197 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:31:09,200 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 102 to run in 10 seconds
2017-08-28 15:31:11,849 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:31:11,850 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:31:11,855 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:31:11,855 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:31:11,855 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:31:11,856 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:31:16,159 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.125 seconds
2017-08-28 15:31:19,236 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 102
2017-08-28 15:31:19,237 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:31:19,239 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:31:19,244 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:31:19,244 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:31:19,245 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:31:19,245 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:31:19,245 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 103 to run in 10 seconds
2017-08-28 15:31:26,883 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:31:26,902 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:31:26,903 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:31:26,903 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:31:26,909 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:31:26,910 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:31:29,261 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 103
2017-08-28 15:31:29,267 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:31:29,267 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:31:29,270 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:31:29,271 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:31:29,273 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:31:29,273 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:31:29,274 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 104 to run in 10 seconds
2017-08-28 15:31:39,276 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 104
2017-08-28 15:31:39,279 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:31:39,280 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:31:39,287 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:31:39,288 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:31:39,288 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:31:39,288 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:31:39,288 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 105 to run in 10 seconds
2017-08-28 15:31:41,899 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:31:41,900 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:31:41,900 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:31:41,901 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:31:41,902 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:31:41,904 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:31:49,290 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 105
2017-08-28 15:31:49,296 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:31:49,297 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:31:49,297 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:31:49,316 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:31:49,317 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:31:49,317 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:31:49,317 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 106 to run in 10 seconds
2017-08-28 15:31:56,913 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:31:56,914 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:31:56,914 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:31:56,915 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:31:56,915 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:31:56,923 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:31:59,342 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 106
2017-08-28 15:31:59,351 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:31:59,351 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:31:59,352 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:31:59,367 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:31:59,367 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:31:59,368 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:31:59,368 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 107 to run in 10 seconds
2017-08-28 15:32:07,110 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 101
2017-08-28 15:32:07,111 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:32:07,114 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:32:07,114 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 108 to run in 60 seconds
2017-08-28 15:32:07,115 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 15
2017-08-28 15:32:07,115 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:32:07,115 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 16 to run in 60 seconds
2017-08-28 15:32:09,384 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 107
2017-08-28 15:32:09,384 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:32:09,385 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:32:09,393 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:32:09,393 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:09,394 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:32:09,394 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:32:09,394 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 109 to run in 10 seconds
2017-08-28 15:32:11,927 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:32:11,940 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:32:11,940 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:32:11,940 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:32:11,941 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:32:11,941 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:32:19,430 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 109
2017-08-28 15:32:19,435 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:32:19,435 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:32:19,436 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:32:19,436 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:19,438 | TRACE    | monitor.py           ( 391) | isMasterAvgReqLatencyTooHigh | Node6 found difference between master and backups avg latencies to be acceptable
2017-08-28 15:32:19,438 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:32:19,448 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 110 to run in 10 seconds
2017-08-28 15:32:21,297 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6C got 1 messages through listener
2017-08-28 15:32:21,323 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated XhYtvJqezMUKfF6KVNaGmT signature on  request 1503934341241353
2017-08-28 15:32:21,323 | TRACE    | node.py              (1370) | validateClientMsg | Node6C received CLIENT message: SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,342 | DEBUG    | node.py              (1416) | processClientInBox | Node6C processing b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' request SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,348 | DEBUG    | node.py              (1690) | processRequest | Node6 received client request: SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'} from b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:32:21,357 | TRACE    | propagator.py        ( 164) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,358 | DEBUG    | propagator.py        ( 148) | propagate | Node6 propagating XhYtvJqezMUKfF6KVNaGmT request 1503934341241353 from client b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:32:21,362 | DEBUG    | node.py              (2566) | send | Node6 sending message PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:21,363 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'} to its replicas since not finalised
2017-08-28 15:32:21,381 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934341241353,"op":"REQACK","identifier":"XhYtvJqezMUKfF6KVNaGmT"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:32:21,384 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:32:21,397 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:32:21,401 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:32:21,410 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:32:21,419 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:32:21,426 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:32:21,427 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:32:21,436 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:32:21,451 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:32:21,457 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk","reqId":1503934341241353,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.7","node_port":9701,"client_ip":"10.0.0.7","alias":"Node6","services":[]},"dest":"6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G"},"identifier":"XhYtvJqezMUKfF6KVNaGmT"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:32:21,478 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 5 messages through listener
2017-08-28 15:32:21,493 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated XhYtvJqezMUKfF6KVNaGmT signature on propagate  request 1503934341241353
2017-08-28 15:32:21,493 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,496 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}, 'op': 'PROPAGATE'}, 'Node5')
2017-08-28 15:32:21,496 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,505 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated XhYtvJqezMUKfF6KVNaGmT signature on propagate  request 1503934341241353
2017-08-28 15:32:21,521 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,522 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}, 'op': 'PROPAGATE'}, 'Node3')
2017-08-28 15:32:21,522 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,531 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated XhYtvJqezMUKfF6KVNaGmT signature on propagate  request 1503934341241353
2017-08-28 15:32:21,538 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,538 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}, 'op': 'PROPAGATE'}, 'Node4')
2017-08-28 15:32:21,549 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,551 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated XhYtvJqezMUKfF6KVNaGmT signature on propagate  request 1503934341241353
2017-08-28 15:32:21,562 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'services': [], 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,563 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'services': [], 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}, 'op': 'PROPAGATE'}, 'Node1')
2017-08-28 15:32:21,572 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'services': [], 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,573 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated XhYtvJqezMUKfF6KVNaGmT signature on propagate  request 1503934341241353
2017-08-28 15:32:21,584 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'services': [], 'client_ip': '10.0.0.7', 'alias': 'Node6', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,585 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'services': [], 'client_ip': '10.0.0.7', 'alias': 'Node6', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}, 'op': 'PROPAGATE'}, 'Node2')
2017-08-28 15:32:21,585 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'services': [], 'client_ip': '10.0.0.7', 'alias': 'Node6', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,597 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,609 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,617 | DEBUG    | propagator.py        ( 213) | forward | Node6 forwarding request ('XhYtvJqezMUKfF6KVNaGmT', 1503934341241353) to 0 replicas
2017-08-28 15:32:21,618 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,624 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,632 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'} to its replicas since already forwarded
2017-08-28 15:32:21,645 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,650 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,655 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'node_port': 9701, 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'} to its replicas since already forwarded
2017-08-28 15:32:21,661 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'services': [], 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,662 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'services': [], 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,672 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node6', 'services': [], 'client_ip': '10.0.0.7', 'node_ip': '10.0.0.7', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'} to its replicas since already forwarded
2017-08-28 15:32:21,681 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'services': [], 'client_ip': '10.0.0.7', 'alias': 'Node6', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}}
2017-08-28 15:32:21,687 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'services': [], 'client_ip': '10.0.0.7', 'alias': 'Node6', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}
2017-08-28 15:32:21,690 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'services': [], 'client_ip': '10.0.0.7', 'alias': 'Node6', 'node_port': 9701}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'} to its replicas since already forwarded
2017-08-28 15:32:21,765 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.496 seconds
2017-08-28 15:32:21,811 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:32:21,821 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [['XhYtvJqezMUKfF6KVNaGmT', 1503934341241353]], 'ppTime': 1503934341, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:32:21,827 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'discarded': 1, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'ppSeqNo': 7, 'instId': 1, 'reqIdr': [['XhYtvJqezMUKfF6KVNaGmT', 1503934341241353]], 'stateRootHash': None, 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node4')
2017-08-28 15:32:21,828 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [['XhYtvJqezMUKfF6KVNaGmT', 1503934341241353]], 'ppTime': 1503934341, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:32:21,841 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:1 received PRE-PREPARE(2, 7) from Node4:1 at 250637.990665112
2017-08-28 15:32:21,856 | WARNING  | replica.py           ( 983) | __is_next_pre_prepare | Node6:1 missing PRE-PREPAREs between 7 and 0
2017-08-28 15:32:21,856 | DEBUG    | replica.py           ( 512) | _setup_for_non_master | Node6:1 Setting last ordered for non-master as (2, 0)
2017-08-28 15:32:21,856 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 0)
2017-08-28 15:32:21,865 | DEBUG    | replica.py           (1822) | enqueue_pre_prepare | Queueing pre-prepares due to unavailability of previous pre-prepares. PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [('XhYtvJqezMUKfF6KVNaGmT', 1503934341241353)], 'ppTime': 1503934341, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None} from Node4:1
2017-08-28 15:32:21,869 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:32:21,870 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [['XhYtvJqezMUKfF6KVNaGmT', 1503934341241353]], 'ppTime': 1503934341, 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:32:21,882 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'reqIdr': [['XhYtvJqezMUKfF6KVNaGmT', 1503934341241353]], 'instId': 0, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'discarded': 0, 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'viewNo': 2, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node3')
2017-08-28 15:32:21,883 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [['XhYtvJqezMUKfF6KVNaGmT', 1503934341241353]], 'ppTime': 1503934341, 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:32:21,906 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:0 received PRE-PREPARE(2, 7) from Node3:0 at 250638.054801153
2017-08-28 15:32:21,913 | DEBUG    | replica.py           (1034) | validate_pre_prepare | Node6:0 state root before processing PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [('XhYtvJqezMUKfF6KVNaGmT', 1503934341241353)], 'ppTime': 1503934341, 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} is b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9', 67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ
2017-08-28 15:32:21,926 | WARNING  | replica.py           ( 655) | processReqDuringBatch | Node6:0 encountered exception UnauthorizedClientRequest('XhYtvJqezMUKfF6KVNaGmT is not a steward of node 6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G',) while processing Request: {'signature': '5r2UjHCY4xdr1gZMc6RbtWBPX28UpwRj4CgdHkkiABEaS5MqrSjswS3VUssmGEBTzpJ3cnKjmuqqJ2dakXecWNKk', 'reqId': 1503934341241353, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.7', 'node_port': 9701, 'client_ip': '10.0.0.7', 'alias': 'Node6', 'services': []}, 'dest': '6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G'}, 'identifier': 'XhYtvJqezMUKfF6KVNaGmT'}, will reject
2017-08-28 15:32:21,927 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:0 Sending PREPARE(2, 7) at 250638.075941849
2017-08-28 15:32:21,928 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Prepare
2017-08-28 15:32:21,941 | TRACE    | replica.py           (2134) | send | Node6:0 sending PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:21,941 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:21,941 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:0 tracking batch for PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [('XhYtvJqezMUKfF6KVNaGmT', 1503934341241353)], 'ppTime': 1503934341, 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} with state root b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9'
2017-08-28 15:32:21,946 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:0 processed incoming PRE-PREPARE(2, 7)
2017-08-28 15:32:21,947 | WARNING  | replica.py           ( 983) | __is_next_pre_prepare | Node6:1 missing PRE-PREPAREs between 7 and 0
2017-08-28 15:32:21,947 | DEBUG    | replica.py           ( 512) | _setup_for_non_master | Node6:1 Setting last ordered for non-master as (2, 0)
2017-08-28 15:32:21,955 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 0)
2017-08-28 15:32:21,985 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934341241353,"reason":"client request invalid: UnauthorizedClientRequest(\'XhYtvJqezMUKfF6KVNaGmT is not a steward of node 6G9QhQa3HWjRKeRmEvEkLbWWf2t7cw6KLtafzi494G4G\',)","op":"REJECT","identifier":"XhYtvJqezMUKfF6KVNaGmT"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:32:21,993 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:21,994 | WARNING  | replica.py           ( 983) | __is_next_pre_prepare | Node6:1 missing PRE-PREPAREs between 7 and 0
2017-08-28 15:32:21,994 | DEBUG    | replica.py           ( 512) | _setup_for_non_master | Node6:1 Setting last ordered for non-master as (2, 0)
2017-08-28 15:32:21,994 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 0)
2017-08-28 15:32:22,006 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:32:22,016 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,027 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 7, 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node4')
2017-08-28 15:32:22,028 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,028 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,033 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'viewNo': 2, 'ppSeqNo': 7, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node3')
2017-08-28 15:32:22,034 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,034 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,041 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppSeqNo': 7, 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:32:22,041 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,065 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node5
2017-08-28 15:32:22,066 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node5
2017-08-28 15:32:22,075 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node4
2017-08-28 15:32:22,076 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node4
2017-08-28 15:32:22,076 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node1
2017-08-28 15:32:22,093 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node1
2017-08-28 15:32:22,094 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node3
2017-08-28 15:32:22,108 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node3
2017-08-28 15:32:22,108 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node2
2017-08-28 15:32:22,109 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}' to Node2
2017-08-28 15:32:22,110 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 7) from Node4:0
2017-08-28 15:32:22,110 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,110 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,110 | WARNING  | replica.py           ( 983) | __is_next_pre_prepare | Node6:1 missing PRE-PREPAREs between 7 and 0
2017-08-28 15:32:22,110 | DEBUG    | replica.py           ( 512) | _setup_for_non_master | Node6:1 Setting last ordered for non-master as (2, 0)
2017-08-28 15:32:22,111 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 0)
2017-08-28 15:32:22,111 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node3:1
2017-08-28 15:32:22,111 | INFO     | replica.py           (1867) | enqueue_prepare | Node6:1 queueing prepare due to unavailability of PRE-PREPARE. Prepare PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None} from Node3:1
2017-08-28 15:32:22,111 | DEBUG    | replica.py           (1969) | _request_pre_prepare_if_possible | Node6:1 not requesting a PRE-PREPARE because does not have sufficient PREPAREs for (2, 7)
2017-08-28 15:32:22,111 | DEBUG    | replica.py           ( 881) | processPrepare | Node6:1 cannot process incoming PREPARE
2017-08-28 15:32:22,111 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node2:1
2017-08-28 15:32:22,143 | INFO     | replica.py           (1867) | enqueue_prepare | Node6:1 queueing prepare due to unavailability of PRE-PREPARE. Prepare PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None} from Node2:1
2017-08-28 15:32:22,152 | DEBUG    | replica.py           (1969) | _request_pre_prepare_if_possible | Node6:1 not requesting a PRE-PREPARE because does not have sufficient PREPAREs for (2, 7)
2017-08-28 15:32:22,154 | DEBUG    | replica.py           ( 881) | processPrepare | Node6:1 cannot process incoming PREPARE
2017-08-28 15:32:22,162 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:32:22,171 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,172 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'stateRootHash': None, 'txnRootHash': None, 'ppTime': 1503934341, 'instId': 1, 'ppSeqNo': 7, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:32:22,172 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,182 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,183 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppSeqNo': 7, 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:32:22,183 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,183 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"instId":0,"ppTime":1503934341,"ppSeqNo":7,"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934341,"ppSeqNo":7,"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}']}
2017-08-28 15:32:22,191 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"ppTime":1503934341,"ppSeqNo":7,"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934341,"ppSeqNo":7,"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:32:22,192 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"ppTime":1503934341,"ppSeqNo":7,"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934341,"ppSeqNo":7,"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}']}
2017-08-28 15:32:22,200 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,201 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'op': 'PREPARE', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ppSeqNo': 7}, 'Node5')
2017-08-28 15:32:22,206 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,207 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,207 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'op': 'PREPARE', 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'ppSeqNo': 7}, 'Node5')
2017-08-28 15:32:22,207 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,239 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 7) from Node2:0
2017-08-28 15:32:22,246 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,246 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,246 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 7) from Node5:0
2017-08-28 15:32:22,247 | DEBUG    | replica.py           ( 953) | doCommit | Node6:0 Sending COMMIT(2, 7) at 250638.395837021
2017-08-28 15:32:22,254 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Commit
2017-08-28 15:32:22,254 | TRACE    | replica.py           (2134) | send | Node6:0 sending COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,255 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,255 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,264 | WARNING  | replica.py           ( 983) | __is_next_pre_prepare | Node6:1 missing PRE-PREPAREs between 7 and 0
2017-08-28 15:32:22,264 | DEBUG    | replica.py           ( 512) | _setup_for_non_master | Node6:1 Setting last ordered for non-master as (2, 0)
2017-08-28 15:32:22,264 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 0)
2017-08-28 15:32:22,264 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node1:1
2017-08-28 15:32:22,267 | INFO     | replica.py           (1867) | enqueue_prepare | Node6:1 queueing prepare due to unavailability of PRE-PREPARE. Prepare PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None} from Node1:1
2017-08-28 15:32:22,277 | DEBUG    | replica.py           (1969) | _request_pre_prepare_if_possible | Node6:1 not requesting a PRE-PREPARE because does not have sufficient PREPAREs for (2, 7)
2017-08-28 15:32:22,277 | DEBUG    | replica.py           ( 881) | processPrepare | Node6:1 cannot process incoming PREPARE
2017-08-28 15:32:22,277 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node5:1
2017-08-28 15:32:22,278 | INFO     | replica.py           (1867) | enqueue_prepare | Node6:1 queueing prepare due to unavailability of PRE-PREPARE. Prepare PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None} from Node5:1
2017-08-28 15:32:22,278 | DEBUG    | replica.py           (1979) | _request_pre_prepare_if_possible | Node6:1 not requesting a PRE-PREPARE since already found stashed for (2, 7)
2017-08-28 15:32:22,288 | DEBUG    | replica.py           ( 881) | processPrepare | Node6:1 cannot process incoming PREPARE
2017-08-28 15:32:22,289 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:32:22,304 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,309 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppTime': 1503934341, 'instId': 0, 'ppSeqNo': 7, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:32:22,309 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,330 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:22,331 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 7) from Node1:0
2017-08-28 15:32:22,341 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 7, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:22,341 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,342 | WARNING  | replica.py           ( 983) | __is_next_pre_prepare | Node6:1 missing PRE-PREPAREs between 7 and 0
2017-08-28 15:32:22,347 | DEBUG    | replica.py           ( 512) | _setup_for_non_master | Node6:1 Setting last ordered for non-master as (2, 0)
2017-08-28 15:32:22,348 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 6)
2017-08-28 15:32:22,358 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:32:22,358 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: BATCH{'signature': None, 'messages': ['{"ppSeqNo":7,"instId":0,"viewNo":2,"op":"COMMIT"}', '{"ppSeqNo":7,"instId":1,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:32:22,370 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"ppSeqNo":7,"instId":0,"viewNo":2,"op":"COMMIT"}', '{"ppSeqNo":7,"instId":1,"viewNo":2,"op":"COMMIT"}'], 'op': 'BATCH'}, 'Node4')
2017-08-28 15:32:22,370 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"ppSeqNo":7,"instId":0,"viewNo":2,"op":"COMMIT"}', '{"ppSeqNo":7,"instId":1,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:32:22,371 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,371 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:32:22,379 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,385 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,385 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:32:22,385 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,404 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node5
2017-08-28 15:32:22,412 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node5
2017-08-28 15:32:22,412 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node4
2017-08-28 15:32:22,412 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node4
2017-08-28 15:32:22,421 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node1
2017-08-28 15:32:22,422 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node1
2017-08-28 15:32:22,431 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node3
2017-08-28 15:32:22,432 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node3
2017-08-28 15:32:22,438 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node2
2017-08-28 15:32:22,438 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}' to Node2
2017-08-28 15:32:22,447 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 7) from Node4:0
2017-08-28 15:32:22,456 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,457 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,457 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:1 received PRE-PREPARE(2, 7) from Node4:1 at 250638.606152761
2017-08-28 15:32:22,457 | DEBUG    | replica.py           (1885) | dequeue_prepares | Node6:1 popping stashed PREPARE(2, 7)
2017-08-28 15:32:22,468 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node3:1
2017-08-28 15:32:22,468 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,468 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,474 | DEBUG    | replica.py           (1885) | dequeue_prepares | Node6:1 popping stashed PREPARE(2, 7)
2017-08-28 15:32:22,474 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node2:1
2017-08-28 15:32:22,474 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,475 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,475 | DEBUG    | replica.py           (1885) | dequeue_prepares | Node6:1 popping stashed PREPARE(2, 7)
2017-08-28 15:32:22,484 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node1:1
2017-08-28 15:32:22,491 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,492 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,492 | DEBUG    | replica.py           (1885) | dequeue_prepares | Node6:1 popping stashed PREPARE(2, 7)
2017-08-28 15:32:22,492 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 7) from Node5:1
2017-08-28 15:32:22,493 | DEBUG    | replica.py           ( 953) | doCommit | Node6:1 Sending COMMIT(2, 7) at 250638.641722142
2017-08-28 15:32:22,502 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Commit
2017-08-28 15:32:22,502 | TRACE    | replica.py           (2134) | send | Node6:1 sending COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,503 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,503 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 7)
2017-08-28 15:32:22,511 | DEBUG    | replica.py           (1891) | dequeue_prepares | Node6:1 processed 4 PREPAREs waiting for PRE-PREPARE for view no 2 and seq no 7
2017-08-28 15:32:22,512 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:1 Sending PREPARE(2, 7) at 250638.660855598
2017-08-28 15:32:22,512 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Prepare
2017-08-28 15:32:22,512 | TRACE    | replica.py           (2134) | send | Node6:1 sending PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,518 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None}
2017-08-28 15:32:22,519 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:1 tracking batch for PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'reqIdr': [('XhYtvJqezMUKfF6KVNaGmT', 1503934341241353)], 'ppTime': 1503934341, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None} with state root None
2017-08-28 15:32:22,519 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:1 processed incoming PRE-PREPARE(2, 7)
2017-08-28 15:32:22,526 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 7) from Node4:1
2017-08-28 15:32:22,527 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,527 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,538 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 4 messages through listener
2017-08-28 15:32:22,547 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"COMMIT","viewNo":2,"ppSeqNo":7,"instId":0}', '{"op":"COMMIT","viewNo":2,"ppSeqNo":7,"instId":1}']}
2017-08-28 15:32:22,548 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"COMMIT","viewNo":2,"ppSeqNo":7,"instId":0}', '{"op":"COMMIT","viewNo":2,"ppSeqNo":7,"instId":1}'], 'op': 'BATCH'}, 'Node3')
2017-08-28 15:32:22,548 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"COMMIT","viewNo":2,"ppSeqNo":7,"instId":0}', '{"op":"COMMIT","viewNo":2,"ppSeqNo":7,"instId":1}']}
2017-08-28 15:32:22,548 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,557 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 0, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:32:22,558 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,558 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,571 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 1, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:32:22,572 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,572 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":7}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":7}']}
2017-08-28 15:32:22,578 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":7}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":7}'], 'op': 'BATCH'}, 'Node1')
2017-08-28 15:32:22,578 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":7}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":7}']}
2017-08-28 15:32:22,579 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,583 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:32:22,583 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,586 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,586 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:32:22,587 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,595 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":7,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":7,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:32:22,596 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"ppSeqNo":7,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":7,"viewNo":2,"op":"COMMIT"}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:32:22,596 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":7,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":7,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:32:22,604 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,604 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:32:22,611 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,611 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,616 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:32:22,616 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,616 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":7}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":7}']}
2017-08-28 15:32:22,623 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":7}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":7}'], 'op': 'BATCH'}, 'Node2')
2017-08-28 15:32:22,624 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":7}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":7}']}
2017-08-28 15:32:22,624 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,624 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:32:22,636 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,638 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,638 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 7, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:32:22,639 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7}
2017-08-28 15:32:22,647 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:22,673 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': 'f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17', 'ppTime': 1503934341, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 7, 'stateRootHash': None} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:22,673 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 7) from Node3:0
2017-08-28 15:32:22,673 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,684 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,685 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 7) from Node1:0
2017-08-28 15:32:22,685 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,685 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,685 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 7) from Node5:0
2017-08-28 15:32:22,707 | TRACE    | replica.py           ( 925) | tryOrder | Node6:0 returning request to node
2017-08-28 15:32:22,707 | DEBUG    | replica.py           (1437) | doOrder | Node6:0 ordering COMMIT (2, 7)
2017-08-28 15:32:22,707 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:0 set last ordered as (2, 7)
2017-08-28 15:32:22,708 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Ordered
2017-08-28 15:32:22,708 | TRACE    | replica.py           (2134) | send | Node6:0 sending ORDERED{'viewNo': 2, 'ppSeqNo': 7, 'ppTime': 1503934341, 'instId': 0, 'reqIdr': [], 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:32:22,708 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:0 ordered request (2, 7)
2017-08-28 15:32:22,709 | DEBUG    | replica.py           (1540) | _newCheckpointState | Node6:0 adding new checkpoint state for (7, 106)
2017-08-28 15:32:22,709 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,709 | TRACE    | message_processor.py (  29) | discard | Node6:0 discarding message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 7} because already ordered 3 phase message
2017-08-28 15:32:22,718 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 7) from Node3:1
2017-08-28 15:32:22,723 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,724 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,724 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 7) from Node1:1
2017-08-28 15:32:22,724 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7} commits where f is 1
2017-08-28 15:32:22,725 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,730 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 7) from Node5:1
2017-08-28 15:32:22,730 | TRACE    | replica.py           ( 925) | tryOrder | Node6:1 returning request to node
2017-08-28 15:32:22,730 | DEBUG    | replica.py           (1437) | doOrder | Node6:1 ordering COMMIT (2, 7)
2017-08-28 15:32:22,735 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 7)
2017-08-28 15:32:22,737 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Ordered
2017-08-28 15:32:22,739 | TRACE    | replica.py           (2134) | send | Node6:1 sending ORDERED{'viewNo': 2, 'ppSeqNo': 7, 'ppTime': 1503934341, 'instId': 1, 'reqIdr': [('XhYtvJqezMUKfF6KVNaGmT', 1503934341241353)], 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:32:22,744 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:1 ordered request (2, 7)
2017-08-28 15:32:22,745 | DEBUG    | replica.py           (1540) | _newCheckpointState | Node6:1 adding new checkpoint state for (7, 106)
2017-08-28 15:32:22,745 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 7)
2017-08-28 15:32:22,745 | TRACE    | message_processor.py (  29) | discard | Node6:1 discarding message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 7} because already ordered 3 phase message
2017-08-28 15:32:22,762 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node5 into one transmission
2017-08-28 15:32:22,776 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":1,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}', b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}'])
2017-08-28 15:32:22,776 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node5: b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}'
2017-08-28 15:32:22,777 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}' to Node5
2017-08-28 15:32:22,777 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node4 into one transmission
2017-08-28 15:32:22,791 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":1,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}', b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}'])
2017-08-28 15:32:22,792 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node4: b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}'
2017-08-28 15:32:22,795 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}' to Node4
2017-08-28 15:32:22,795 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node1 into one transmission
2017-08-28 15:32:22,799 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":1,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}', b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}'])
2017-08-28 15:32:22,799 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node1: b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}'
2017-08-28 15:32:22,800 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}' to Node1
2017-08-28 15:32:22,800 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node3 into one transmission
2017-08-28 15:32:22,811 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":1,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}', b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}'])
2017-08-28 15:32:22,812 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node3: b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}'
2017-08-28 15:32:22,819 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}' to Node3
2017-08-28 15:32:22,823 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node2 into one transmission
2017-08-28 15:32:22,824 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":1,"viewNo":2,"ppSeqNo":7,"op":"COMMIT"}', b'{"digest":"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934341,"viewNo":2,"ppSeqNo":7}'])
2017-08-28 15:32:22,824 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node2: b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}'
2017-08-28 15:32:22,824 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":7,\\"op\\":\\"COMMIT\\"}","{\\"digest\\":\\"f4083a5551140ae7e2ce519b4ba3a2e73265a576ea7afca97a1510e5596f4b17\\",\\"txnRootHash\\":null,\\"stateRootHash\\":null,\\"op\\":\\"PREPARE\\",\\"instId\\":1,\\"ppTime\\":1503934341,\\"viewNo\\":2,\\"ppSeqNo\\":7}"],"op":"BATCH"}' to Node2
2017-08-28 15:32:22,840 | DEBUG    | node.py              (1828) | processOrdered | Node6 executing Ordered batch 2 7 of 0 requests
2017-08-28 15:32:22,840 | DEBUG    | ledger.py            (  52) | commitTxns | Committed 0 txns, 0 are uncommitted
2017-08-28 15:32:22,851 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:22,852 | TRACE    | node.py              (1839) | processOrdered | Node6 got ordered requests from backup replica 1
2017-08-28 15:32:22,887 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 1.084 seconds
2017-08-28 15:32:26,931 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:32:26,932 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:32:26,933 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:32:26,934 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:32:26,935 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:32:26,939 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:32:29,464 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 110
2017-08-28 15:32:29,464 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:32:29,466 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:32:29,467 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:32:29,467 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:29,471 | TRACE    | monitor.py           ( 378) | isMasterAvgReqLatencyTooHigh | Node6 found master had no record yet for XhYtvJqezMUKfF6KVNaGmT
2017-08-28 15:32:29,473 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:32:29,478 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 111 to run in 10 seconds
2017-08-28 15:32:39,479 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 111
2017-08-28 15:32:39,502 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:32:39,502 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:32:39,503 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:32:39,503 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:39,507 | TRACE    | monitor.py           ( 378) | isMasterAvgReqLatencyTooHigh | Node6 found master had no record yet for XhYtvJqezMUKfF6KVNaGmT
2017-08-28 15:32:39,508 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:32:39,510 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 112 to run in 10 seconds
2017-08-28 15:32:41,951 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:32:41,951 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:32:41,955 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:32:41,956 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:32:41,956 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:32:41,959 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:32:49,539 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 112
2017-08-28 15:32:49,565 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:32:49,566 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:32:49,567 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:32:49,594 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:49,595 | TRACE    | monitor.py           ( 378) | isMasterAvgReqLatencyTooHigh | Node6 found master had no record yet for XhYtvJqezMUKfF6KVNaGmT
2017-08-28 15:32:49,598 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:32:49,598 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 113 to run in 10 seconds
2017-08-28 15:32:49,637 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6C got 1 messages through listener
2017-08-28 15:32:49,647 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated FBQ5W7LxRp8bLeVNEafjpd signature on  request 1503934369465182
2017-08-28 15:32:49,648 | TRACE    | node.py              (1370) | validateClientMsg | Node6C received CLIENT message: SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:49,656 | DEBUG    | node.py              (1416) | processClientInBox | Node6C processing b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' request SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:49,657 | DEBUG    | node.py              (1690) | processRequest | Node6 received client request: SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'} from b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:32:49,668 | TRACE    | propagator.py        ( 164) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:49,677 | DEBUG    | propagator.py        ( 148) | propagate | Node6 propagating FBQ5W7LxRp8bLeVNEafjpd request 1503934369465182 from client b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:32:49,677 | DEBUG    | node.py              (2566) | send | Node6 sending message PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:49,685 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'} to its replicas since not finalised
2017-08-28 15:32:49,686 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934369465182,"op":"REQACK","identifier":"FBQ5W7LxRp8bLeVNEafjpd"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:32:49,697 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:32:49,708 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:32:49,712 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:32:49,723 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:32:49,730 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:32:49,731 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:32:49,740 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:32:49,748 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:32:49,756 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:32:49,757 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss","reqId":1503934369465182,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"FBQ5W7LxRp8bLeVNEafjpd"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:32:49,776 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 5 messages through listener
2017-08-28 15:32:49,793 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated FBQ5W7LxRp8bLeVNEafjpd signature on propagate  request 1503934369465182
2017-08-28 15:32:49,799 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,800 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}, 'op': 'PROPAGATE'}, 'Node4')
2017-08-28 15:32:49,809 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,810 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated FBQ5W7LxRp8bLeVNEafjpd signature on propagate  request 1503934369465182
2017-08-28 15:32:49,821 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,822 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}, 'op': 'PROPAGATE'}, 'Node1')
2017-08-28 15:32:49,831 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,837 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated FBQ5W7LxRp8bLeVNEafjpd signature on propagate  request 1503934369465182
2017-08-28 15:32:49,838 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,846 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}, 'op': 'PROPAGATE'}, 'Node2')
2017-08-28 15:32:49,847 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,862 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated FBQ5W7LxRp8bLeVNEafjpd signature on propagate  request 1503934369465182
2017-08-28 15:32:49,863 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,870 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}, 'op': 'PROPAGATE'}, 'Node5')
2017-08-28 15:32:49,871 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,881 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated FBQ5W7LxRp8bLeVNEafjpd signature on propagate  request 1503934369465182
2017-08-28 15:32:49,891 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,893 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}, 'op': 'PROPAGATE'}, 'Node3')
2017-08-28 15:32:49,894 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,908 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,917 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:49,926 | DEBUG    | propagator.py        ( 213) | forward | Node6 forwarding request ('FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182) to 0 replicas
2017-08-28 15:32:49,927 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,936 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:49,937 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'} to its replicas since already forwarded
2017-08-28 15:32:49,951 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,953 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:49,965 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'} to its replicas since already forwarded
2017-08-28 15:32:49,966 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:49,966 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:49,983 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'} to its replicas since already forwarded
2017-08-28 15:32:49,988 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}}
2017-08-28 15:32:50,001 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}
2017-08-28 15:32:50,002 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'} to its replicas since already forwarded
2017-08-28 15:32:50,062 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.526 seconds
2017-08-28 15:32:50,102 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:32:50,114 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'reqIdr': [['FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182]], 'ppTime': 1503934370, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:32:50,123 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'discarded': 1, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'ppSeqNo': 8, 'instId': 1, 'reqIdr': [['FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182]], 'stateRootHash': None, 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node4')
2017-08-28 15:32:50,123 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'reqIdr': [['FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182]], 'ppTime': 1503934370, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:32:50,146 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:1 received PRE-PREPARE(2, 8) from Node4:1 at 250666.295320085
2017-08-28 15:32:50,155 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:1 Sending PREPARE(2, 8) at 250666.304449765
2017-08-28 15:32:50,156 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Prepare
2017-08-28 15:32:50,156 | TRACE    | replica.py           (2134) | send | Node6:1 sending PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,156 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,156 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:1 tracking batch for PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'reqIdr': [('FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182)], 'ppTime': 1503934370, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None} with state root None
2017-08-28 15:32:50,156 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:1 processed incoming PRE-PREPARE(2, 8)
2017-08-28 15:32:50,157 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:32:50,158 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'reqIdr': [['FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182]], 'ppTime': 1503934370, 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:32:50,158 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'reqIdr': [['FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182]], 'instId': 0, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'discarded': 0, 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'viewNo': 2, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node3')
2017-08-28 15:32:50,158 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'reqIdr': [['FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182]], 'ppTime': 1503934370, 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:32:50,197 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:50,219 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:0 received PRE-PREPARE(2, 8) from Node3:0 at 250666.368196302
2017-08-28 15:32:50,220 | DEBUG    | replica.py           (1034) | validate_pre_prepare | Node6:0 state root before processing PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'reqIdr': [('FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182)], 'ppTime': 1503934370, 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} is b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9', 67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ
2017-08-28 15:32:50,221 | WARNING  | replica.py           ( 655) | processReqDuringBatch | Node6:0 encountered exception UnauthorizedClientRequest('FBQ5W7LxRp8bLeVNEafjpd is not a steward of node 4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc',) while processing Request: {'signature': '5rdFRSA94d7UYhzsY4rrUSM3taAEFoxpgz39vYz6eZdd9kXZJMH5hpiHTUxnWQr1vvrKAARqKUei5FHhYaLzbsss', 'reqId': 1503934369465182, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'FBQ5W7LxRp8bLeVNEafjpd'}, will reject
2017-08-28 15:32:50,232 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:0 Sending PREPARE(2, 8) at 250666.380834438
2017-08-28 15:32:50,232 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Prepare
2017-08-28 15:32:50,242 | TRACE    | replica.py           (2134) | send | Node6:0 sending PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,243 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,249 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:0 tracking batch for PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'reqIdr': [('FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182)], 'ppTime': 1503934370, 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} with state root b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9'
2017-08-28 15:32:50,249 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:0 processed incoming PRE-PREPARE(2, 8)
2017-08-28 15:32:50,283 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node5
2017-08-28 15:32:50,289 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node5
2017-08-28 15:32:50,289 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node4
2017-08-28 15:32:50,290 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node4
2017-08-28 15:32:50,296 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node1
2017-08-28 15:32:50,297 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node1
2017-08-28 15:32:50,297 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node3
2017-08-28 15:32:50,298 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node3
2017-08-28 15:32:50,316 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node2
2017-08-28 15:32:50,316 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node2
2017-08-28 15:32:50,335 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934369465182,"reason":"client request invalid: UnauthorizedClientRequest(\'FBQ5W7LxRp8bLeVNEafjpd is not a steward of node 4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc\',)","op":"REJECT","identifier":"FBQ5W7LxRp8bLeVNEafjpd"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:32:50,350 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:50,362 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 2 messages through listener
2017-08-28 15:32:50,374 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,374 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 8, 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node4')
2017-08-28 15:32:50,374 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,384 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,385 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'viewNo': 2, 'ppSeqNo': 8, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node3')
2017-08-28 15:32:50,385 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,406 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node5
2017-08-28 15:32:50,406 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node5
2017-08-28 15:32:50,407 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node4
2017-08-28 15:32:50,407 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node4
2017-08-28 15:32:50,416 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node1
2017-08-28 15:32:50,416 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node1
2017-08-28 15:32:50,423 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node3
2017-08-28 15:32:50,426 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node3
2017-08-28 15:32:50,427 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node2
2017-08-28 15:32:50,437 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934370,"viewNo":2,"ppSeqNo":8}' to Node2
2017-08-28 15:32:50,438 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 8) from Node4:0
2017-08-28 15:32:50,453 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,453 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,453 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 8) from Node3:1
2017-08-28 15:32:50,454 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,462 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,475 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:32:50,477 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,477 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'stateRootHash': None, 'txnRootHash': None, 'ppTime': 1503934370, 'instId': 1, 'ppSeqNo': 8, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:32:50,477 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,477 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: BATCH{'signature': None, 'messages': ['{"op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","instId":0,"viewNo":2,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","ppSeqNo":8,"ppTime":1503934370,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89"}', '{"op":"PREPARE","stateRootHash":null,"instId":1,"viewNo":2,"txnRootHash":null,"ppSeqNo":8,"ppTime":1503934370,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89"}']}
2017-08-28 15:32:50,477 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","instId":0,"viewNo":2,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","ppSeqNo":8,"ppTime":1503934370,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89"}', '{"op":"PREPARE","stateRootHash":null,"instId":1,"viewNo":2,"txnRootHash":null,"ppSeqNo":8,"ppTime":1503934370,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89"}'], 'op': 'BATCH'}, 'Node2')
2017-08-28 15:32:50,477 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","instId":0,"viewNo":2,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","ppSeqNo":8,"ppTime":1503934370,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89"}', '{"op":"PREPARE","stateRootHash":null,"instId":1,"viewNo":2,"txnRootHash":null,"ppSeqNo":8,"ppTime":1503934370,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89"}']}
2017-08-28 15:32:50,478 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,478 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppSeqNo': 8, 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:32:50,478 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,478 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,497 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppSeqNo': 8, 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:32:50,497 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,515 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"instId":0,"ppTime":1503934370,"ppSeqNo":8,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934370,"ppSeqNo":8,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}']}
2017-08-28 15:32:50,515 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"ppTime":1503934370,"ppSeqNo":8,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934370,"ppSeqNo":8,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:32:50,515 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"ppTime":1503934370,"ppSeqNo":8,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934370,"ppSeqNo":8,"digest":"a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}']}
2017-08-28 15:32:50,525 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,525 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'op': 'PREPARE', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ppSeqNo': 8}, 'Node5')
2017-08-28 15:32:50,525 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,537 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,537 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'op': 'PREPARE', 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'ppSeqNo': 8}, 'Node5')
2017-08-28 15:32:50,537 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,557 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 8) from Node2:0
2017-08-28 15:32:50,572 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,572 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,572 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 8) from Node5:0
2017-08-28 15:32:50,580 | DEBUG    | replica.py           ( 953) | doCommit | Node6:0 Sending COMMIT(2, 8) at 250666.728753633
2017-08-28 15:32:50,580 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Commit
2017-08-28 15:32:50,580 | TRACE    | replica.py           (2134) | send | Node6:0 sending COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,588 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,588 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,588 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 8) from Node1:1
2017-08-28 15:32:50,602 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,603 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,603 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 8) from Node2:1
2017-08-28 15:32:50,607 | DEBUG    | replica.py           ( 953) | doCommit | Node6:1 Sending COMMIT(2, 8) at 250666.756614292
2017-08-28 15:32:50,608 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Commit
2017-08-28 15:32:50,608 | TRACE    | replica.py           (2134) | send | Node6:1 sending COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,608 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,609 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,616 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 8) from Node5:1
2017-08-28 15:32:50,616 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 8, 'stateRootHash': None}
2017-08-28 15:32:50,627 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,634 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:32:50,635 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,645 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppTime': 1503934370, 'instId': 0, 'ppSeqNo': 8, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:32:50,646 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,661 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:50,671 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:32:50,671 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 8) from Node1:0
2017-08-28 15:32:50,680 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': 'a59b7a209feecc73354aa1179af0065f6a0b7a2e894f20ee74302ef8031d1e89', 'ppTime': 1503934370, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 8, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:32:50,680 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 8)
2017-08-28 15:32:50,681 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 2 messages through listener
2017-08-28 15:32:50,682 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,682 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:32:50,682 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,683 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,683 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 1, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:32:50,683 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,710 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node5 into one transmission
2017-08-28 15:32:50,722 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}'])
2017-08-28 15:32:50,722 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node5: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:32:50,729 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node5
2017-08-28 15:32:50,729 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node4 into one transmission
2017-08-28 15:32:50,729 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}'])
2017-08-28 15:32:50,730 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node4: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:32:50,739 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node4
2017-08-28 15:32:50,745 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node1 into one transmission
2017-08-28 15:32:50,745 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}'])
2017-08-28 15:32:50,745 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node1: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:32:50,746 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node1
2017-08-28 15:32:50,757 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node3 into one transmission
2017-08-28 15:32:50,757 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}'])
2017-08-28 15:32:50,757 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node3: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:32:50,758 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node3
2017-08-28 15:32:50,769 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node2 into one transmission
2017-08-28 15:32:50,769 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":8,"op":"COMMIT"}'])
2017-08-28 15:32:50,769 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node2: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:32:50,775 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":8,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node2
2017-08-28 15:32:50,780 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 8) from Node4:1
2017-08-28 15:32:50,789 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,789 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 8)
2017-08-28 15:32:50,790 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 8) from Node3:1
2017-08-28 15:32:50,797 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,797 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 8)
2017-08-28 15:32:50,798 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 5 messages through listener
2017-08-28 15:32:50,810 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":8,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":8,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:32:50,819 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"ppSeqNo":8,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":8,"viewNo":2,"op":"COMMIT"}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:32:50,819 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":8,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":8,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:32:50,820 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,825 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:32:50,826 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,826 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,826 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:32:50,835 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,838 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":8}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":8}']}
2017-08-28 15:32:50,838 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":8}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":8}'], 'op': 'BATCH'}, 'Node2')
2017-08-28 15:32:50,838 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":8}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":8}']}
2017-08-28 15:32:50,839 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,847 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:32:50,850 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,850 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,851 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:32:50,861 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,861 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,861 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 0, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:32:50,871 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,871 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":8}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":8}']}
2017-08-28 15:32:50,876 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":8}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":8}'], 'op': 'BATCH'}, 'Node1')
2017-08-28 15:32:50,877 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":8}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":8}']}
2017-08-28 15:32:50,877 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,877 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:32:50,891 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,892 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,892 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:32:50,895 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,896 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,896 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 8, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:32:50,896 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8}
2017-08-28 15:32:50,930 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 8) from Node5:0
2017-08-28 15:32:50,930 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,930 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 8)
2017-08-28 15:32:50,931 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 8) from Node2:0
2017-08-28 15:32:50,939 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,940 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 8)
2017-08-28 15:32:50,940 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 8) from Node3:0
2017-08-28 15:32:50,948 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,949 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 8)
2017-08-28 15:32:50,949 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 8) from Node1:0
2017-08-28 15:32:50,949 | TRACE    | replica.py           ( 925) | tryOrder | Node6:0 returning request to node
2017-08-28 15:32:50,958 | DEBUG    | replica.py           (1437) | doOrder | Node6:0 ordering COMMIT (2, 8)
2017-08-28 15:32:50,958 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:0 set last ordered as (2, 8)
2017-08-28 15:32:50,959 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Ordered
2017-08-28 15:32:50,971 | TRACE    | replica.py           (2134) | send | Node6:0 sending ORDERED{'viewNo': 2, 'ppSeqNo': 8, 'ppTime': 1503934370, 'instId': 0, 'reqIdr': [], 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:32:50,971 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:0 ordered request (2, 8)
2017-08-28 15:32:50,972 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 8)
2017-08-28 15:32:50,972 | TRACE    | message_processor.py (  29) | discard | Node6:0 discarding message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 8} because already ordered 3 phase message
2017-08-28 15:32:50,973 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 8) from Node5:1
2017-08-28 15:32:50,979 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8} commits where f is 1
2017-08-28 15:32:50,980 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 8)
2017-08-28 15:32:50,980 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 8) from Node2:1
2017-08-28 15:32:50,991 | TRACE    | replica.py           ( 925) | tryOrder | Node6:1 returning request to node
2017-08-28 15:32:50,991 | DEBUG    | replica.py           (1437) | doOrder | Node6:1 ordering COMMIT (2, 8)
2017-08-28 15:32:50,994 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 8)
2017-08-28 15:32:50,994 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Ordered
2017-08-28 15:32:50,995 | TRACE    | replica.py           (2134) | send | Node6:1 sending ORDERED{'viewNo': 2, 'ppSeqNo': 8, 'ppTime': 1503934370, 'instId': 1, 'reqIdr': [('FBQ5W7LxRp8bLeVNEafjpd', 1503934369465182)], 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:32:50,995 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:1 ordered request (2, 8)
2017-08-28 15:32:51,003 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 8)
2017-08-28 15:32:51,004 | TRACE    | message_processor.py (  29) | discard | Node6:1 discarding message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 8} because already ordered 3 phase message
2017-08-28 15:32:51,032 | DEBUG    | node.py              (1828) | processOrdered | Node6 executing Ordered batch 2 8 of 0 requests
2017-08-28 15:32:51,032 | DEBUG    | ledger.py            (  52) | commitTxns | Committed 0 txns, 0 are uncommitted
2017-08-28 15:32:51,039 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:51,044 | TRACE    | node.py              (1839) | processOrdered | Node6 got ordered requests from backup replica 1
2017-08-28 15:32:51,092 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.992 seconds
2017-08-28 15:32:56,978 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:32:56,984 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:32:56,992 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:32:56,992 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:32:56,992 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:32:56,993 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:32:59,613 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 113
2017-08-28 15:32:59,615 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:32:59,616 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:32:59,618 | DEBUG    | monitor.py           ( 335) | isMasterThroughputTooLow | Node6 master throughput is not measurable.
2017-08-28 15:32:59,618 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:32:59,627 | TRACE    | monitor.py           ( 378) | isMasterAvgReqLatencyTooHigh | Node6 found master had no record yet for FBQ5W7LxRp8bLeVNEafjpd
2017-08-28 15:32:59,627 | DEBUG    | node.py              (1976) | checkPerformance | Node6's master has higher performance than backups
2017-08-28 15:32:59,638 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 114 to run in 10 seconds
2017-08-28 15:33:05,919 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6C got 1 messages through listener
2017-08-28 15:33:05,926 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated 6XiqM9DwYDgFnmyQQBFr72 signature on  request 1503934385861338
2017-08-28 15:33:05,936 | TRACE    | node.py              (1370) | validateClientMsg | Node6C received CLIENT message: SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:05,937 | DEBUG    | node.py              (1416) | processClientInBox | Node6C processing b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' request SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:05,937 | DEBUG    | node.py              (1690) | processRequest | Node6 received client request: SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'} from b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:33:05,937 | TRACE    | propagator.py        ( 164) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:05,938 | DEBUG    | propagator.py        ( 148) | propagate | Node6 propagating 6XiqM9DwYDgFnmyQQBFr72 request 1503934385861338 from client b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:33:05,938 | DEBUG    | node.py              (2566) | send | Node6 sending message PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:05,938 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'} to its replicas since not finalised
2017-08-28 15:33:05,938 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934385861338,"op":"REQACK","identifier":"6XiqM9DwYDgFnmyQQBFr72"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:33:05,972 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:33:05,980 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:33:05,991 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:33:05,992 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:33:06,003 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:33:06,013 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:33:06,014 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:33:06,023 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:33:06,032 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:33:06,032 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw","reqId":1503934385861338,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"6XiqM9DwYDgFnmyQQBFr72"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:33:06,056 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 5 messages through listener
2017-08-28 15:33:06,060 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated 6XiqM9DwYDgFnmyQQBFr72 signature on propagate  request 1503934385861338
2017-08-28 15:33:06,073 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,073 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}, 'op': 'PROPAGATE'}, 'Node4')
2017-08-28 15:33:06,075 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,086 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated 6XiqM9DwYDgFnmyQQBFr72 signature on propagate  request 1503934385861338
2017-08-28 15:33:06,097 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,097 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}, 'op': 'PROPAGATE'}, 'Node2')
2017-08-28 15:33:06,103 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,105 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated 6XiqM9DwYDgFnmyQQBFr72 signature on propagate  request 1503934385861338
2017-08-28 15:33:06,116 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,117 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}, 'op': 'PROPAGATE'}, 'Node1')
2017-08-28 15:33:06,127 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,132 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated 6XiqM9DwYDgFnmyQQBFr72 signature on propagate  request 1503934385861338
2017-08-28 15:33:06,141 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,142 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}, 'op': 'PROPAGATE'}, 'Node3')
2017-08-28 15:33:06,150 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,152 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated 6XiqM9DwYDgFnmyQQBFr72 signature on propagate  request 1503934385861338
2017-08-28 15:33:06,164 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,171 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}, 'op': 'PROPAGATE'}, 'Node5')
2017-08-28 15:33:06,176 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,177 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,194 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:06,194 | DEBUG    | propagator.py        ( 213) | forward | Node6 forwarding request ('6XiqM9DwYDgFnmyQQBFr72', 1503934385861338) to 0 replicas
2017-08-28 15:33:06,195 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,210 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:06,211 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'} to its replicas since already forwarded
2017-08-28 15:33:06,221 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,222 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:06,233 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'} to its replicas since already forwarded
2017-08-28 15:33:06,240 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,240 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:06,252 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'} to its replicas since already forwarded
2017-08-28 15:33:06,252 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}}
2017-08-28 15:33:06,268 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}
2017-08-28 15:33:06,269 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'} to its replicas since already forwarded
2017-08-28 15:33:06,338 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.432 seconds
2017-08-28 15:33:06,384 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:33:06,396 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'reqIdr': [['6XiqM9DwYDgFnmyQQBFr72', 1503934385861338]], 'ppTime': 1503934386, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:33:06,397 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'discarded': 1, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'ppSeqNo': 9, 'instId': 1, 'reqIdr': [['6XiqM9DwYDgFnmyQQBFr72', 1503934385861338]], 'stateRootHash': None, 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node4')
2017-08-28 15:33:06,397 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'reqIdr': [['6XiqM9DwYDgFnmyQQBFr72', 1503934385861338]], 'ppTime': 1503934386, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:33:06,417 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:1 received PRE-PREPARE(2, 9) from Node4:1 at 250682.565783403
2017-08-28 15:33:06,428 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:1 Sending PREPARE(2, 9) at 250682.577578278
2017-08-28 15:33:06,429 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Prepare
2017-08-28 15:33:06,439 | TRACE    | replica.py           (2134) | send | Node6:1 sending PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,439 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,449 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:1 tracking batch for PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'reqIdr': [('6XiqM9DwYDgFnmyQQBFr72', 1503934385861338)], 'ppTime': 1503934386, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None} with state root None
2017-08-28 15:33:06,449 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:1 processed incoming PRE-PREPARE(2, 9)
2017-08-28 15:33:06,450 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:33:06,458 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'reqIdr': [['6XiqM9DwYDgFnmyQQBFr72', 1503934385861338]], 'ppTime': 1503934386, 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:33:06,467 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'reqIdr': [['6XiqM9DwYDgFnmyQQBFr72', 1503934385861338]], 'instId': 0, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'discarded': 0, 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'viewNo': 2, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node3')
2017-08-28 15:33:06,468 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'reqIdr': [['6XiqM9DwYDgFnmyQQBFr72', 1503934385861338]], 'ppTime': 1503934386, 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:33:06,485 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:06,501 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:0 received PRE-PREPARE(2, 9) from Node3:0 at 250682.649844996
2017-08-28 15:33:06,502 | DEBUG    | replica.py           (1034) | validate_pre_prepare | Node6:0 state root before processing PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'reqIdr': [('6XiqM9DwYDgFnmyQQBFr72', 1503934385861338)], 'ppTime': 1503934386, 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} is b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9', 67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ
2017-08-28 15:33:06,508 | WARNING  | replica.py           ( 655) | processReqDuringBatch | Node6:0 encountered exception UnauthorizedClientRequest('6XiqM9DwYDgFnmyQQBFr72 is not a steward so cannot update a node',) while processing Request: {'signature': '674RWKwdAoxi8aY6vuUifCByKbsXYcZ9ogBRAT4DSnve6k4RcRf6zngXCqZYEk1ULZRs5MBvehmGV5dvYxu17Qzw', 'reqId': 1503934385861338, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '6XiqM9DwYDgFnmyQQBFr72'}, will reject
2017-08-28 15:33:06,519 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:0 Sending PREPARE(2, 9) at 250682.667706289
2017-08-28 15:33:06,519 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Prepare
2017-08-28 15:33:06,528 | TRACE    | replica.py           (2134) | send | Node6:0 sending PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,528 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,532 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:0 tracking batch for PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'reqIdr': [('6XiqM9DwYDgFnmyQQBFr72', 1503934385861338)], 'ppTime': 1503934386, 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} with state root b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9'
2017-08-28 15:33:06,532 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:0 processed incoming PRE-PREPARE(2, 9)
2017-08-28 15:33:06,558 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node5
2017-08-28 15:33:06,559 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node5
2017-08-28 15:33:06,573 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node4
2017-08-28 15:33:06,573 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node4
2017-08-28 15:33:06,574 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node1
2017-08-28 15:33:06,576 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node1
2017-08-28 15:33:06,576 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node3
2017-08-28 15:33:06,576 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node3
2017-08-28 15:33:06,586 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node2
2017-08-28 15:33:06,587 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node2
2017-08-28 15:33:06,594 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934385861338,"reason":"client request invalid: UnauthorizedClientRequest(\'6XiqM9DwYDgFnmyQQBFr72 is not a steward so cannot update a node\',)","op":"REJECT","identifier":"6XiqM9DwYDgFnmyQQBFr72"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:33:06,605 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:06,606 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 2 messages through listener
2017-08-28 15:33:06,621 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,621 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'viewNo': 2, 'ppSeqNo': 9, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node3')
2017-08-28 15:33:06,622 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,631 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,636 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 9, 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node4')
2017-08-28 15:33:06,637 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,653 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node5
2017-08-28 15:33:06,663 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node5
2017-08-28 15:33:06,664 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node4
2017-08-28 15:33:06,664 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node4
2017-08-28 15:33:06,679 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node1
2017-08-28 15:33:06,679 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node1
2017-08-28 15:33:06,691 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node3
2017-08-28 15:33:06,692 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node3
2017-08-28 15:33:06,692 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node2
2017-08-28 15:33:06,692 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934386,"viewNo":2,"ppSeqNo":9}' to Node2
2017-08-28 15:33:06,710 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 9) from Node4:0
2017-08-28 15:33:06,727 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,727 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,728 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 9) from Node3:1
2017-08-28 15:33:06,728 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,737 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,738 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:33:06,750 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: BATCH{'signature': None, 'messages': ['{"op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","instId":0,"viewNo":2,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","ppSeqNo":9,"ppTime":1503934386,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d"}', '{"op":"PREPARE","stateRootHash":null,"instId":1,"viewNo":2,"txnRootHash":null,"ppSeqNo":9,"ppTime":1503934386,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d"}']}
2017-08-28 15:33:06,762 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","instId":0,"viewNo":2,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","ppSeqNo":9,"ppTime":1503934386,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d"}', '{"op":"PREPARE","stateRootHash":null,"instId":1,"viewNo":2,"txnRootHash":null,"ppSeqNo":9,"ppTime":1503934386,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d"}'], 'op': 'BATCH'}, 'Node2')
2017-08-28 15:33:06,762 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","instId":0,"viewNo":2,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","ppSeqNo":9,"ppTime":1503934386,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d"}', '{"op":"PREPARE","stateRootHash":null,"instId":1,"viewNo":2,"txnRootHash":null,"ppSeqNo":9,"ppTime":1503934386,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d"}']}
2017-08-28 15:33:06,771 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,772 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppSeqNo': 9, 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:33:06,772 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,772 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,781 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppSeqNo': 9, 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:33:06,781 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,782 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"instId":0,"ppTime":1503934386,"ppSeqNo":9,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934386,"ppSeqNo":9,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}']}
2017-08-28 15:33:06,791 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"ppTime":1503934386,"ppSeqNo":9,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934386,"ppSeqNo":9,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:33:06,792 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"ppTime":1503934386,"ppSeqNo":9,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","viewNo":2,"op":"PREPARE","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ"}', '{"instId":1,"ppTime":1503934386,"ppSeqNo":9,"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","stateRootHash":null,"viewNo":2,"op":"PREPARE","txnRootHash":null}']}
2017-08-28 15:33:06,802 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,802 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'op': 'PREPARE', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ppSeqNo': 9}, 'Node5')
2017-08-28 15:33:06,802 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,803 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,813 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'op': 'PREPARE', 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'ppSeqNo': 9}, 'Node5')
2017-08-28 15:33:06,814 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,814 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: BATCH{'signature': None, 'messages': ['{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","ppTime":1503934386,"ppSeqNo":9,"instId":0,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","viewNo":2}', '{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","op":"PREPARE","stateRootHash":null,"ppTime":1503934386,"ppSeqNo":9,"instId":1,"txnRootHash":null,"viewNo":2}']}
2017-08-28 15:33:06,825 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","ppTime":1503934386,"ppSeqNo":9,"instId":0,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","viewNo":2}', '{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","op":"PREPARE","stateRootHash":null,"ppTime":1503934386,"ppSeqNo":9,"instId":1,"txnRootHash":null,"viewNo":2}'], 'op': 'BATCH'}, 'Node1')
2017-08-28 15:33:06,830 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","op":"PREPARE","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","ppTime":1503934386,"ppSeqNo":9,"instId":0,"txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","viewNo":2}', '{"digest":"01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d","op":"PREPARE","stateRootHash":null,"ppTime":1503934386,"ppSeqNo":9,"instId":1,"txnRootHash":null,"viewNo":2}']}
2017-08-28 15:33:06,830 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,830 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppTime': 1503934386, 'instId': 0, 'ppSeqNo': 9, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:33:06,830 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,831 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,831 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'stateRootHash': None, 'txnRootHash': None, 'ppTime': 1503934386, 'instId': 1, 'ppSeqNo': 9, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:33:06,831 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,873 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 9) from Node2:0
2017-08-28 15:33:06,882 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,882 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,889 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 9) from Node5:0
2017-08-28 15:33:06,889 | DEBUG    | replica.py           ( 953) | doCommit | Node6:0 Sending COMMIT(2, 9) at 250683.038410596
2017-08-28 15:33:06,890 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Commit
2017-08-28 15:33:06,890 | TRACE    | replica.py           (2134) | send | Node6:0 sending COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:06,899 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:06,900 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,900 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 9) from Node1:0
2017-08-28 15:33:06,901 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 9, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:06,910 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,910 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 9) from Node2:1
2017-08-28 15:33:06,911 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,911 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,922 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 9) from Node5:1
2017-08-28 15:33:06,922 | DEBUG    | replica.py           ( 953) | doCommit | Node6:1 Sending COMMIT(2, 9) at 250683.071493404
2017-08-28 15:33:06,923 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Commit
2017-08-28 15:33:06,923 | TRACE    | replica.py           (2134) | send | Node6:1 sending COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:06,923 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:06,932 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,932 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 9) from Node1:1
2017-08-28 15:33:06,932 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': '01c43924d5705be935bda848f1a02d1ab151e9721aa0975e09db012d6637d84d', 'ppTime': 1503934386, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 9, 'stateRootHash': None}
2017-08-28 15:33:06,935 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 9)
2017-08-28 15:33:06,968 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:06,968 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:06,992 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node5 into one transmission
2017-08-28 15:33:06,992 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}'])
2017-08-28 15:33:07,001 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node5: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:33:07,001 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node5
2017-08-28 15:33:07,002 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node4 into one transmission
2017-08-28 15:33:07,013 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}'])
2017-08-28 15:33:07,013 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node4: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:33:07,014 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node4
2017-08-28 15:33:07,014 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node1 into one transmission
2017-08-28 15:33:07,026 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}'])
2017-08-28 15:33:07,026 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node1: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:33:07,033 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node1
2017-08-28 15:33:07,033 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node3 into one transmission
2017-08-28 15:33:07,033 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}'])
2017-08-28 15:33:07,043 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node3: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:33:07,044 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node3
2017-08-28 15:33:07,044 | DEBUG    | batched.py           (  99) | flushOutBoxes | Node6 batching 2 msgs to Node2 into one transmission
2017-08-28 15:33:07,051 | TRACE    | batched.py           ( 100) | flushOutBoxes |     messages: deque([b'{"instId":0,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}', b'{"instId":1,"viewNo":2,"ppSeqNo":9,"op":"COMMIT"}'])
2017-08-28 15:33:07,052 | TRACE    | batched.py           ( 108) | flushOutBoxes | Node6 sending payload to Node2: b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}'
2017-08-28 15:33:07,052 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"signature":null,"messages":["{\\"instId\\":0,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}","{\\"instId\\":1,\\"viewNo\\":2,\\"ppSeqNo\\":9,\\"op\\":\\"COMMIT\\"}"],"op":"BATCH"}' to Node2
2017-08-28 15:33:07,062 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 5 messages through listener
2017-08-28 15:33:07,074 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: BATCH{'signature': None, 'messages': ['{"ppSeqNo":9,"instId":0,"viewNo":2,"op":"COMMIT"}', '{"ppSeqNo":9,"instId":1,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:33:07,082 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"ppSeqNo":9,"instId":0,"viewNo":2,"op":"COMMIT"}', '{"ppSeqNo":9,"instId":1,"viewNo":2,"op":"COMMIT"}'], 'op': 'BATCH'}, 'Node4')
2017-08-28 15:33:07,082 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"ppSeqNo":9,"instId":0,"viewNo":2,"op":"COMMIT"}', '{"ppSeqNo":9,"instId":1,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:33:07,083 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,083 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:33:07,092 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,092 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,092 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:33:07,093 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,103 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: BATCH{'signature': None, 'messages': ['{"op":"COMMIT","viewNo":2,"ppSeqNo":9,"instId":0}', '{"op":"COMMIT","viewNo":2,"ppSeqNo":9,"instId":1}']}
2017-08-28 15:33:07,105 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"op":"COMMIT","viewNo":2,"ppSeqNo":9,"instId":0}', '{"op":"COMMIT","viewNo":2,"ppSeqNo":9,"instId":1}'], 'op': 'BATCH'}, 'Node3')
2017-08-28 15:33:07,105 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"op":"COMMIT","viewNo":2,"ppSeqNo":9,"instId":0}', '{"op":"COMMIT","viewNo":2,"ppSeqNo":9,"instId":1}']}
2017-08-28 15:33:07,106 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,106 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 0, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:33:07,114 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,114 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,121 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 1, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:33:07,121 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,121 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":9,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":9,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:33:07,122 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"ppSeqNo":9,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":9,"viewNo":2,"op":"COMMIT"}'], 'op': 'BATCH'}, 'Node5')
2017-08-28 15:33:07,133 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"ppSeqNo":9,"viewNo":2,"op":"COMMIT"}', '{"instId":1,"ppSeqNo":9,"viewNo":2,"op":"COMMIT"}']}
2017-08-28 15:33:07,137 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,137 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:33:07,137 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,137 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,138 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:33:07,146 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,146 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":9}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":9}']}
2017-08-28 15:33:07,147 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":9}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":9}'], 'op': 'BATCH'}, 'Node1')
2017-08-28 15:33:07,154 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":9}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":9}']}
2017-08-28 15:33:07,154 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,155 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:33:07,155 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,162 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,163 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:33:07,163 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,164 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":9}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":9}']}
2017-08-28 15:33:07,174 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":9}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":9}'], 'op': 'BATCH'}, 'Node2')
2017-08-28 15:33:07,175 | DEBUG    | node.py              (1254) | unpackNodeMsg | Node6 processing a batch BATCH{'signature': None, 'messages': ['{"instId":0,"op":"COMMIT","viewNo":2,"ppSeqNo":9}', '{"instId":1,"op":"COMMIT","viewNo":2,"ppSeqNo":9}']}
2017-08-28 15:33:07,175 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,181 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:33:07,182 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,182 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,193 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 9, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:33:07,194 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9}
2017-08-28 15:33:07,217 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 108
2017-08-28 15:33:07,217 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:33:07,222 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:33:07,222 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 115 to run in 60 seconds
2017-08-28 15:33:07,222 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 16
2017-08-28 15:33:07,223 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:33:07,223 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 17 to run in 60 seconds
2017-08-28 15:33:07,234 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 9) from Node4:0
2017-08-28 15:33:07,248 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:07,248 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,248 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 9) from Node3:0
2017-08-28 15:33:07,256 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:07,257 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,257 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 9) from Node5:0
2017-08-28 15:33:07,257 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:07,258 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,270 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 9) from Node1:0
2017-08-28 15:33:07,271 | TRACE    | replica.py           ( 925) | tryOrder | Node6:0 returning request to node
2017-08-28 15:33:07,271 | DEBUG    | replica.py           (1437) | doOrder | Node6:0 ordering COMMIT (2, 9)
2017-08-28 15:33:07,277 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:0 set last ordered as (2, 9)
2017-08-28 15:33:07,277 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Ordered
2017-08-28 15:33:07,277 | TRACE    | replica.py           (2134) | send | Node6:0 sending ORDERED{'viewNo': 2, 'ppSeqNo': 9, 'ppTime': 1503934386, 'instId': 0, 'reqIdr': [], 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:33:07,285 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:0 ordered request (2, 9)
2017-08-28 15:33:07,286 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,286 | TRACE    | message_processor.py (  29) | discard | Node6:0 discarding message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 9} because already ordered 3 phase message
2017-08-28 15:33:07,294 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 9) from Node4:1
2017-08-28 15:33:07,295 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:07,295 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,304 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 9) from Node3:1
2017-08-28 15:33:07,304 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:07,305 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,308 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 9) from Node5:1
2017-08-28 15:33:07,316 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9} commits where f is 1
2017-08-28 15:33:07,316 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,316 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 9) from Node1:1
2017-08-28 15:33:07,325 | TRACE    | replica.py           ( 925) | tryOrder | Node6:1 returning request to node
2017-08-28 15:33:07,325 | DEBUG    | replica.py           (1437) | doOrder | Node6:1 ordering COMMIT (2, 9)
2017-08-28 15:33:07,325 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 9)
2017-08-28 15:33:07,326 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Ordered
2017-08-28 15:33:07,335 | TRACE    | replica.py           (2134) | send | Node6:1 sending ORDERED{'viewNo': 2, 'ppSeqNo': 9, 'ppTime': 1503934386, 'instId': 1, 'reqIdr': [('6XiqM9DwYDgFnmyQQBFr72', 1503934385861338)], 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:33:07,335 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:1 ordered request (2, 9)
2017-08-28 15:33:07,336 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 9)
2017-08-28 15:33:07,336 | TRACE    | message_processor.py (  29) | discard | Node6:1 discarding message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 9} because already ordered 3 phase message
2017-08-28 15:33:07,357 | DEBUG    | node.py              (1828) | processOrdered | Node6 executing Ordered batch 2 9 of 0 requests
2017-08-28 15:33:07,371 | DEBUG    | ledger.py            (  52) | commitTxns | Committed 0 txns, 0 are uncommitted
2017-08-28 15:33:07,372 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:33:07,372 | TRACE    | node.py              (1839) | processOrdered | Node6 got ordered requests from backup replica 1
2017-08-28 15:33:07,420 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 1.037 seconds
2017-08-28 15:33:09,661 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 114
2017-08-28 15:33:09,665 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:33:09,666 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:33:09,666 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:33:09,666 | DEBUG    | throttler.py         (  31) | acquire | now: 250685.815644201, len(actionsLog): 0
2017-08-28 15:33:09,667 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 0
2017-08-28 15:33:09,673 | DEBUG    | throttler.py         (  39) | acquire | len(actionsLog) was 0, after append, len(actionsLog): 1
2017-08-28 15:33:09,673 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:33:09,674 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 3}
            ordered request durations: {0: 0, 1: 3.5333490909833927}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8490528172423086}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8490528172423086
            master throughput ratio: 0.0
2017-08-28 15:33:09,692 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:09,694 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:09,699 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:33:09,700 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:09,700 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 116 to run in 10 seconds
2017-08-28 15:33:09,717 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:09,719 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:09,723 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:09,735 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:09,738 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:09,738 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:09,741 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:09,741 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:09,742 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:09,742 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:09,784 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.134 seconds
2017-08-28 15:33:11,990 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:33:12,003 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:33:12,004 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:33:12,004 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:33:12,004 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:33:12,020 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:33:19,740 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 116
2017-08-28 15:33:19,740 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:33:19,740 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:33:19,753 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:33:19,753 | DEBUG    | throttler.py         (  31) | acquire | now: 250695.90259035, len(actionsLog): 1
2017-08-28 15:33:19,754 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 1
2017-08-28 15:33:19,755 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.1025421927520482, timePassed: 10.086946148978313
2017-08-28 15:33:19,757 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 2
2017-08-28 15:33:19,757 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:33:19,758 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 3}
            ordered request durations: {0: 0, 1: 3.5333490909833927}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8490528172423086}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8490528172423086
            master throughput ratio: 0.0
2017-08-28 15:33:19,777 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:19,778 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:19,779 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:33:19,781 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:19,782 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 117 to run in 10 seconds
2017-08-28 15:33:19,797 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:19,798 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:19,804 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:19,804 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:19,804 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:19,805 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:19,815 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:19,816 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:19,820 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:19,821 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:19,850 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.124 seconds
2017-08-28 15:33:26,923 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6C got 1 messages through listener
2017-08-28 15:33:26,943 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated Rhx2qwaeiGyhU9vn4ynHSS signature on  request 1503934406877168
2017-08-28 15:33:26,945 | TRACE    | node.py              (1370) | validateClientMsg | Node6C received CLIENT message: SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:26,953 | DEBUG    | node.py              (1416) | processClientInBox | Node6C processing b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' request SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:26,962 | DEBUG    | node.py              (1690) | processRequest | Node6 received client request: SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'} from b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:33:26,972 | TRACE    | propagator.py        ( 164) | createPropagate | Creating PROPAGATE for REQUEST SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:26,973 | DEBUG    | propagator.py        ( 148) | propagate | Node6 propagating Rhx2qwaeiGyhU9vn4ynHSS request 1503934406877168 from client b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR'
2017-08-28 15:33:26,979 | DEBUG    | node.py              (2566) | send | Node6 sending message PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:26,985 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'} to its replicas since not finalised
2017-08-28 15:33:26,993 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934406877168,"op":"REQACK","identifier":"Rhx2qwaeiGyhU9vn4ynHSS"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:33:27,002 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:33:27,011 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node5
2017-08-28 15:33:27,015 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:33:27,025 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node4
2017-08-28 15:33:27,031 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:33:27,039 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node1
2017-08-28 15:33:27,040 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:33:27,050 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node3
2017-08-28 15:33:27,051 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:33:27,061 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"senderClient":"9C6z!fc&gD7!wTtPlKQeS.=AY[vp\\/BMSXqDx@uRR","request":{"signature":"2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj","reqId":1503934406877168,"operation":{"type":"0","data":{"client_port":9702,"node_ip":"10.0.0.6","node_port":9701,"client_ip":"10.0.0.6","alias":"Node5","services":[]},"dest":"4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc"},"identifier":"Rhx2qwaeiGyhU9vn4ynHSS"},"op":"PROPAGATE"}' to Node2
2017-08-28 15:33:27,074 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:33:27,074 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:33:27,080 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:33:27,080 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:33:27,080 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:33:27,087 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:33:27,097 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 5 messages through listener
2017-08-28 15:33:27,106 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated Rhx2qwaeiGyhU9vn4ynHSS signature on propagate  request 1503934406877168
2017-08-28 15:33:27,117 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,121 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}, 'op': 'PROPAGATE'}, 'Node4')
2017-08-28 15:33:27,122 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,134 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated Rhx2qwaeiGyhU9vn4ynHSS signature on propagate  request 1503934406877168
2017-08-28 15:33:27,134 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,145 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}, 'op': 'PROPAGATE'}, 'Node3')
2017-08-28 15:33:27,145 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,147 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated Rhx2qwaeiGyhU9vn4ynHSS signature on propagate  request 1503934406877168
2017-08-28 15:33:27,159 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,159 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}, 'op': 'PROPAGATE'}, 'Node2')
2017-08-28 15:33:27,167 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,178 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated Rhx2qwaeiGyhU9vn4ynHSS signature on propagate  request 1503934406877168
2017-08-28 15:33:27,178 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,178 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}, 'op': 'PROPAGATE'}, 'Node1')
2017-08-28 15:33:27,188 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,197 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated Rhx2qwaeiGyhU9vn4ynHSS signature on propagate  request 1503934406877168
2017-08-28 15:33:27,205 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,213 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}, 'op': 'PROPAGATE'}, 'Node5')
2017-08-28 15:33:27,213 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,223 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,231 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:27,232 | DEBUG    | propagator.py        ( 213) | forward | Node6 forwarding request ('Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168) to 0 replicas
2017-08-28 15:33:27,240 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,250 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:27,251 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'} to its replicas since already forwarded
2017-08-28 15:33:27,259 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,260 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:27,270 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'services': [], 'client_ip': '10.0.0.6', 'alias': 'Node5', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'} to its replicas since already forwarded
2017-08-28 15:33:27,271 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,282 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:27,294 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'services': [], 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'node_port': 9701}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'} to its replicas since already forwarded
2017-08-28 15:33:27,305 | DEBUG    | node.py              (1738) | processPropagate | Node Node6 received propagated request: PROPAGATE{'senderClient': '9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR', 'request': {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}}
2017-08-28 15:33:27,306 | TRACE    | propagator.py        ( 141) | propagate | Node6 already propagated SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}
2017-08-28 15:33:27,307 | DEBUG    | propagator.py        ( 245) | tryForwarding | Node6 not forwarding request SafeRequest: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'} to its replicas since already forwarded
2017-08-28 15:33:27,377 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.474 seconds
2017-08-28 15:33:27,422 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:33:27,435 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'reqIdr': [['Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168]], 'ppTime': 1503934407, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:33:27,435 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'discarded': 1, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'ppSeqNo': 10, 'instId': 1, 'reqIdr': [['Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168]], 'stateRootHash': None, 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node4')
2017-08-28 15:33:27,444 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'reqIdr': [['Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168]], 'ppTime': 1503934407, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:33:27,447 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:1 received PRE-PREPARE(2, 10) from Node4:1 at 250703.59596593
2017-08-28 15:33:27,470 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:1 Sending PREPARE(2, 10) at 250703.61936828
2017-08-28 15:33:27,471 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Prepare
2017-08-28 15:33:27,471 | TRACE    | replica.py           (2134) | send | Node6:1 sending PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,477 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,477 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:1 tracking batch for PREPREPARE{'discarded': 1, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'reqIdr': [('Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168)], 'ppTime': 1503934407, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None} with state root None
2017-08-28 15:33:27,478 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:1 processed incoming PRE-PREPARE(2, 10)
2017-08-28 15:33:27,479 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:33:27,493 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'reqIdr': [['Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168]], 'ppTime': 1503934407, 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:33:27,494 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'reqIdr': [['Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168]], 'instId': 0, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'discarded': 0, 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'viewNo': 2, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'op': 'PREPREPARE'}, 'Node3')
2017-08-28 15:33:27,501 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'reqIdr': [['Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168]], 'ppTime': 1503934407, 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:33:27,522 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:27,532 | DEBUG    | replica.py           ( 821) | processPrePrepare | Node6:0 received PRE-PREPARE(2, 10) from Node3:0 at 250703.681569053
2017-08-28 15:33:27,533 | DEBUG    | replica.py           (1034) | validate_pre_prepare | Node6:0 state root before processing PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'reqIdr': [('Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168)], 'ppTime': 1503934407, 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} is b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9', 67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ
2017-08-28 15:33:27,544 | WARNING  | replica.py           ( 655) | processReqDuringBatch | Node6:0 encountered exception UnauthorizedClientRequest('Rhx2qwaeiGyhU9vn4ynHSS is not a steward so cannot update a node',) while processing Request: {'signature': '2XNEbhmLphnQFgkovKFqgpenPhZooc24XrcFxoneYGoMfuzePhDs4vcFjgPQX9bsuemfbP6kGSkLg3r94uDWzDRj', 'reqId': 1503934406877168, 'operation': {'type': '0', 'data': {'client_port': 9702, 'alias': 'Node5', 'node_port': 9701, 'client_ip': '10.0.0.6', 'node_ip': '10.0.0.6', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': 'Rhx2qwaeiGyhU9vn4ynHSS'}, will reject
2017-08-28 15:33:27,545 | DEBUG    | replica.py           ( 934) | doPrepare | Node6:0 Sending PREPARE(2, 10) at 250703.694162517
2017-08-28 15:33:27,552 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Prepare
2017-08-28 15:33:27,552 | TRACE    | replica.py           (2134) | send | Node6:0 sending PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,552 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,560 | DEBUG    | replica.py           ( 612) | trackBatches | Node6:0 tracking batch for PREPREPARE{'discarded': 0, 'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'reqIdr': [('Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168)], 'ppTime': 1503934407, 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'} with state root b'8\xb0\xff\x81\xf2\x91\xbc\xac]w\xf3g\xea{RB\x15[\xbf\xfa\xaf\xff#\xf4\x03W@S,\xf1.\xa9'
2017-08-28 15:33:27,561 | DEBUG    | replica.py           ( 840) | processPrePrepare | Node6:0 processed incoming PRE-PREPARE(2, 10)
2017-08-28 15:33:27,585 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node5
2017-08-28 15:33:27,586 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node5
2017-08-28 15:33:27,593 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node4
2017-08-28 15:33:27,593 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node4
2017-08-28 15:33:27,604 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node1
2017-08-28 15:33:27,605 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node1
2017-08-28 15:33:27,605 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node3
2017-08-28 15:33:27,617 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node3
2017-08-28 15:33:27,618 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node2
2017-08-28 15:33:27,618 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":null,"stateRootHash":null,"op":"PREPARE","instId":1,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node2
2017-08-28 15:33:27,636 | TRACE    | zstack.py            ( 752) | transmitThroughListener | Node6C transmitting b'{"reqId":1503934406877168,"reason":"client request invalid: UnauthorizedClientRequest(\'Rhx2qwaeiGyhU9vn4ynHSS is not a steward so cannot update a node\',)","op":"REJECT","identifier":"Rhx2qwaeiGyhU9vn4ynHSS"}' to b'9C6z!fc&gD7!wTtPlKQeS.=AY[vp/BMSXqDx@uRR' through listener socket
2017-08-28 15:33:27,647 | DEBUG    | node.py              (2566) | send | Node6 sending message PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:27,649 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 4 messages through listener
2017-08-28 15:33:27,666 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,670 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppSeqNo': 10, 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:33:27,671 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,679 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,681 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'viewNo': 2, 'ppSeqNo': 10, 'stateRootHash': None, 'op': 'PREPARE'}, 'Node3')
2017-08-28 15:33:27,681 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,682 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,695 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'op': 'PREPARE', 'txnRootHash': None, 'instId': 1, 'stateRootHash': None, 'ppSeqNo': 10}, 'Node5')
2017-08-28 15:33:27,695 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,702 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,702 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppSeqNo': 10, 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node4')
2017-08-28 15:33:27,703 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,748 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node5
2017-08-28 15:33:27,748 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node5
2017-08-28 15:33:27,758 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node4
2017-08-28 15:33:27,758 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node4
2017-08-28 15:33:27,771 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node1
2017-08-28 15:33:27,772 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node1
2017-08-28 15:33:27,772 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node3
2017-08-28 15:33:27,780 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node3
2017-08-28 15:33:27,780 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node2
2017-08-28 15:33:27,781 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"digest":"77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08","txnRootHash":"67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ","stateRootHash":"4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv","op":"PREPARE","instId":0,"ppTime":1503934407,"viewNo":2,"ppSeqNo":10}' to Node2
2017-08-28 15:33:27,794 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 10) from Node4:0
2017-08-28 15:33:27,810 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,811 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 10)
2017-08-28 15:33:27,823 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 10) from Node2:1
2017-08-28 15:33:27,823 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,824 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 10)
2017-08-28 15:33:27,824 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 10) from Node3:1
2017-08-28 15:33:27,832 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,833 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 10)
2017-08-28 15:33:27,833 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 10) from Node5:1
2017-08-28 15:33:27,833 | DEBUG    | replica.py           ( 953) | doCommit | Node6:1 Sending COMMIT(2, 10) at 250703.982469157
2017-08-28 15:33:27,842 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Commit
2017-08-28 15:33:27,843 | TRACE    | replica.py           (2134) | send | Node6:1 sending COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:27,851 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:27,851 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 10)
2017-08-28 15:33:27,852 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 3 messages through listener
2017-08-28 15:33:27,865 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,865 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'stateRootHash': None, 'txnRootHash': None, 'ppTime': 1503934407, 'instId': 1, 'ppSeqNo': 10, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:33:27,873 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,874 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,874 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppSeqNo': 10, 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'op': 'PREPARE'}, 'Node2')
2017-08-28 15:33:27,887 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,888 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,888 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'op': 'PREPARE', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ppSeqNo': 10}, 'Node5')
2017-08-28 15:33:27,895 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,922 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:27,923 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 10) from Node2:0
2017-08-28 15:33:27,929 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since does not have prepare quorum for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,930 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 10)
2017-08-28 15:33:27,937 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 10) from Node5:0
2017-08-28 15:33:27,938 | DEBUG    | replica.py           ( 953) | doCommit | Node6:0 Sending COMMIT(2, 10) at 250704.086769228
2017-08-28 15:33:27,938 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Commit
2017-08-28 15:33:27,945 | TRACE    | replica.py           (2134) | send | Node6:0 sending COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:27,945 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:27,945 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 10)
2017-08-28 15:33:27,951 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:1 received PREPARE(2, 10) from Node1:1
2017-08-28 15:33:27,952 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:1 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': None, 'instId': 1, 'ppSeqNo': 10, 'stateRootHash': None}
2017-08-28 15:33:27,952 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:1 processed incoming PREPARE (2, 10)
2017-08-28 15:33:27,953 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 2 messages through listener
2017-08-28 15:33:27,968 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,968 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'ppTime': 1503934407, 'instId': 0, 'ppSeqNo': 10, 'op': 'PREPARE'}, 'Node1')
2017-08-28 15:33:27,978 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:27,979 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:27,983 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 1, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:33:27,983 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:27,999 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node5
2017-08-28 15:33:28,005 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node5
2017-08-28 15:33:28,005 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node4
2017-08-28 15:33:28,014 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node4
2017-08-28 15:33:28,014 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node1
2017-08-28 15:33:28,015 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node1
2017-08-28 15:33:28,021 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node3
2017-08-28 15:33:28,021 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node3
2017-08-28 15:33:28,022 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node2
2017-08-28 15:33:28,031 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":1,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node2
2017-08-28 15:33:28,033 | DEBUG    | node.py              (2566) | send | Node6 sending message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:28,047 | DEBUG    | replica.py           ( 865) | processPrepare | Node6:0 received PREPARE(2, 10) from Node1:0
2017-08-28 15:33:28,048 | DEBUG    | replica.py           ( 917) | tryCommit | Node6:0 cannot send COMMIT since has already sent COMMIT for PREPARE{'viewNo': 2, 'digest': '77e502d87a5ac0d47523e40804f47a9bcca2626a6da34ef7cde961e4a7075d08', 'ppTime': 1503934407, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ', 'instId': 0, 'ppSeqNo': 10, 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv'}
2017-08-28 15:33:28,048 | DEBUG    | replica.py           ( 876) | processPrepare | Node6:0 processed incoming PREPARE (2, 10)
2017-08-28 15:33:28,054 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 10) from Node3:1
2017-08-28 15:33:28,054 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:28,054 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,066 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 4 messages through listener
2017-08-28 15:33:28,075 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,076 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:33:28,076 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,076 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,083 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:33:28,084 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,084 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,093 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:33:28,094 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,094 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node3: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,099 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'viewNo': 2, 'instId': 0, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node3')
2017-08-28 15:33:28,100 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,119 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node5
2017-08-28 15:33:28,122 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node5
2017-08-28 15:33:28,122 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node4
2017-08-28 15:33:28,123 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node4
2017-08-28 15:33:28,131 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node1
2017-08-28 15:33:28,132 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node1
2017-08-28 15:33:28,140 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node3
2017-08-28 15:33:28,140 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node3
2017-08-28 15:33:28,147 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node2
2017-08-28 15:33:28,148 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"instId":0,"viewNo":2,"ppSeqNo":10,"op":"COMMIT"}' to Node2
2017-08-28 15:33:28,149 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 10) from Node3:0
2017-08-28 15:33:28,165 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:28,165 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,165 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 10) from Node4:1
2017-08-28 15:33:28,166 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:28,174 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,175 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 10) from Node2:1
2017-08-28 15:33:28,175 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:1 cannot return request to node: no quorum (5): COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:28,175 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,183 | DEBUG    | replica.py           ( 894) | processCommit | Node6:1 received COMMIT(2, 10) from Node5:1
2017-08-28 15:33:28,189 | TRACE    | replica.py           ( 925) | tryOrder | Node6:1 returning request to node
2017-08-28 15:33:28,189 | DEBUG    | replica.py           (1437) | doOrder | Node6:1 ordering COMMIT (2, 10)
2017-08-28 15:33:28,189 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:1 set last ordered as (2, 10)
2017-08-28 15:33:28,190 | DEBUG    | replica.py           (2133) | send | Node6:1 sending Ordered
2017-08-28 15:33:28,198 | TRACE    | replica.py           (2134) | send | Node6:1 sending ORDERED{'viewNo': 2, 'ppSeqNo': 10, 'ppTime': 1503934407, 'instId': 1, 'reqIdr': [('Rhx2qwaeiGyhU9vn4ynHSS', 1503934406877168)], 'stateRootHash': None, 'ledgerId': 0, 'txnRootHash': None}
2017-08-28 15:33:28,199 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:1 ordered request (2, 10)
2017-08-28 15:33:28,206 | DEBUG    | replica.py           ( 905) | processCommit | Node6:1 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,218 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 4 messages through listener
2017-08-28 15:33:28,226 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node4: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,226 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node4')
2017-08-28 15:33:28,227 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,234 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,235 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 1, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:33:28,235 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,246 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node2: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,247 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node2')
2017-08-28 15:33:28,254 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,254 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node5: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,262 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node5')
2017-08-28 15:33:28,262 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,307 | TRACE    | node.py              (1839) | processOrdered | Node6 got ordered requests from backup replica 1
2017-08-28 15:33:28,331 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 10) from Node4:0
2017-08-28 15:33:28,342 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:28,343 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,343 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 10) from Node2:0
2017-08-28 15:33:28,361 | DEBUG    | replica.py           ( 929) | tryOrder | Node6:0 cannot return request to node: no quorum (5): COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10} commits where f is 1
2017-08-28 15:33:28,362 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,375 | DEBUG    | replica.py           ( 894) | processCommit | Node6:0 received COMMIT(2, 10) from Node5:0
2017-08-28 15:33:28,379 | TRACE    | replica.py           ( 925) | tryOrder | Node6:0 returning request to node
2017-08-28 15:33:28,388 | DEBUG    | replica.py           (1437) | doOrder | Node6:0 ordering COMMIT (2, 10)
2017-08-28 15:33:28,388 | DEBUG    | replica.py           ( 321) | last_ordered_3pc | Node6:0 set last ordered as (2, 10)
2017-08-28 15:33:28,389 | DEBUG    | replica.py           (2133) | send | Node6:0 sending Ordered
2017-08-28 15:33:28,394 | TRACE    | replica.py           (2134) | send | Node6:0 sending ORDERED{'viewNo': 2, 'ppSeqNo': 10, 'ppTime': 1503934407, 'instId': 0, 'reqIdr': [], 'stateRootHash': '4pJNQZ9fekfYgv7aw1YRNPURiZDfcp6AGE69J8osPxdv', 'ledgerId': 0, 'txnRootHash': '67d13kYEkmxc7hE3dBQ9mkkHKFZaZ2bBnUNt7nxQZ8yQ'}
2017-08-28 15:33:28,395 | DEBUG    | replica.py           (1469) | order_3pc_key | Node6:0 ordered request (2, 10)
2017-08-28 15:33:28,395 | DEBUG    | replica.py           ( 905) | processCommit | Node6:0 processed incoming COMMIT(2, 10)
2017-08-28 15:33:28,404 | TRACE    | message_processor.py (  29) | discard | Node6:1 discarding message COMMIT{'instId': 1, 'viewNo': 2, 'ppSeqNo': 10} because already ordered 3 phase message
2017-08-28 15:33:28,413 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6 got 1 messages through listener
2017-08-28 15:33:28,424 | DEBUG    | node.py              (1239) | validateNodeMsg | Node6 received node message from Node1: COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,435 | DEBUG    | node.py              (1201) | handleOneNodeMsg | Node6 msg validated ({'instId': 0, 'viewNo': 2, 'ppSeqNo': 10, 'op': 'COMMIT'}, 'Node1')
2017-08-28 15:33:28,435 | DEBUG    | node.py              (1268) | postToNodeInBox | Node6 appending to nodeInbox COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10}
2017-08-28 15:33:28,449 | DEBUG    | node.py              (1828) | processOrdered | Node6 executing Ordered batch 2 10 of 0 requests
2017-08-28 15:33:28,462 | DEBUG    | ledger.py            (  52) | commitTxns | Committed 0 txns, 0 are uncommitted
2017-08-28 15:33:28,479 | TRACE    | monitor.py           ( 361) | isMasterReqLatencyTooHigh | Node6 found master's latency to be lower than the threshold for all requests.
2017-08-28 15:33:28,481 | TRACE    | message_processor.py (  29) | discard | Node6:0 discarding message COMMIT{'instId': 0, 'viewNo': 2, 'ppSeqNo': 10} because already ordered 3 phase message
2017-08-28 15:33:28,516 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 1.097 seconds
2017-08-28 15:33:29,805 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 117
2017-08-28 15:33:29,814 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:33:29,815 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:33:29,822 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:33:29,823 | DEBUG    | throttler.py         (  31) | acquire | now: 250705.972537004, len(actionsLog): 2
2017-08-28 15:33:29,825 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 2
2017-08-28 15:33:29,827 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.2103418361512954, timePassed: 10.069946654024534
2017-08-28 15:33:29,842 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 3
2017-08-28 15:33:29,843 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:33:29,850 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:33:29,854 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:29,855 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:29,872 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:33:29,872 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:29,872 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 118 to run in 10 seconds
2017-08-28 15:33:29,876 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:29,888 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:29,888 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:29,888 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:29,899 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:29,900 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:29,900 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:29,907 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:29,907 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:29,915 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:29,951 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.148 seconds
2017-08-28 15:33:39,884 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 118
2017-08-28 15:33:39,885 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:33:39,903 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:33:39,904 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:33:39,904 | DEBUG    | throttler.py         (  31) | acquire | now: 250716.053230275, len(actionsLog): 3
2017-08-28 15:33:39,904 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 3
2017-08-28 15:33:39,905 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.323668485456566, timePassed: 10.08069327098201
2017-08-28 15:33:39,908 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 4
2017-08-28 15:33:39,909 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:33:39,909 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:33:39,916 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:39,917 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:39,933 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:33:39,933 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:39,934 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 119 to run in 10 seconds
2017-08-28 15:33:39,940 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:39,951 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:39,952 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:39,952 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:39,952 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:39,953 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:39,956 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:39,957 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:39,957 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:39,958 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:39,987 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.109 seconds
2017-08-28 15:33:42,081 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:33:42,082 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:33:42,083 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:33:42,090 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:33:42,090 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:33:42,100 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:33:49,934 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 119
2017-08-28 15:33:49,935 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:33:49,936 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:33:49,958 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:33:49,958 | DEBUG    | throttler.py         (  31) | acquire | now: 250726.107102447, len(actionsLog): 4
2017-08-28 15:33:49,958 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 4
2017-08-28 15:33:49,959 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.4428055163203397, timePassed: 10.053872171993135
2017-08-28 15:33:49,959 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 5
2017-08-28 15:33:49,979 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:33:49,980 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:33:49,981 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:33:49,981 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:49,982 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:33:49,982 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:33:49,982 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 120 to run in 10 seconds
2017-08-28 15:33:49,988 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:49,988 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:33:49,999 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:50,003 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:33:50,006 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:50,007 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:33:50,022 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:50,022 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:33:50,031 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:50,033 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:33:50,050 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.118 seconds
2017-08-28 15:33:57,091 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:33:57,091 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:33:57,099 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:33:57,100 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:33:57,103 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:33:57,114 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:33:59,993 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 120
2017-08-28 15:34:00,002 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:34:00,013 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:34:00,014 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:34:00,014 | DEBUG    | throttler.py         (  31) | acquire | now: 250736.163128876, len(actionsLog): 5
2017-08-28 15:34:00,017 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:34:00,018 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.056026429025223
2017-08-28 15:34:00,025 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:34:00,025 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:34:00,026 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:34:00,053 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:34:00,053 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:00,054 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:34:00,055 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:00,067 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 121 to run in 10 seconds
2017-08-28 15:34:00,073 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:00,078 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:00,078 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:00,078 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:00,081 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:00,096 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:00,096 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:00,096 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:00,097 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:00,097 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:00,144 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.153 seconds
2017-08-28 15:34:07,226 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 115
2017-08-28 15:34:07,228 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:34:07,233 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:34:07,233 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 122 to run in 60 seconds
2017-08-28 15:34:07,233 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 17
2017-08-28 15:34:07,235 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:34:07,239 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 18 to run in 60 seconds
2017-08-28 15:34:10,082 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 121
2017-08-28 15:34:10,084 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:34:10,087 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:34:10,092 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:34:10,093 | DEBUG    | throttler.py         (  31) | acquire | now: 250746.241818406, len(actionsLog): 6
2017-08-28 15:34:10,093 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:34:10,093 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.078689529997064
2017-08-28 15:34:10,093 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:34:10,094 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:34:10,094 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:34:10,111 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:34:10,116 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:10,116 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:34:10,116 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:10,116 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 123 to run in 10 seconds
2017-08-28 15:34:10,132 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:10,133 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:10,141 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:10,141 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:10,148 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:10,148 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:10,155 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:10,155 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:10,156 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:10,156 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:10,198 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.132 seconds
2017-08-28 15:34:12,093 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:34:12,093 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:34:12,094 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:34:12,095 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:34:12,097 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:34:12,098 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:34:20,130 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 123
2017-08-28 15:34:20,139 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:34:20,139 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:34:20,145 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:34:20,150 | DEBUG    | throttler.py         (  31) | acquire | now: 250756.299422657, len(actionsLog): 6
2017-08-28 15:34:20,151 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:34:20,151 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.057604250992881
2017-08-28 15:34:20,152 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:34:20,152 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:34:20,160 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:34:20,170 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:34:20,174 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:20,181 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:34:20,183 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:20,187 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 124 to run in 10 seconds
2017-08-28 15:34:20,188 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:20,189 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:20,203 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:20,204 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:20,207 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:20,207 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:20,211 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:20,212 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:20,213 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:20,227 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:20,256 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.128 seconds
2017-08-28 15:34:27,105 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:34:27,106 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:34:27,106 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:34:27,108 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:34:27,113 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:34:27,122 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:34:30,199 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 124
2017-08-28 15:34:30,200 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:34:30,200 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:34:30,201 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:34:30,201 | DEBUG    | throttler.py         (  31) | acquire | now: 250766.350224571, len(actionsLog): 6
2017-08-28 15:34:30,215 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:34:30,216 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.050801914010663
2017-08-28 15:34:30,216 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:34:30,216 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:34:30,222 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:34:30,230 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:34:30,231 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:30,231 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:34:30,245 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:30,245 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 125 to run in 10 seconds
2017-08-28 15:34:30,256 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:30,257 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:30,266 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:30,266 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:30,270 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:30,270 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:30,275 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:30,276 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:30,283 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:30,289 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:30,328 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.132 seconds
2017-08-28 15:34:40,264 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 125
2017-08-28 15:34:40,267 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:34:40,269 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:34:40,271 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:34:40,273 | DEBUG    | throttler.py         (  31) | acquire | now: 250776.422530448, len(actionsLog): 6
2017-08-28 15:34:40,274 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:34:40,274 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.072305876994506
2017-08-28 15:34:40,274 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:34:40,274 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:34:40,275 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:34:40,293 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:34:40,298 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:40,308 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:34:40,308 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:40,309 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 126 to run in 10 seconds
2017-08-28 15:34:40,315 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:40,316 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:40,324 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:40,324 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:40,325 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:40,335 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:40,338 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:40,338 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:40,346 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:40,347 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:40,367 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.115 seconds
2017-08-28 15:34:42,130 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:34:42,142 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:34:42,143 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:34:42,143 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:34:42,144 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:34:42,154 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:34:50,309 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 126
2017-08-28 15:34:50,309 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:34:50,309 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:34:50,310 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:34:50,311 | DEBUG    | throttler.py         (  31) | acquire | now: 250786.460311047, len(actionsLog): 6
2017-08-28 15:34:50,311 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:34:50,312 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.037780598999234
2017-08-28 15:34:50,312 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:34:50,312 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:34:50,327 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:34:50,332 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:34:50,333 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:50,333 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:34:50,348 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:34:50,351 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 127 to run in 10 seconds
2017-08-28 15:34:50,356 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:50,359 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:34:50,363 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:50,364 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:34:50,367 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:50,367 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:34:50,368 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:50,368 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:34:50,369 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:50,387 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:34:50,414 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.117 seconds
2017-08-28 15:34:57,141 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:34:57,152 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:34:57,152 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:34:57,152 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:34:57,154 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:34:57,156 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:35:00,361 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 127
2017-08-28 15:35:00,366 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:35:00,367 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:35:00,373 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:35:00,375 | DEBUG    | throttler.py         (  31) | acquire | now: 250796.523897, len(actionsLog): 6
2017-08-28 15:35:00,375 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:35:00,378 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.063585953001166
2017-08-28 15:35:00,379 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:35:00,382 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:35:00,383 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:35:00,401 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:35:00,403 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:00,404 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:35:00,409 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:00,409 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 128 to run in 10 seconds
2017-08-28 15:35:00,410 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:00,429 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:00,430 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:00,430 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:00,431 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:00,431 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:00,441 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:00,441 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:00,447 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:00,448 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:00,483 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.126 seconds
2017-08-28 15:35:07,244 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 122
2017-08-28 15:35:07,244 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:35:07,244 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:35:07,245 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 129 to run in 60 seconds
2017-08-28 15:35:07,250 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 18
2017-08-28 15:35:07,251 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:35:07,254 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 19 to run in 60 seconds
2017-08-28 15:35:10,421 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 128
2017-08-28 15:35:10,423 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:35:10,425 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:35:10,425 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:35:10,426 | DEBUG    | throttler.py         (  31) | acquire | now: 250806.574868221, len(actionsLog): 6
2017-08-28 15:35:10,439 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:35:10,440 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.050971221004147
2017-08-28 15:35:10,440 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:35:10,440 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:35:10,443 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:35:10,455 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:35:10,466 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:10,466 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:35:10,467 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:10,467 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 130 to run in 10 seconds
2017-08-28 15:35:10,470 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:10,478 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:10,478 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:10,479 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:10,490 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:10,491 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:10,491 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:10,497 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:10,497 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:10,498 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:10,536 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.120 seconds
2017-08-28 15:35:12,146 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:35:12,158 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:35:12,162 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:35:12,162 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:35:12,162 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:35:12,163 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:35:20,470 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 130
2017-08-28 15:35:20,485 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:35:20,485 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:35:20,485 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:35:20,487 | DEBUG    | throttler.py         (  31) | acquire | now: 250816.636288383, len(actionsLog): 6
2017-08-28 15:35:20,487 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:35:20,488 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.061420161975548
2017-08-28 15:35:20,489 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:35:20,491 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:35:20,506 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:35:20,512 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:35:20,512 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:20,514 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:35:20,514 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:20,514 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 131 to run in 10 seconds
2017-08-28 15:35:20,534 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:20,535 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:20,535 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:20,543 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:20,543 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:20,544 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:20,552 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:20,553 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:20,553 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:20,554 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:20,598 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.106 seconds
2017-08-28 15:35:27,163 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:35:27,170 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:35:27,171 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:35:27,171 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:35:27,171 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:35:27,189 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:35:30,535 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 131
2017-08-28 15:35:30,539 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:35:30,540 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:35:30,547 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:35:30,548 | DEBUG    | throttler.py         (  31) | acquire | now: 250826.696777835, len(actionsLog): 6
2017-08-28 15:35:30,548 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:35:30,548 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.06048945200746
2017-08-28 15:35:30,548 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:35:30,549 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:35:30,549 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:35:30,549 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:35:30,549 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:30,549 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:35:30,549 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:30,550 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 132 to run in 10 seconds
2017-08-28 15:35:30,577 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:30,577 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:30,593 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:30,595 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:30,596 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:30,597 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:30,597 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:30,597 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:30,611 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:30,612 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:30,648 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.121 seconds
2017-08-28 15:35:40,569 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 132
2017-08-28 15:35:40,576 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:35:40,576 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:35:40,576 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:35:40,576 | DEBUG    | throttler.py         (  31) | acquire | now: 250836.725603219, len(actionsLog): 6
2017-08-28 15:35:40,579 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:35:40,590 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.028825384011725
2017-08-28 15:35:40,591 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:35:40,591 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:35:40,594 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:35:40,594 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:35:40,603 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:40,616 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:35:40,616 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:40,617 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 133 to run in 10 seconds
2017-08-28 15:35:40,629 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:40,629 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:40,630 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:40,630 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:40,634 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:40,635 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:40,646 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:40,650 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:40,651 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:40,651 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:40,695 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.137 seconds
2017-08-28 15:35:42,173 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:35:42,180 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:35:42,181 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:35:42,181 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:35:42,182 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:35:42,183 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:35:50,630 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 133
2017-08-28 15:35:50,631 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:35:50,639 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:35:50,640 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:35:50,640 | DEBUG    | throttler.py         (  31) | acquire | now: 250846.789108727, len(actionsLog): 6
2017-08-28 15:35:50,640 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:35:50,641 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.063505508005619
2017-08-28 15:35:50,642 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:35:50,642 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:35:50,643 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:35:50,668 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:35:50,668 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:50,669 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:35:50,669 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:35:50,670 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 134 to run in 10 seconds
2017-08-28 15:35:50,691 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:50,693 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:35:50,693 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:50,698 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:35:50,698 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:50,699 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:35:50,703 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:50,705 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:35:50,711 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:50,712 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:35:50,734 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.106 seconds
2017-08-28 15:35:57,179 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:35:57,187 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:35:57,188 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:35:57,188 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:35:57,188 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:35:57,189 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:36:00,679 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 134
2017-08-28 15:36:00,694 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:36:00,695 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:36:00,695 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:36:00,698 | DEBUG    | throttler.py         (  31) | acquire | now: 250856.847407256, len(actionsLog): 6
2017-08-28 15:36:00,699 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:36:00,700 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.058298528980231
2017-08-28 15:36:00,700 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:36:00,714 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:36:00,719 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:36:00,720 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:36:00,723 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:00,725 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:36:00,725 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:00,728 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 135 to run in 10 seconds
2017-08-28 15:36:00,739 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:00,745 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:00,746 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:00,746 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:00,752 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:00,753 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:00,762 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:00,764 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:00,766 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:00,766 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:00,789 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.113 seconds
2017-08-28 15:36:07,250 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 129
2017-08-28 15:36:07,265 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:36:07,268 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:36:07,269 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 136 to run in 60 seconds
2017-08-28 15:36:07,269 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 19
2017-08-28 15:36:07,269 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:36:07,270 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 20 to run in 60 seconds
2017-08-28 15:36:10,742 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 135
2017-08-28 15:36:10,746 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:36:10,746 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:36:10,747 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:36:10,752 | DEBUG    | throttler.py         (  31) | acquire | now: 250866.900944793, len(actionsLog): 6
2017-08-28 15:36:10,752 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:36:10,752 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.053537536994554
2017-08-28 15:36:10,753 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:36:10,753 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:36:10,753 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:36:10,754 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:36:10,754 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:10,754 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:36:10,754 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:10,754 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 137 to run in 10 seconds
2017-08-28 15:36:10,778 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:10,779 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:10,780 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:10,780 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:10,781 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:10,803 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:10,803 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:10,809 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:10,810 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:10,810 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:10,843 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.103 seconds
2017-08-28 15:36:12,197 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:36:12,199 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:36:12,200 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:36:12,200 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:36:12,205 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:36:12,218 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:36:20,765 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 137
2017-08-28 15:36:20,774 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:36:20,774 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:36:20,774 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:36:20,775 | DEBUG    | throttler.py         (  31) | acquire | now: 250876.923794872, len(actionsLog): 6
2017-08-28 15:36:20,775 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:36:20,777 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.022850079025375
2017-08-28 15:36:20,783 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:36:20,784 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:36:20,784 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:36:20,801 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:36:20,802 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:20,802 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:36:20,804 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:20,805 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 138 to run in 10 seconds
2017-08-28 15:36:20,807 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:20,807 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:20,808 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:20,808 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:20,808 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:20,830 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:20,830 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:20,831 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:20,839 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:20,840 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:20,870 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.108 seconds
2017-08-28 15:36:27,212 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:36:27,216 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:36:27,217 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:36:27,219 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:36:27,219 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:36:27,223 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:36:30,816 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 138
2017-08-28 15:36:30,821 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:36:30,821 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:36:30,821 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:36:30,822 | DEBUG    | throttler.py         (  31) | acquire | now: 250886.970799376, len(actionsLog): 6
2017-08-28 15:36:30,822 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:36:30,823 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.047004503983771
2017-08-28 15:36:30,827 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:36:30,827 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:36:30,827 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:36:30,828 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:36:30,847 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:30,850 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:36:30,850 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:30,851 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 139 to run in 10 seconds
2017-08-28 15:36:30,856 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:30,856 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:30,871 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:30,875 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:30,879 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:30,880 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:30,880 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:30,880 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:30,888 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:30,889 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:30,913 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.119 seconds
2017-08-28 15:36:40,867 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 139
2017-08-28 15:36:40,873 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:36:40,873 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:36:40,874 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:36:40,875 | DEBUG    | throttler.py         (  31) | acquire | now: 250897.023961362, len(actionsLog): 6
2017-08-28 15:36:40,875 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:36:40,888 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.053161985997576
2017-08-28 15:36:40,888 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:36:40,888 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:36:40,889 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:36:40,896 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:36:40,898 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:40,901 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:36:40,902 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:40,903 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 140 to run in 10 seconds
2017-08-28 15:36:40,908 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:40,909 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:40,925 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:40,925 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:40,931 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:40,933 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:40,935 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:40,936 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:40,947 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:40,951 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:40,979 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.130 seconds
2017-08-28 15:36:42,240 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:36:42,241 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:36:42,241 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:36:42,241 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:36:42,242 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:36:42,244 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:36:50,920 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 140
2017-08-28 15:36:50,927 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:36:50,929 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:36:50,929 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:36:50,930 | DEBUG    | throttler.py         (  31) | acquire | now: 250907.078869078, len(actionsLog): 6
2017-08-28 15:36:50,930 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:36:50,930 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.054907715995796
2017-08-28 15:36:50,930 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:36:50,951 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:36:50,951 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:36:50,952 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:36:50,953 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:50,954 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:36:50,954 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:36:50,957 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 141 to run in 10 seconds
2017-08-28 15:36:50,976 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:50,976 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:36:50,978 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:50,979 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:36:50,990 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:50,991 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:36:50,991 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:50,999 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:36:51,005 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:51,006 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:36:51,037 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.135 seconds
2017-08-28 15:36:57,250 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:36:57,259 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:36:57,264 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:36:57,268 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:36:57,268 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:36:57,280 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:37:00,968 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 141
2017-08-28 15:37:00,968 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:37:00,968 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:37:00,969 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:37:00,975 | DEBUG    | throttler.py         (  31) | acquire | now: 250917.123795965, len(actionsLog): 6
2017-08-28 15:37:00,975 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:37:00,980 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.044926886999747
2017-08-28 15:37:00,980 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:37:00,987 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:37:00,988 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:37:00,988 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:37:00,988 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:00,989 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:37:00,989 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:00,989 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 142 to run in 10 seconds
2017-08-28 15:37:01,002 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:01,002 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:01,004 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:01,005 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:01,005 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:01,005 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:01,005 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:01,006 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:01,006 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:01,006 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:01,053 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.105 seconds
2017-08-28 15:37:07,273 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 136
2017-08-28 15:37:07,277 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:37:07,283 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:37:07,284 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 143 to run in 60 seconds
2017-08-28 15:37:07,284 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 20
2017-08-28 15:37:07,284 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:37:07,285 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 21 to run in 60 seconds
2017-08-28 15:37:11,030 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 142
2017-08-28 15:37:11,031 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:37:11,035 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:37:11,039 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:37:11,040 | DEBUG    | throttler.py         (  31) | acquire | now: 250927.188730977, len(actionsLog): 6
2017-08-28 15:37:11,040 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:37:11,042 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.064935012022033
2017-08-28 15:37:11,051 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:37:11,052 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:37:11,052 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:37:11,066 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:37:11,068 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:11,068 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:37:11,070 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:11,070 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 144 to run in 10 seconds
2017-08-28 15:37:11,083 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:11,088 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:11,088 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:11,088 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:11,098 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:11,099 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:11,099 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:11,100 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:11,100 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:11,106 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:11,145 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.126 seconds
2017-08-28 15:37:12,251 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:37:12,267 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:37:12,269 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:37:12,269 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:37:12,269 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:37:12,270 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:37:21,087 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 144
2017-08-28 15:37:21,091 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:37:21,095 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:37:21,095 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:37:21,099 | DEBUG    | throttler.py         (  31) | acquire | now: 250937.24775966, len(actionsLog): 6
2017-08-28 15:37:21,100 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:37:21,100 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.05902868299745
2017-08-28 15:37:21,100 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:37:21,101 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:37:21,101 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:37:21,123 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:37:21,124 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:21,124 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:37:21,127 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:21,127 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 145 to run in 10 seconds
2017-08-28 15:37:21,144 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:21,145 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:21,152 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:21,152 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:21,153 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:21,156 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:21,157 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:21,157 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:21,172 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:21,173 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:21,197 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.122 seconds
2017-08-28 15:37:27,271 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:37:27,282 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:37:27,282 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:37:27,283 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:37:27,295 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:37:27,296 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:37:31,142 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 145
2017-08-28 15:37:31,143 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:37:31,146 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:37:31,147 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:37:31,148 | DEBUG    | throttler.py         (  31) | acquire | now: 250947.297254868, len(actionsLog): 6
2017-08-28 15:37:31,152 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:37:31,152 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.049495207989821
2017-08-28 15:37:31,152 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:37:31,155 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:37:31,155 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:37:31,176 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:37:31,181 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:31,181 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:37:31,183 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:31,183 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 146 to run in 10 seconds
2017-08-28 15:37:31,205 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:31,206 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:31,206 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:31,212 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:31,212 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:31,213 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:31,213 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:31,224 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:31,225 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:31,225 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:31,261 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.122 seconds
2017-08-28 15:37:41,194 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 146
2017-08-28 15:37:41,196 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:37:41,196 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:37:41,197 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:37:41,197 | DEBUG    | throttler.py         (  31) | acquire | now: 250957.346146684, len(actionsLog): 6
2017-08-28 15:37:41,211 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:37:41,211 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.048891815997194
2017-08-28 15:37:41,222 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:37:41,222 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:37:41,223 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:37:41,226 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:37:41,228 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:41,228 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:37:41,228 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:41,229 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 147 to run in 10 seconds
2017-08-28 15:37:41,247 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:41,251 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:41,252 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:41,252 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:41,253 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:41,253 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:41,261 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:41,261 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:41,269 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:41,269 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:41,309 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.131 seconds
2017-08-28 15:37:42,300 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:37:42,300 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:37:42,300 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:37:42,301 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:37:42,301 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:37:42,318 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:37:51,262 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 147
2017-08-28 15:37:51,263 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:37:51,269 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:37:51,269 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:37:51,269 | DEBUG    | throttler.py         (  31) | acquire | now: 250967.418573039, len(actionsLog): 6
2017-08-28 15:37:51,270 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:37:51,271 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.072426355007337
2017-08-28 15:37:51,271 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:37:51,287 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:37:51,290 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:37:51,291 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:37:51,292 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:51,302 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:37:51,302 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:37:51,303 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 148 to run in 10 seconds
2017-08-28 15:37:51,320 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:51,321 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:37:51,321 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:51,325 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:37:51,325 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:51,333 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:37:51,334 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:51,334 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:37:51,345 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:51,346 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:37:51,373 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.123 seconds
2017-08-28 15:37:57,295 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:37:57,298 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:37:57,299 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:37:57,313 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:37:57,313 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:37:57,314 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:38:01,340 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 148
2017-08-28 15:38:01,342 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:38:01,342 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:38:01,343 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:38:01,343 | DEBUG    | throttler.py         (  31) | acquire | now: 250977.49240672, len(actionsLog): 6
2017-08-28 15:38:01,346 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:38:01,346 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.073833681002725
2017-08-28 15:38:01,346 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:38:01,347 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:38:01,347 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:38:01,368 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:38:01,369 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:01,371 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:38:01,372 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:01,374 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 149 to run in 10 seconds
2017-08-28 15:38:01,391 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:01,392 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:01,395 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:01,404 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:01,404 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:01,404 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:01,404 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:01,405 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:01,409 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:01,409 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:01,441 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.121 seconds
2017-08-28 15:38:07,303 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkNodeRequestSpike with id 143
2017-08-28 15:38:07,303 | DEBUG    | node.py              (1980) | checkNodeRequestSpike | Node6 checking its request amount
2017-08-28 15:38:07,306 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:38:07,307 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkNodeRequestSpike with id 150 to run in 60 seconds
2017-08-28 15:38:07,326 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 21
2017-08-28 15:38:07,327 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a ClusterThroughputSuspiciousSpike spike
2017-08-28 15:38:07,327 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 22 to run in 60 seconds
2017-08-28 15:38:11,394 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 149
2017-08-28 15:38:11,401 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:38:11,402 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:38:11,402 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:38:11,402 | DEBUG    | throttler.py         (  31) | acquire | now: 250987.551579192, len(actionsLog): 6
2017-08-28 15:38:11,403 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:38:11,403 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.059172471985221
2017-08-28 15:38:11,403 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:38:11,403 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:38:11,404 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:38:11,428 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:38:11,429 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:11,429 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:38:11,429 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:11,429 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 151 to run in 10 seconds
2017-08-28 15:38:11,440 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:11,453 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:11,454 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:11,455 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:11,455 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:11,459 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:11,466 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:11,478 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:11,479 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:11,479 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:11,509 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.118 seconds
2017-08-28 15:38:12,296 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:38:12,309 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:38:12,310 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:38:12,310 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:38:12,310 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:38:12,311 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:38:21,435 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 151
2017-08-28 15:38:21,438 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:38:21,438 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:38:21,439 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:38:21,439 | DEBUG    | throttler.py         (  31) | acquire | now: 250997.588064417, len(actionsLog): 6
2017-08-28 15:38:21,439 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:38:21,443 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.036485225020442
2017-08-28 15:38:21,444 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:38:21,444 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:38:21,459 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:38:21,464 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:38:21,466 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:21,469 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:38:21,470 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:21,471 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 152 to run in 10 seconds
2017-08-28 15:38:21,478 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:21,478 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:21,495 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:21,495 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:21,496 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:21,496 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:21,498 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:21,506 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:21,512 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:21,513 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:21,552 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.130 seconds
2017-08-28 15:38:27,323 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node1 HA(host='10.0.0.2', port=9701)
2017-08-28 15:38:27,323 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node5 HA(host='10.0.0.6', port=9701)
2017-08-28 15:38:27,324 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node3 HA(host='10.0.0.4', port=9705)
2017-08-28 15:38:27,324 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node2 HA(host='10.0.0.3', port=9703)
2017-08-28 15:38:27,324 | DEBUG    | kit_zstack.py        (  67) | reconcileNodeReg | Node6 matched remote Node4 HA(host='10.0.0.5', port=9707)
2017-08-28 15:38:27,331 | DEBUG    | kit_zstack.py        (  50) | maintainConnections | Node6 next check for retries in 15.00 seconds
2017-08-28 15:38:31,479 | TRACE    | has_action_queue.py  (  66) | _serviceActions | Node6 running action checkPerformance with id 152
2017-08-28 15:38:31,482 | TRACE    | node.py              (1959) | checkPerformance | Node6 checking its performance
2017-08-28 15:38:31,483 | DEBUG    | notifier_plugin_manager.py (  74) | sendMessageUponSuspiciousSpike | Not enough data to detect a NodeRequestSuspiciousSpike spike
2017-08-28 15:38:31,484 | INFO     | monitor.py           ( 341) | isMasterThroughputTooLow | MONITORING: Node6 master throughput ratio 0.0 is lower than Delta 0.4.
2017-08-28 15:38:31,484 | DEBUG    | throttler.py         (  31) | acquire | now: 251007.633174024, len(actionsLog): 6
2017-08-28 15:38:31,484 | DEBUG    | throttler.py         (  34) | acquire | after trim, len(actionsLog): 5
2017-08-28 15:38:31,484 | DEBUG    | throttler.py         (  44) | acquire | timeToWaitAfterPreviousTry: 3.568050833375483, timePassed: 10.045109606988262
2017-08-28 15:38:31,484 | DEBUG    | throttler.py         (  49) | acquire | timeToWaitAfterPreviousTry < timePassed was true, after append, len(actionsLog): 6
2017-08-28 15:38:31,485 | INFO     | node.py              (2029) | sendInstanceChange | VIEW CHANGE: Node6 sending an instance change with view_no 3 since Primary of master protocol instance degraded the performance
2017-08-28 15:38:31,485 | INFO     | node.py              (2032) | sendInstanceChange | MONITORING: Node6 metrics for monitor: Node6 Monitor metrics:: None
            Delta: 0.4
            Lambda: 60
            Omega: 5
            instances started: [249722.956898051, 249722.957477207]
            ordered request counts: {0: 0, 1: 4}
            ordered request durations: {0: 0, 1: 4.624991182965459}
            master request latencies: {}
            client avg request latencies: [{}, {'6XiqM9DwYDgFnmyQQBFr72': (1, 1.1802929400000721), 'Rhx2qwaeiGyhU9vn4ynHSS': (1, 1.0916420919820666), 'FBQ5W7LxRp8bLeVNEafjpd': (1, 1.1182073199888691), 'XhYtvJqezMUKfF6KVNaGmT': (1, 1.2348488309944514)}]
            throughput: {0: 0, 1: 0.8648665136341457}
            master throughput: 0
            total requests: 0
            avg backup throughput: 0.8648665136341457
            master throughput ratio: 0.0
2017-08-28 15:38:31,485 | DEBUG    | node.py              (2566) | send | Node6 sending message INSTANCE_CHANGE{'viewNo': 3, 'reason': 25} to all recipients: ['Node1', 'Node5', 'Node3', 'Node2', 'Node4']
2017-08-28 15:38:31,485 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:31,485 | INFO     | node.py              (1971) | checkPerformance | Node6 sent view change since performance degraded of master instance
2017-08-28 15:38:31,486 | DEBUG    | node.py              (1939) | do_view_change_if_possible | Node6 has no quorum for view 3
2017-08-28 15:38:31,486 | TRACE    | has_action_queue.py  (  36) | _schedule | Node6 scheduling action checkPerformance with id 153 to run in 10 seconds
2017-08-28 15:38:31,519 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:31,534 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node5
2017-08-28 15:38:31,534 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:31,535 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node4
2017-08-28 15:38:31,540 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:31,544 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node1
2017-08-28 15:38:31,544 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:31,545 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node3
2017-08-28 15:38:31,551 | DEBUG    | zstack.py            ( 721) | transmit | Node6 transmitting message b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:31,552 | TRACE    | batched.py           (  95) | flushOutBoxes | Node6 sending msg b'{"viewNo":3,"reason":25,"op":"INSTANCE_CHANGE"}' to Node2
2017-08-28 15:38:31,592 | WARNING  | base_events.py       (1308) | _run_once | Executing <Task pending coro=<Looper.runForever() running at /usr/local/lib/python3.5/dist-packages/stp_core/loop/looper.py:222> wait_for=<Future pending cb=[Task._wakeup()] created at /usr/lib/python3.5/asyncio/base_events.py:252> cb=[_run_until_complete_cb() at /usr/lib/python3.5/asyncio/base_events.py:164]> took 0.128 seconds
2017-08-28 15:38:36,171 | TRACE    | zstack.py            ( 477) | _receiveFromListener | Node6C got 1 messages through listener
2017-08-28 15:38:36,194 | DEBUG    | node.py              (2217) | verifySignature | Node6 authenticated 5qhSKMkpqui6taGLTecwqf signature on  request 1503934716120212
2017-08-28 15:38:36,215 | TRACE    | node.py              (1370) | validateClientMsg | Node6C received CLIENT message: SafeRequest: {'signature': '4FWtAeYRYzkrdr6Uopc6umZKdhaUZyWvD1o3i7UkyNdJsQk37cedA96NJ16dm1WKD45NgGy7KMpVm7teDMtio6Wb', 'reqId': 1503934716120212, 'operation': {'type': '0', 'data': {'client_port': 9702, 'node_ip': '10.0.0.6', 'node_port': 9701, 'client_ip': '10.0.0.6', 'alias': 'Node5', 'services': []}, 'dest': '4Tn3wZMNCvhSTXPcLinQDnHyj56DTLQtL61ki4jo2Loc'}, 'identifier': '5qhSKMkpqui6taGLTecwqf'}
2017-08-28 15:38:36,216 | DEBUG    | node.py              (1416) | processClientInBox | Node6C processing b'9C6z!fc&