Details
-
Bug
-
Status: Complete
-
Highest
-
Resolution: Done
-
None
-
None
-
None
-
None
-
STN, running 1.3.55
-
18.07 Stability & Monitoring, EV 18.09 Stability-RocksDB, 18.08 Stability-Monitoring
Description
We lost the ability to post transactions to the STN ledger. Looking in the logs, a view change, cause unknown, occurred on all the nodes that I have access to at March 30, 02:42 GMT. The last transaction posted to the ledger is timestamped 1522356428, which is March 29, 20:47. I have access to 7 of 10 nodes, and their logs are attached.
At 17:16 GMT I restarted the indy-node service on those nodes and saw that in the view change the original primary was restored, and I was again able to post transactions.
Please analyze the attached logs, and determine insofar as is possible:
- What caused the view change?
- Why were we unable to post transactions?
- Following the restart, were all 10 of the nodes participating in consensus?
- The nodes that did not restart are NewtonD, RFCU, and ibm
Secondary questions:
- Were there attempts to post transactions to the ledger before the view change that failed?
- When did the fault state begin, where transactions were not posting?
- Did the view change cause the lack of consensus, and why would it?
Attachments
Issue Links
- relates to
-
INDY-1261 Provisional network has lost consensus
-
- Complete
-
-
INDY-1295 Reboot machine running Node leads to issues with consensus
-
- Complete
-
-
INDY-1197 Pool does not work after not simultaneous manual pool upgrade
-
- Complete
-
-
INDY-1179 Ambiguous behavior after node demotion
-
- Complete
-
-
INDY-1199 A node need to hook up to a lower viewChange
-
- Complete
-