14, INDY 17.21, INDY 17.22
When adding a new validator node to an existing pool the ledgers are unable to sync. Due to this issue the new node cannot participate in consensus and counts as a failed node.
The issue may be due to the use of the 'force=True' parameter in the POOL_UPGRADE transaction.
Setup pool with indy-node 1.0.28 using live pool configuration settings
Upgrade to 1.1.37
Add a new node to the pool from a fresh install of 1.1.37
1. Setup a pool using the provisional live build
2. Before starting the pool change the configuration to use the live transaction files. As the sovrin user edit ".sovrin/sovrin_config.py"
Add the following lines
3. Start the sovrin-node service. The ledgers for pool and domain will be created in the following directories
4. Send a few transactions from the CLI to make sure the pool is working correctly.
*Upgrade *to indy-node 1.1.37
Note- The upgrade to 1.1.37 introduced serialized ledgers. Due to this significant change it was necessary that all validator nodes in the pool upgraded simultaneously. The instructions to those upgrading the pool included the use of the upgrade parameter "force=True".
5. Send the an upgrade transaction with the parameter "force=True" like in the example below.
6. After upgrading successfully the pool version on each node showed
7. Send some transactions to make sure the pool is functioning. I sent 15 transactions.
Note - At this point the pool was functioning and all nodes are in sync
Add Node - Now install a new node to add to the pool
8. Install the latest stable (indy-node 1.1.37) to a new machine.
9. Initialize the node, but do not start the services
10. From one of the nodes in the pool copy the following files to the .sovrin directory of the new node
11. Before starting the node change the configuration file to use the live transaction files. As the sovrin user edit ".sovrin/sovrin_config.py"
Add the following lines
Note the difference here. After the upgrade the transactions_live file was renamed to "domain_transactions_live" and the format change to be in a json format
12. Now start the sovrin-node service
13. You can verify the ledger has data using the read_ledger tool as the sovrin user.
14. From the CLI add a new steward for this node
15. Using the CLI as the new Steward add the node with the send node transaction like below
16. You should see the node show up in the CLI as connected.
The domain ledger will not sync.
The other nodes show they are connected to the new node.
The logs in on the new node show the following error