Change chaincode lifecycle management to be config tree based

Today's State

Chaincode lifecycle governs which chaincodes at which versions are executed by a peer in a channel, as well as their instantiation and endorsement policies.

The current approach to initial instantiation as well as chaincode upgrade, is for a party satisfying the chaincode instantiation policy to send a proposal to a peer with the updated version of the chaincode, which attempts to start the chaincode (if not started), invoke the Init function for that chaincode, and return the results of this invocation as part of the MVCC RW set of the proposal response.

In the commit path, although not all peers have access to the chaincode being instantiated or upgraded, the instantiation policy is available through the instantiate transaction (and may not be forged without corrupting the hash computations in the chaincode spec). The peers verify that this instantiation policy is satisfied and that the RW set does not maliciously manipulate the LSCC space.  Finally, the peers perform standard validation checks like MVCC validation.

This current behavior is undesirable for some use cases. In particular, depending on the instantiation policy as the sole gatekeeper for changes to a chaincode's version may be insufficient. Just because a chaincode is available on a peer does not necessarily imply that the channel wishes to have that chaincode utilized within the channel. Thus, the instantiation policy does a good job of protecting the chaincode from being instantiated in channels where it should not be, but additional control is needed in the other direction.

Because chaincode lifecycle is part of the transaction validation path, any modifications here will be non-backwards compatible and must therefore be gated behind FAB-5999.

New Lifecycle Proposal

The chaincode lifecycle is implemented as an endorser transaction, because it modifies the state database. However, all of the real validation of the instantiation is actually done in an ad-hoc psuedo-simulation style in the VSCC. Therefore, it makes sense to remove the endorsement step, and simply perform all validation during the commit phase.

The channel config already supports this style of zero-endorsement, validation-only transactions with multiple signatures, so it is a natural fit for managing chaincode lifecycle. However, unlike channel config operations which are processed by the orderer, chaincode lifecycle falls entirely within the scope of of the peer and should not require (nor is it desirable to require) the orderer's involvement for these operations.

Therefore, we propose a new configuration tree for the peer with the following structure

<Resources>
    <Chaincodes>
        <Chaincode1>
            (ID) : {
                CodeHash
                InstalledVersion
            }
            (EndorsementPolicy) : {
                PolicyReference
            }
        ...
        <ChaincodeN>
    <GlobalPolicies>
        |Policy1|
        ...
        |PolicyN|

Where `<xxx>` indicates a group of name `xxx`, `|xxx|` indicates a policy of name `xxx`, and `(xxx)` indicates a value of name `xxx`. Eventually, in the future, the tree may be enhanced to look like this:

<Resources>
    <Chaincodes>
        <Chaincode1>
            (ID) : {
                CodeHash
                InstalledVersion
            }
            (EndorsementPolicy) : {
                PolicyReferenceName
            }
            (InitializationParameters) : {
                PolicyReferenceName
            }
            (Status) : {
                Endorsing bool
                Committing bool
            }
            <Functions>
                (Function1) : {
                    InvocationPolicyReference
                    EndorsementPolicyReference
                }
                ...
                (FunctionN) : {
                    InvocationPolicyReference
                    EndorsementPolicyReference
                }
            <Collections>
                (Collection1) : {
                    ReadPolicyReference
                    EndorsementPolicyReference
                }
                ...
                (CollectionN) : {
                    ReadPolicyReference
                    EndorsementPolicyReference
                }
        ...
        <ChaincodeN>
    <API>
        (APIFunction1) : {
            InvocationPolicyReference
         }
        ...
        (APIFunctionN) : {
            InvocationPolicyReference
         }
    <GlobalPolicies>
        |Policy1|
        ...
        |PolicyN|

However, in the interest of proving out the lifecycle as managed by configuration type transactions quickly, these additional new features will most likely not be included into v1.1 of fabric.

Because this is a configuration tree, each element has an associated mod_policy which must be satisfied. The default for all of the mod_policy values will be /Channel/Application/Admins, requiring a majority of admins of the application network to approve modifying this config tree.

To define a new chaincode, the client will send a reconfiguration transaction which creates a node in the tree under the Chaincodes group which minimally defines the ID portion of the chaincode (its code hash and version), and its endorsement policy. The peer code will be modified to trigger chaincode lifecycle management internally when this tree is modified.

The client will need the peer to provide a new API to retrieve the resources config tree.  Once the current resources config tree has been retrieved, the client must compute the config update which performs the desired resource reconfiguration (such as deploying a chaincode). Potentially, this new API should also include the ability to simulate a configuration update so that a client may validate its update before submitting it to ordering.

Time permitting, the config tree should be stored in the state database as a flattened tree, with canonical paths for keys, and the config element as value. Time not permitting, a single blob may be stored in the config database, which is less useful for performing MVCC read locks, but, is easy/simple.