Uploaded image for project: 'Fabric'
  1. Fabric
  2. FAB-16292

All peers crash after one peer is brought down for certain amount of time

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Highest
    • Resolution: Done
    • v1.4.2
    • v2.0.0, v1.4.3
    • fabric-gossip
    • None
    • (Please add steps to reproduce)

    Description

       

      This happens in 1.4.2 if you have peers from other organizations that go offline and as a result their messages get garbage collected and the peer tries to access the secret envelope which doesn't exist, and it panics as a result.

       

      I think it can be worked around by putting the aliveExpirationTimeout to a very high value, but I haven't checked.

       

       

         388	2019-08-13 10:06:37.841 UTC [gossip.discovery] expireDeadMembers -> WARN 070 Entering [d87ff79603d9a50f5788d0727dca141dcdb44f0625f97c712844d410e69645d8]
         389	2019-08-13 10:06:37.841 UTC [gossip.discovery] expireDeadMembers -> WARN 071 Closing connection to Endpoint: peer0.org1.sitech.com:7051, InternalEndpoint: , PKI-ID: d87ff79603d9a50f5788d0727dca141dcdb44f0625f97c712844d410e69645d8, Metadata: 
         390	2019-08-13 10:06:37.841 UTC [gossip.discovery] expireDeadMembers -> WARN 072 Exiting
         391	2019-08-13 10:06:42.690 UTC [gossip.channel] reportMembershipChanges -> INFO 073 Membership view has changed. peers went offline:  [[peer0.org1.sitech.com:7051 ]] , current view:  []
         392	panic: runtime error: invalid memory address or nil pointer dereference
         393	[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xaed020]
         394	
         395	goroutine 91 [running]:
         396	github.com/hyperledger/fabric/protos/gossip.(*SecretEnvelope).InternalEndpoint(0x0, 0x40c445, 0x7f7eb42e7390)
         397		/opt/gopath/src/github.com/hyperledger/fabric/protos/gossip/extensions.go:480 +0x40
         398	github.com/hyperledger/fabric/gossip/discovery.newAliveMsgStore.func4(0x1243280, 0xc003ccd360)
         399		/opt/gopath/src/github.com/hyperledger/fabric/gossip/discovery/discovery_impl.go:1032 +0xfd
         400	github.com/hyperledger/fabric/gossip/gossip/msgstore.(*messageStoreImpl).expireMessages(0xc000964280)
         401		/opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/msgstore/msgs.go:212 +0x34c
         402	github.com/hyperledger/fabric/gossip/gossip/msgstore.(*messageStoreImpl).expirationRoutine(0xc000964280)
         403		/opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/msgstore/msgs.go:253 +0x141
         404	created by github.com/hyperledger/fabric/gossip/gossip/msgstore.NewMessageStoreExpirable
         405		/opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/msgstore/msgs.go:53 +0xb2
      

      Attachments

        Issue Links

          Activity

            People

              yacovm Yacov Manevich
              guoger Jay Guo
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: