I am opening the new issue because the issue I reported previously(https://jira.hyperledger.org/browse/FAB-15557) had wrong description.
This is what exactly happening:
When the peers are communicating via gossip, it uses pki-ids. If one of the peers re-joins the channel with new certificate(in case of the original certificate was compromised and revoked) at the same peer IP address, it joins with new pki-id.
However, the gossip membership still has the old, not expiring pki-id mapped with the peer IP address even though the peer joined with the new pki-id at the same ip address.
Because the gossip membership has both the old pki-id and new pki-id at the same ip address, the ip address mapped with the old pki-id is taken offline from the gossip membership.
Meanwhile, the peer with the new pki-id keeps sending requests to join the gossip membership and becomes online on the gossip membership temporarily.
These two contradictory offline/online action happens repeatedly until it drains all local ports available on the host machine. When it drains all the local ports, the peer gets isolated.
These are the exact steps to reproduce the issue I described above:
1. A channel has two orgs. Let's call them "Org1" and "Org2"
2. Org1 has two peers and Org2 has one peer.
3. All peers join the channel (with the anchor peers set)
4. Stop the peer on Org2
5. Re-start the peer with !Unable to render embedded object: File (NEW CERTIFICATE) not found.!. Must be restarted with the same ip address as before.
6. Re-join the channel. This will make the peer to join with the new pki-id.
7. If you check the logs of peers of Org1, you can see offline/online issue about the Org2 peer.
Thus, it should be possible to keep the peer with the new pki-id and same ip address online from the gossip membership, even if the ip address was already mapped with the old pki-id.