[Blockchain 네트워크 구성] Peer-to-Peer 연결 오류 해결

참고 : [Blockchain 네트워크 구성] Hyperledger Fabric v1.2에서 Private Data 저장 및 조회

오류 발생

Hyperledger Fabric v1.2에서 Private Data 저장 테스트 진행 시, collections_config.json 파일에서 권한을 설정해주었음에도 불구하고 데이터가 공유되지 않는 문제가 발생하였습니다.

각 org별로 서로 다른 호스트에서 실행중이어서 그런지 org1에서 등록한 데이터는 org1에서만 조회가 가능하고 다른 org에서는 조회가 불가능했습니다.

peer0.org2 컨테이너 로그 확인

피어 컨테이너의 로그를 확인해보니 다음과 같은 오류가 출력되었습니다.

2018-07-11 01:50:30.293 UTC [gossip/discovery] func1 -> WARN 071 Could not connect to {peer0.org4.honeybee.com:7051 [] [] peer0.org4.honeybee.com:7051 <nil> <nil>} : context deadline exceeded
2018-07-11 01:50:30.294 UTC [gossip/gossip] func1 -> WARN 072 Deep probe of peer0.org3.honeybee.com:7051 failed: context deadline exceeded
github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).learnAnchorPeers.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:242
github.com/hyperledger/fabric/gossip/discovery.(*gossipDiscoveryImpl).Connect.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/discovery/discovery_impl.go:150
runtime.goexit
        /opt/go/src/runtime/asm_amd64.s:2361
2018-07-11 01:50:30.294 UTC [gossip/discovery] func1 -> WARN 073 Could not connect to {peer0.org3.honeybee.com:7051 [] [] peer0.org3.honeybee.com:7051 <nil> <nil>} : context deadline exceeded
2018-07-11 01:50:30.294 UTC [gossip/gossip] func1 -> WARN 074 Deep probe of peer0.org1.honeybee.com:7051 failed: context deadline exceeded
github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).learnAnchorPeers.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:242
github.com/hyperledger/fabric/gossip/discovery.(*gossipDiscoveryImpl).Connect.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/discovery/discovery_impl.go:150
runtime.goexit
        /opt/go/src/runtime/asm_amd64.s:2361
2018-07-11 01:50:30.295 UTC [gossip/discovery] func1 -> WARN 075 Could not connect to {peer0.org1.honeybee.com:7051 [] [] peer0.org1.honeybee.com:7051 <nil> <nil>} : context deadline exceeded
2018-07-11 01:50:33.285 UTC [gossip/gossip] learnAnchorPeers -> INFO 076 Anchor peer with same endpoint, skipping connecting to myself
2018-07-11 01:50:33.321 UTC [kvledger] CommitWithPvtData -> INFO 077 Channel [honeybeechannel]: Committed block [5] with 1 transaction(s)
2018-07-11 01:50:36.286 UTC [gossip/gossip] func1 -> WARN 078 Deep probe of peer0.org5.honeybee.com:7051 failed: context deadline exceeded
github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).learnAnchorPeers.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:242
github.com/hyperledger/fabric/gossip/discovery.(*gossipDiscoveryImpl).Connect.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/discovery/discovery_impl.go:150
runtime.goexit
        /opt/go/src/runtime/asm_amd64.s:2361
2018-07-11 01:50:36.286 UTC [gossip/discovery] func1 -> WARN 079 Could not connect to {peer0.org5.honeybee.com:7051 [] [] peer0.org5.honeybee.com:7051 <nil> <nil>} : context deadline exceeded
2018-07-11 01:50:36.286 UTC [gossip/gossip] func1 -> WARN 07a Deep probe of peer0.org3.honeybee.com:7051 failed: context deadline exceeded
github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).learnAnchorPeers.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:242
github.com/hyperledger/fabric/gossip/discovery.(*gossipDiscoveryImpl).Connect.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/discovery/discovery_impl.go:150
runtime.goexit
        /opt/go/src/runtime/asm_amd64.s:2361
2018-07-11 01:50:36.286 UTC [gossip/gossip] func1 -> WARN 07b Deep probe of peer0.org4.honeybee.com:7051 failed: context deadline exceeded
github.com/hyperledger/fabric/gossip/gossip.(*gossipServiceImpl).learnAnchorPeers.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/gossip/gossip_impl.go:242
github.com/hyperledger/fabric/gossip/discovery.(*gossipDiscoveryImpl).Connect.func1
        /opt/gopath/src/github.com/hyperledger/fabric/gossip/discovery/discovery_impl.go:150
runtime.goexit
        /opt/go/src/runtime/asm_amd64.s:2361

오류 원인 파악

로그 확인 결과, Private Data는 가십 프로토콜을 통해 피어 투 피어로 승인된 조직 구성원들에게 Private Data를 전달하는데, 피어 간의 연결이 되지 않아 가십 프로토콜이 제대로 동작하지 않은 것으로 추정됩니다.

Private Data에 대한 공식 문서 내용을 참조하세요.

• Private data
• Private Data

각 org별로 다른 호스트에서 실행되고 있는데, peer 컨테이너에는 orderer 컨테이너와만 extra_hosts 설정이 되어있어서 Peer-to-Peer 연결이 이루어지지 않고 있습니다.

오류 해결

extra_hosts 설정

다른 호스트에 있는 피어들과 통신을 하기 위해 모든 피어 컨테이너에서 extra_hosts 설정을 추가해주어야 합니다.

여기서 주의할 점은 자신의 컨테이너는 extra_hosts 설정을 하지 않아야 peer.chaincodeListenAddress가 정상적으로 동작할 수 있습니다. 예를 들어 peer0.org1의 컨테이너는 자신의 컨테이너, 즉 peer0.org1을 제외한 모든 peer 및 orderer 컨테이너를 extra_hosts로 설정합니다.

편의를 위해 honeybee_hl_byfn.sh 스크립트에서 cli_hosts 파일을 기반으로 peer0_hosts와 peer1_hosts 파일을 생성하게 됩니다. 이 파일을 각각의 peer0, peer1에서 /etc/hosts 파일과 볼륨으로 설정해줍니다.

환경변수 변경

이제 extra_hosts 설정을 통해 다른 호스트에 있는 피어들과 통신을 할 수 있게 되었습니다.

하지만 peer1 컨테이너에 대해서는 여전히 연결이 이루어지지 않는 오류가 발생합니다. 이는 peer1의 외부 포트가 7056으로 설정되어있기 때문입니다.

모든 org의 peer 컨테이너들이 같은 호스트 내에서 실행중일 경우에는, 링크 연결을 통해 peer 컨테이너의 내부 포트인 7051과 통신하는데에 문제가 없었지만 다른 호스트에서 실행중인 peer 컨테이너와 통신하기 위해서는 컨테이너 외부 포트와 연결을 해주어야 합니다.

이를 위해 peer1 컨테이너의 환경변수를 다음과 같이 변경해주어야 합니다.

CORE_PEER_ADDRESS=peer1.<org명>.<DOMAIN명>:7056
CORE_PEER_GOSSIP_EXTERNALENDPOINT=peer1.<org명>.<DOMAIN명>:7056
CORE_PEER_GOSSIP_BOOTSTRAP=peer0.<org명>.<DOMAIN명>:7051

각각의 환경변수에 대한 설명은 다음과 같습니다.

• CORE_PEER_GOSSIP_BOOTSTRAP : 앵커 피어의 포인트 또는 주소(앵커가 아닌 피어에 설정)
• CORE_PEER_ADDRESS : 패브릭 네트워크에서 이 특정 피어의 주소
• CORE_PEER_GOSSIP_EXTERNALENDPOINT : 이 피어의 모든 외부 통신을 위한 포인트 또는 주소.

peer0 컨테이너는 내부 포트와 외부 포트 모두 7051번을 사용하고 있기 때문에 따로 설정을 변경해주지 않아도 됩니다. 만약, peer0 컨테이너의 외부 포트가 7051번이 아니라면 peer0 컨테이너 또한 환경변수를 수정해주어야 합니다.

네트워크 재시작

수정한 설정들을 적용하기 위해 블록체인 네트워크를 다시 시작하고 peer 컨테이너의 로그를 확인합니다.

hyper@ubuntu:/opt/gopath/src/github.com/hyperledger$ docker container logs peer0.org1.honeybee.com -f
2018-07-11 04:25:49.800 UTC [msp] getMspConfig -> INFO 001 Loading NodeOUs
2018-07-11 04:25:49.943 UTC [nodeCmd] serve -> INFO 002 Starting peer:
 Version: 1.2.0
 Commit SHA: cae2ad4
 Go version: go1.10
 OS/Arch: linux/amd64
 Experimental features: false
 Chaincode:
  Base Image Version: 0.4.10
  Base Docker Namespace: hyperledger
  Base Docker Label: org.hyperledger.fabric
  Docker Namespace: hyperledger

2018-07-11 04:25:49.945 UTC [ledgermgmt] initialize -> INFO 003 Initializing ledger mgmt
2018-07-11 04:25:49.945 UTC [kvledger] NewProvider -> INFO 004 Initializing ledger provider
2018-07-11 04:25:50.028 UTC [couchdb] handleRequest -> WARN 005 Retrying couchdb request in 125ms. Attempt:1  Error:Get http://couchdb0.org1:5984/: dial tcp 172.18.0.6:5984: connect: connection refused
2018-07-11 04:25:50.155 UTC [couchdb] handleRequest -> WARN 006 Retrying couchdb request in 250ms. Attempt:2  Error:Get http://couchdb0.org1:5984/: dial tcp 172.18.0.6:5984: connect: connection refused
2018-07-11 04:25:50.406 UTC [couchdb] handleRequest -> WARN 007 Retrying couchdb request in 500ms. Attempt:3  Error:Get http://couchdb0.org1:5984/: dial tcp 172.18.0.6:5984: connect: connection refused
2018-07-11 04:25:50.910 UTC [couchdb] handleRequest -> WARN 008 Retrying couchdb request in 1s. Attempt:4  Error:Get http://couchdb0.org1:5984/: dial tcp 172.18.0.6:5984: connect: connection refused
2018-07-11 04:25:51.911 UTC [couchdb] handleRequest -> WARN 009 Retrying couchdb request in 2s. Attempt:5  Error:Get http://couchdb0.org1:5984/: dial tcp 172.18.0.6:5984: connect: connection refused
2018-07-11 04:25:53.918 UTC [couchdb] handleRequest -> WARN 00a Retrying couchdb request in 4s. Attempt:6  Error:Get http://couchdb0.org1:5984/: dial tcp 172.18.0.6:5984: connect: connection refused
2018-07-11 04:25:58.265 UTC [couchdb] CreateDatabaseIfNotExist -> INFO 00b Created state database _users
2018-07-11 04:25:58.318 UTC [couchdb] CreateDatabaseIfNotExist -> INFO 00c Created state database _replicator
2018-07-11 04:25:58.370 UTC [couchdb] CreateDatabaseIfNotExist -> INFO 00d Created state database _global_changes
2018-07-11 04:25:58.453 UTC [kvledger] NewProvider -> INFO 00e ledger provider Initialized
2018-07-11 04:25:58.454 UTC [ledgermgmt] initialize -> INFO 00f ledger mgmt initialized
2018-07-11 04:25:58.454 UTC [peer] func1 -> INFO 010 Auto-detected peer address: 172.18.0.10:7051
2018-07-11 04:25:58.454 UTC [peer] func1 -> INFO 011 Returning peer0.org1.honeybee.com:7051
2018-07-11 04:25:58.454 UTC [peer] func1 -> INFO 012 Auto-detected peer address: 172.18.0.10:7051
2018-07-11 04:25:58.454 UTC [peer] func1 -> INFO 013 Returning peer0.org1.honeybee.com:7051
2018-07-11 04:25:58.455 UTC [nodeCmd] serve -> INFO 014 Starting peer with TLS enabled
2018-07-11 04:25:58.457 UTC [eventhub_producer] start -> INFO 015 Event processor started
2018-07-11 04:25:58.457 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 016 Entering computeChaincodeEndpoint with peerHostname: peer0.org1.honeybee.com
2018-07-11 04:25:58.458 UTC [nodeCmd] computeChaincodeEndpoint -> INFO 017 Exit with ccEndpoint: peer0.org1.honeybee.com:7052
2018-07-11 04:25:58.460 UTC [nodeCmd] createChaincodeServer -> WARN 018 peer.chaincodeListenAddress is not set, using peer0.org1.honeybee.com:7052
2018-07-11 04:25:58.461 UTC [sccapi] registerSysCC -> INFO 019 system chaincode cscc(github.com/hyperledger/fabric/core/scc/cscc) registered
2018-07-11 04:25:58.462 UTC [sccapi] registerSysCC -> INFO 01a system chaincode lscc(github.com/hyperledger/fabric/core/scc/lscc) registered
2018-07-11 04:25:58.462 UTC [sccapi] registerSysCC -> INFO 01b system chaincode qscc(github.com/hyperledger/fabric/core/scc/qscc) registered
2018-07-11 04:25:58.466 UTC [gossip/gossip] NewGossipService -> INFO 01c Creating gossip service with self membership of {peer0.org1.honeybee.com:7051 [] [64 135 82 80 118 169 212 118 235 162 134 140 243 205 86 214 11 148 1 152 166 39 80 72 20 139 140 199 138 253 127 69] peer0.org1.honeybee.com:7051 <nil> <nil>}
2018-07-11 04:25:58.471 UTC [gossip/gossip] start -> INFO 01d Gossip instance peer0.org1.honeybee.com:7051 started
2018-07-11 04:25:58.487 UTC [cscc] Init -> INFO 01e Init CSCC
2018-07-11 04:25:58.487 UTC [sccapi] deploySysCC -> INFO 01f system chaincode cscc/(github.com/hyperledger/fabric/core/scc/cscc) deployed
2018-07-11 04:25:58.488 UTC [sccapi] deploySysCC -> INFO 020 system chaincode lscc/(github.com/hyperledger/fabric/core/scc/lscc) deployed
2018-07-11 04:25:58.488 UTC [qscc] Init -> INFO 021 Init QSCC
2018-07-11 04:25:58.488 UTC [sccapi] deploySysCC -> INFO 022 system chaincode qscc/(github.com/hyperledger/fabric/core/scc/qscc) deployed
2018-07-11 04:25:58.488 UTC [nodeCmd] serve -> INFO 023 Deployed system chaincodes
2018-07-11 04:25:58.489 UTC [discovery] NewService -> INFO 024 Created with config TLS: true, authCacheMaxSize: 1000, authCachePurgeRatio: 0.750000
2018-07-11 04:25:58.490 UTC [nodeCmd] registerDiscoveryService -> INFO 025 Discovery service activated
2018-07-11 04:25:58.490 UTC [nodeCmd] serve -> INFO 026 Starting peer with ID=[name:"peer0.org1.honeybee.com" ], network ID=[dev], address=[peer0.org1.honeybee.com:7051]
2018-07-11 04:25:58.490 UTC [nodeCmd] serve -> INFO 027 Started peer with ID=[name:"peer0.org1.honeybee.com" ], network ID=[dev], address=[peer0.org1.honeybee.com:7051]
2018-07-11 04:25:58.490 UTC [nodeCmd] func9 -> INFO 028 Starting profiling server with listenAddress = 0.0.0.0:6060
2018-07-11 04:26:03.477 UTC [gossip/gossip] func2 -> INFO 029 Learned of a new certificate: [10 7 111 114 103 49 77 83 80 18 174 6 45 45 45 45 45 66 69 71 73 78 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10 77 73 73 67 75 122 67 67 65 100 71 103 65 119 73 66 65 103 73 81 86 88 80 117 76 51 84 47 67 102 76 90 103 106 88 55 88 88 57 73 100 122 65 75 66 103 103 113 104 107 106 79 80 81 81 68 65 106 66 49 77 81 115 119 10 67 81 89 68 86 81 81 71 69 119 74 86 85 122 69 84 77 66 69 71 65 49 85 69 67 66 77 75 81 50 70 115 97 87 90 118 99 109 53 112 89 84 69 87 77 66 81 71 65 49 85 69 66 120 77 78 85 50 70 117 73 69 90 121 10 89 87 53 106 97 88 78 106 98 122 69 97 77 66 103 71 65 49 85 69 67 104 77 82 98 51 74 110 77 83 53 111 98 50 53 108 101 87 74 108 90 83 53 106 98 50 48 120 72 84 65 98 66 103 78 86 66 65 77 84 70 71 78 104 10 76 109 57 121 90 122 69 117 97 71 57 117 90 88 108 105 90 87 85 117 89 50 57 116 77 66 52 88 68 84 69 52 77 68 99 120 77 84 65 48 77 106 65 48 77 86 111 88 68 84 73 52 77 68 99 119 79 68 65 48 77 106 65 48 10 77 86 111 119 97 122 69 76 77 65 107 71 65 49 85 69 66 104 77 67 86 86 77 120 69 122 65 82 66 103 78 86 66 65 103 84 67 107 78 104 98 71 108 109 98 51 74 117 97 87 69 120 70 106 65 85 66 103 78 86 66 65 99 84 10 68 86 78 104 98 105 66 71 99 109 70 117 89 50 108 122 89 50 56 120 68 84 65 76 66 103 78 86 66 65 115 84 66 72 66 108 90 88 73 120 73 68 65 101 66 103 78 86 66 65 77 84 70 51 66 108 90 88 73 120 76 109 57 121 10 90 122 69 117 97 71 57 117 90 88 108 105 90 87 85 117 89 50 57 116 77 70 107 119 69 119 89 72 75 111 90 73 122 106 48 67 65 81 89 73 75 111 90 73 122 106 48 68 65 81 99 68 81 103 65 69 70 117 119 117 101 51 66 67 10 107 67 86 97 79 112 69 75 112 43 68 74 101 79 67 76 79 78 107 54 72 83 65 49 88 68 122 119 47 70 117 109 107 74 112 43 71 86 49 98 86 112 109 122 82 122 100 54 71 84 69 112 52 110 75 47 73 43 73 110 50 118 84 120 10 67 70 55 118 49 85 68 84 106 54 103 104 98 54 78 78 77 69 115 119 68 103 89 68 86 82 48 80 65 81 72 47 66 65 81 68 65 103 101 65 77 65 119 71 65 49 85 100 69 119 69 66 47 119 81 67 77 65 65 119 75 119 89 68 10 86 82 48 106 66 67 81 119 73 111 65 103 48 104 82 84 83 83 122 67 81 120 111 109 65 115 101 68 68 50 118 103 97 69 70 50 116 72 56 49 66 81 98 119 67 76 80 100 53 78 72 122 88 49 85 119 67 103 89 73 75 111 90 73 10 122 106 48 69 65 119 73 68 83 65 65 119 82 81 73 104 65 74 76 71 105 100 82 69 105 80 69 105 67 49 101 105 57 117 76 102 54 65 101 121 115 84 81 71 43 104 50 90 108 103 83 85 110 57 118 74 81 50 67 107 65 105 65 74 10 76 72 57 53 88 71 77 66 118 47 102 65 81 100 87 119 70 89 66 105 115 99 80 112 43 103 84 102 76 67 51 79 100 77 54 86 90 81 74 80 74 65 61 61 10 45 45 45 45 45 69 78 68 32 67 69 82 84 73 70 73 67 65 84 69 45 45 45 45 45 10]

모든 피어 컨테이너에서 위와 같이 로그가 출력되면 Peer-to-Peer 연결이 정상적으로 완료된 것입니다.

이제 Private Data를 문제 없이 사용할 수 있습니다!