Prerequisites

Before you begin this tutorial, make sure you have the following:

IBM Cloud Private Cluster Version 3.1.0 or higher

Command line tools

  • Install Kubernetes CLI (kubectl)
  • Install Helm cli (helm)
  • Install IBM Cloud Private CLI (cloudctl)

Instructions on how to get these command line tools.

Before you begin, you might also want to read the deployment guide in the official docs for more considerations about network deployment as well as About IBM Blockchain Platform for IBM Cloud Private

Overview

In this tutorial, we’ll go over a basic fabric development configuration with two organizations: one for the orderer and one for a peer. We’ll follow the recommendation in the official docs, by deploying a separate CA for each organization. (Note: It is recommended to deploy one CA per organization. For example, if you deploy a peer that is associated with one organization and an orderer that is associated with another organization, you need to deploy two CAs. One CA is for the peer organization and the other one is for the orderer organization.)

When you finish this tutorial, you’ll have:

  • A running Hyperledger Fabric network
  • A peer that is in a consortium in the orderer’s system channel
  • A newly created application channel in the peer has instantiated chaincode
  • Invoked transactions by using the marbles chaincode

In the future, we can always add new peers to our organization and/or new organizations to our created network, which can have peers of their own.

Steps

  1. Deploy OrdererOrg CA
  2. Operate OrdererOrg CA
  3. Deploy Orderer
  4. Deploy PeerOrgCA
  5. Operate PeerOrgCA
  6. Deploy Peer
  7. Operate Orderer and Peer
  8. Operate Peer on new application channel

Helpful commands to keep in mind throughout this tutorial

Connect to a cluster via a client: cloudctl login -a https://$clusterip:8443 --skip-ssl-validation Note: Where clusterip = your IP (example: clusterip=9.12.19.43)

Make sure your context is set to the correct namespace
To view current-namespace, use:
kubectl config view --minify and look at namespace field.

If it’s not set to namespace, you want to deploy your components to follow the following instructions:

To change the namespace to a different namespace, use the following command by replacing bcaas-usa with your desired namespace:

kubectl config set-context $(kubectl config current-context) --namespace=bcaas-usa

Last 10 helm releases: helm ls -m 10 -dr --tls

Let’s get started

First, please set the directory you will use for this tutorial. (In my case $HOME/fabric-ca-client) and save this change to your ~/.bashrc so it gets set for new terminal windows.

  export ibp4icp_install_dir=$HOME/fabric-ca-client

  echo "export ibp4icp_install_dir=$ibp4icp_install_dir" >> ~/.bashrc

Deploy OrdererOrg CA

Note: See more information on deploying CA.

Create a CA Secret

  1. Set a user and password for your secret that you will use later with CA register:

    export ordererca_user=garrett
    export ordererca_password=secure_password
    
  2. Create the Secret:

    kubectl create secret generic garrett-orderer-ca --from-literal=ca-admin-name=$ordererca_user --from-literal=ca-admin-password=$ordererca_password
    

    You should see something like secret/garrett-ca created if your Secret was successfully created. Note: Special characters such as $, \, *, and ! require escaping. If the password you’re using has special characters, you need to escape them by using the \\\ character like this: \\\\!*

  3. Get the Proxy IP for CA deployment with:

    kubectl get nodes -l "proxy=true" -o jsonpath="{.items[0].status.addresses[0].address}"
    

    Here’s a sample output:

    9.12.19.115
    

    Note: In order to run this command, you need admin access to the cluster. If you don’t have admin access, please get someone with admin access to run this command for you.

    Here’s an example deployment of OrdererOrg CA: order-ca_deploy

  4. Show the release values:

    export release=garrett-orderer-ca

    helm get values $release --tls

    Below is a sample output of the above command:

    ca:
     name: OrdererCA
     enabled: true
     proxyIP: 9.12.19.115
     app:
       arch: s390x
     tlsca:
       name: orderer-tlsca
       cname: orderer-tlsca-common
     dataPVC:
       storageClassName: nfs-recycle
     ca:
       caAdminSecret: garrett-orderer-ca
    license: accept
    
  5. Set the environment variables with values from the helm chart:

    export NODE_IP=9.12.19.115
    
    export ord_caname=OrdererCA
    
    export ord_tlscaname=orderer-tlsca
    
  6. Look at the recently deployed helm release and its instructions:

    helm status $release --tls
    

    Here’s a sample output:

    LAST DEPLOYED: Wed Dec 12 12:52:38 2018
    NAMESPACE: bcaas-usa
    STATUS: DEPLOYED
    
    RESOURCES:
    ==> v1/Pod(related)
    NAME                                                      READY  STATUS   RESTARTS  AGE
    garrett-orderer-ca-fabric-ca-deployment-6d9bb7fd65-8lc4p  1/1    Running  0         29m
    
    ==> v1/ConfigMap
    NAME                             DATA  AGE
    garrett-orderer-ca-tlsca-config  1     29m
    garrett-orderer-ca-ca-configmap  2     29m
    
    ==> v1/PersistentVolumeClaim
    NAME                              STATUS  VOLUME                                    CAPACITY  ACCESS MODES  STORAGECLASS  AGE
    garrett-orderer-ca-fabric-ca-pvc  Bound   pvc-dd0d0a76-fe4f-11e8-bcdf-525400442dbc  2Gi       RWX           nfs-recycle   29m
    
    ==> v1/Service
    NAME                   TYPE      CLUSTER-IP     EXTERNAL-IP  PORT(S)         AGE
    garrett-orderer-ca-ca  NodePort  192.168.1.120  <none>       7054:31491/TCP  29m
    
    ==> v1/Deployment
    NAME                                     DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
    garrett-orderer-ca-fabric-ca-deployment  1        1        1           1          29m
    
    NOTES:
    1. Get the application URL by running these commands:
     export SERVICE_NAME=$(kubectl get services --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-orderer-ca" -o jsonpath="{.items[0].metadata.name}")
     export NODE_PORT=$(kubectl get --namespace bcaas-usa -o jsonpath="{.spec.ports[0].nodePort}" services $SERVICE_NAME)
     export NODE_IP=$(kubectl get nodes --namespace bcaas-usa -l "proxy=true" -o jsonpath="{.items[0].status.addresses[0].address}")
     echo http://$NODE_IP:$NODE_PORT
    2. Get the endpoint URL by running these commands:
     export APISERVER=$(kubectl config view --minify | grep server | cut -f 2- -d ":" | tr -d " ")
     export POD_NAME=$(kubectl get pods --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-orderer-ca" -o jsonpath="{.items[0].metadata.name}")
     export NAMESPACE=bcaas-usa
     export SERVICE_NAME=$(kubectl get services --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-orderer-ca" -o jsonpath="{.items[0].metadata.name}")
     echo $APISERVER/api/v1/namespaces/$NAMESPACE/services/$SERVICE_NAME:7054/proxy
    3. Get the TLS certificate:
    export POD_NAME=$(kubectl get pods --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-orderer-ca" -o jsonpath="{.items[0].metadata.name}")
    kubectl exec $POD_NAME -- cat /etc/hyperledger/fabric-ca-server/ca-cert.pem > tls.pem && cat tls.pem | base64
    4. Enroll the bootstrap admin identity:
     Enroll with TLS CA:
     fabric-ca-client enroll -d -u http://$CA_ADMIN:$CA_PASSWORD@$SERVICE_NAME:7054 --tls.certfiles tls-cert.pem --caname tlsca --enrollment.profile tls
     Enroll with default CA:
     fabric-ca-client enroll -d -u http://$CA_ADMIN:$CA_PASSWORD@$SERVICE_NAME:7054 --tls.certfiles tls-cert.pem
    
  7. Before moving on, check that your ordererCA deployment is available with:

    kubectl get deploy $release-fabric-ca-deployment
    

    Note: It’s normal for this to take a few minutes to become available. If it doesn’t become available after a while, check the logs in Kibana or with kubectl logs.

Operate OrdererOrg CA

Note: See more information on operating CA.

  1. Set the OrdererCA environment variables

    Note: These instructions are available in the helm chart release notes in step 2. In our case, they are modified to use our set environment variables.

    export SERVICE_NAME=$(kubectl get services -l "app=ibm-ibp, release=$release" -o jsonpath="{.items[0].metadata.name}")
    
    export NODE_PORT=$(kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services $SERVICE_NAME)
    
    export ORDCAURL=$NODE_IP:$NODE_PORT
    
    echo http://$ORDCAURL
    
    export POD_NAME=$(kubectl get pods -l "app=ibm-ibp, release=$release" -o jsonpath="{.items[0].metadata.name}")
    
  2. Download and set up fabric-ca-client binaries:

    mkdir $ibp4icp_install_dir
    
    cd $ibp4icp_install_dir
    
    mkdir fabric-binaries
    cd fabric-binaries
    
    curl -sSL https://raw.githubusercontent.com/hyperledger/fabric/master/scripts/bootstrap.sh | bash -s 1.2.1 1.2.1 -d -s
    
    export PATH=$ibp4icp_install_dir/fabric-binaries/bin:$PATH
    
    echo "export PATH=$ibp4icp_install_dir/fabric-binaries/bin:\$PATH" >> ~/.bashrc
    
  3. Set the path where you want to store admin certs for ordererca:

    mkdir $ibp4icp_install_dir/ordca-admin
    export FABRIC_CA_CLIENT_HOME=$ibp4icp_install_dir/ordca-admin
    
  4. Get certs and make base configuration file for deploying components with CA (orderer and peer)

    export ORDCACERT=$ibp4icp_install_dir/ordcatls/tls.pem
    
    kubectl cp $POD_NAME:/etc/hyperledger/fabric-ca-server/ca-cert.pem $ORDCACERT
    

    Note: tar: removing leading '/' from member names is a normal message with exit code 0.

  5. Make certificates by using the deployed CA:

    fabric-ca-client enroll -u https://$ordererca_user:$ordererca_password@$ORDCAURL --caname $ord_caname --tls.certfiles $ORDCACERT
    
    mkdir $ibp4icp_install_dir/connection
    
    cd $ibp4icp_install_dir/connection
    
  6. Set the base of your file by copying this code into your connection.json file:

    {
       "enrollment": {
           "component": {
               "cahost": "",
               "caport": "",
               "caname": "",
               "catls": {
                   "cacert": ""
               },
               "enrollid": "",
               "enrollsecret": "",
               "admincerts": [""]
           },
           "tls": {
               "cahost": "",
               "caport": "",
               "caname": "",
               "catls": {
                   "cacert": ""
               },
               "enrollid": "",
               "enrollsecret": "",
               "csr": {
                   "hosts": [""]
               }
           }
       }
    }
    
  7. Use the connection template for orderer.json:

    cp connection.json orderer.json
    

    An example of creating the connection.json and copying it to orderer.json will look like this: Connection json creation

  8. Get the base values for the file:

    echo $NODE_IP
    

    Here’s a sample output:

    9.12.19.115
    

    The output goes in enrollment.component.cahost and in enrollment.tls.cahost, where enrollment.component.cahost is the field inside of the component section of the enrollment section of the orderer.json file we’re making.

    Here’s the IP example: OrdererCA ip

    echo $NODE_PORT
    

    Here’s a sample output:

    31491
    

    This output goes in enrollment.component.caport and in enrollment.tls.caport.

    Here’s the port example: OrdererCA port

    echo $ord_caname
    

    Here’s a sample output:

    OrdererCA
    

    This goes in enrollment.component.caname.

    Here’s the CA name example: OrdererCA name

    echo $ord_tlscaname
    

    Here’s a sample output:

    orderer-tlsca
    

    Here’s the TLS CA name example: OrdererTLS CA name

    This goes in enrollment.tls.caname.

    Note: For Linux, the following command needs to have -w 0 (i.e., cat $ORDCACERT | base64 -w 0).

    cat $ORDCACERT | base64
    

    Here’s a sample output:

    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNGekNDQWIyZ0F3SUJBZ0lVUFZHL0RwaTlnaGs3WFFPMFVlcCtXNThwRHpRd0NnWUlLb1pJemowRUF3SXcKYURFTE1Ba0dBMVVFQmhNQ1ZWTXhGekFWQmdOVkJBZ1REazV2Y25Sb0lFTmhjbTlzYVc1aE1SUXdFZ1lEVlFRSwpFd3RJZVhCbGNteGxaR2RsY2pFUE1BMEdBMVVFQ3hNR1JtRmljbWxqTVJrd0Z3WURWUVFERXhCbVlXSnlhV010ClkyRXRjMlZ5ZG1WeU1CNFhEVEU0TVRJd05UQXhNall3TUZvWERUTXpNVEl3TVRBeE1qWXdNRm93YURFTE1Ba0cKQTFVRUJoTUNWVk14RnpBVkJnTlZCQWdURGs1dmNuUm9JRU5oY205c2FXNWhNUlF3RWdZRFZRUUtFd3RJZVhCbApjbXhsWkdkbGNqRVBNQTBHQTFVRUN4TUdSbUZpY21sak1Sa3dGd1lEVlFRREV4Qm1ZV0p5YVdNdFkyRXRjMlZ5CmRtVnlNRmt3RXdZSEtvWkl6ajBDQVFZSUtvWkl6ajBEQVFjRFFnQUVFVGFBZ1RCb0k3T01BY3VjUnVtR09KZUsKb1BrYnhxdzlCMDBLMmR1U0VCdHVIZG5kQjNBTFp4ZXJsNnlIMkNPOFFlTWc5TU9SaExDZDZYM1p4YVAzZEtORgpNRU13RGdZRFZSMFBBUUgvQkFRREFnRUdNQklHQTFVZEV3RUIvd1FJTUFZQkFmOENBUUV3SFFZRFZSME9CQllFCkZLWTNrNnl6RmJiK2pFYlVGNjhQWTZWZDREMTVNQW9HQ0NxR1NNNDlCQU1DQTBnQU1FVUNJUUNIbFNGWHVLQzQKMlI5ei9OSWJ6THFrYzRBSWc3TGZUZ1VQaW1aVzVwMG5od0lnY3ZzekRDRy9GRUNSeFNHSzk2UGdPMmpNdVhZdAprYWdiRERwczE4WnBadW89Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    

    This goes in enrollment.component.catls.cacert andenrollment.tls.catls.cacert.

    Here’s the CA cert example: OrdererTLS CA cert

  9. Create an affiliation for OrdererOrg:

    fabric-ca-client affiliation add OrdererOrg --caname $ord_caname --tls.certfiles $ORDCACERT

    fabric-ca-client affiliation list --caname $ord_caname --tls.certfiles $ORDCACERT

    Here’s a sample output:

    affiliation: .
     affiliation: org2
         affiliation: org2.department1
     affiliation: org1
         affiliation: org1.department2
         affiliation: org1.department1
      affiliation: OrdererOrg
    
  10. Register the Orderer with CA:

    Use the newly created affiliation from the above fabric-ca-client affiliation list command.

    fabric-ca-client register --caname $ord_caname --id.affiliation OrdererOrg --id.type orderer --id.name orderer1 --id.secret orderer1_pwd --tls.certfiles $ORDCACERT
    

    Here’s a sample output:

    2018/12/12 15:38:38 [INFO] Configuration file location: /Users/garrettwoodworth/fabric-ca-client/ordca-admin/fabric-ca-client-config.yaml
    2018/12/12 15:38:38 [INFO] TLS Enabled
    2018/12/12 15:38:38 [INFO] TLS Enabled
    Password: orderer1_pwd
    

    Note: The id.name value in the above register command (i.e., orderer1) goes in enrollment.component.enrollid in the orderer.json file. The id.secret value in the above register command (i.e., orderer1_pwd) goes in enrollment.component.enrollsecret in the orderer.json file.

    See the CA Enrollid/pwd example: OrdererCA admin/secret

  11. Make the admin MSP:

    fabric-ca-client register --caname $ord_caname --id.name OrdererOrgadmin --id.affiliation OrdererOrg --id.type user --id.secret secure_password --tls.certfiles $ORDCACERT
    
    fabric-ca-client enroll -u https://OrdererOrgadmin:secure_password@$ORDCAURL --caname $ord_caname -M $ibp4icp_install_dir/orderer-admin/msp --tls.certfiles $ORDCACERT
    
  12. Add Cert to file:

    Note: For Linux, the following command needs to have -w 0 (i.e., cat $ibp4icp_install_dir/orderer-admin/msp/signcerts/cert.pem | base64 -w 0).

    cat $ibp4icp_install_dir/orderer-admin/msp/signcerts/cert.pem | base64
    
    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUMxekNDQW4yZ0F3SUJBZ0lVRnd5OWRUdDVyS0J1c2h4Ui9vbTF3RHZnOU9Bd0NnWUlLb1pJemowRUF3SXcKYURFTE1Ba0dBMVVFQmhNQ1ZWTXhGekFWQmdOVkJBZ1REazV2Y25Sb0lFTmhjbTlzYVc1aE1SUXdFZ1lEVlFRSwpFd3RJZVhCbGNteGxaR2RsY2pFUE1BMEdBMVVFQ3hNR1JtRmljbWxqTVJrd0Z3WURWUVFERXhCbVlXSnlhV010ClkyRXRjMlZ5ZG1WeU1CNFhEVEU0TVRJeE1qSXpOVFl3TUZvWERURTVNVEl4TXpBd01ERXdNRm93ZURFTE1Ba0cKQTFVRUJoTUNWVk14RnpBVkJnTlZCQWdURGs1dmNuUm9JRU5oY205c2FXNWhNUlF3RWdZRFZRUUtFd3RJZVhCbApjbXhsWkdkbGNqRWdNQXNHQTFVRUN4TUVkWE5sY2pBUkJnTlZCQXNUQ2s5eVpHVnlaWEpQY21jeEdEQVdCZ05WCkJBTVREMDl5WkdWeVpYSlBjbWRoWkcxcGJqQlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VIQTBJQUJQb0kKdTBITXp0ZFJXVlV4RTlCUmJpZW5ySjhicFduYmNlU3hpaGk0blZRWDlFVWo2ZWFUaU1HVlVwaUFQSll0SWlscwpodHZ4RDF4d1kxb1Vndm0vMHJpamdmUXdnZkV3RGdZRFZSMFBBUUgvQkFRREFnZUFNQXdHQTFVZEV3RUIvd1FDCk1BQXdIUVlEVlIwT0JCWUVGT2p0RG9adFNkZGdYN0l1S0g2VmV4NTZKWkhvTUI4R0ExVWRJd1FZTUJhQUZNSFAKZ3p4UCtWeTVBTnVSaXpQYUlxS1dmZ3BjTUNVR0ExVWRFUVFlTUJ5Q0drZGhjbkpsZEhSekxVMWhZMEp2YjJzdApVSEp2TG14dlkyRnNNR29HQ0NvREJBVUdCd2dCQkY1N0ltRjBkSEp6SWpwN0ltaG1Ma0ZtWm1sc2FXRjBhVzl1Cklqb2lUM0prWlhKbGNrOXlaeUlzSW1obUxrVnVjbTlzYkcxbGJuUkpSQ0k2SWs5eVpHVnlaWEpQY21kaFpHMXAKYmlJc0ltaG1MbFI1Y0dVaU9pSjFjMlZ5SW4xOU1Bb0dDQ3FHU000OUJBTUNBMGdBTUVVQ0lRQ1M3OE9KVk1pYwoyTnl1VTIyVmVqZnFZdkFTcU1KVnNrQWxrK00vc1hZbTdRSWdjeDA2YU5QTUlIcHA2Rk02T2pBTCtEdjJoZXFNCjZlbTdSOXJ3Yk1qUkFjWT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    

    Note: This goes in enrollment.component.admincert in the orderer.json file.

    Here’s an Orderer Admincert example: Admincert

  13. Set up the TLS CA admin:

    mkdir $ibp4icp_install_dir/ordtlsca-admin
    
    export FABRIC_CA_CLIENT_HOME=$ibp4icp_install_dir/ordtlsca-admin
    
    fabric-ca-client enroll -u https://$ordererca_user:$ordererca_password@$ORDCAURL --caname $ord_tlscaname --tls.certfiles $ORDCACERT
    
  14. Create the affiliation for OrdererOrg:

    fabric-ca-client affiliation add OrdererOrg --caname $ord_tlscaname --tls.certfiles $ORDCACERT
    
    fabric-ca-client affiliation list --caname $ord_tlscaname --tls.certfiles $ORDCACERT
    

    Here’s a sample output:

    affiliation: .
      affiliation: org2
         affiliation: org2.department1
      affiliation: org1
         affiliation: org1.department2
         affiliation: org1.department1
      affiliation: OrdererOrg
    
  15. Register Orderer with TLS CA:

    fabric-ca-client register --caname $ord_tlscaname --id.name orderer1tls --id.affiliation OrdererOrg --id.type orderer --id.secret orderer1tls_pw --tls.certfiles $ORDCACERT
    

    Note: The id.name value in the above register command (i.e., orderer1tls) goes in enrollment.tls.enrollid in the orderer.json file. The id.secret value in the above register command (i.e., orderer1tls_pw) goes in enrollment.tls.enrollsecret in the orderer.json file.

    Here is an Orderer TLS Admin example: OrdererTLSCA admin

  16. Finish the Configuration File with a csr.hosts entry.

    Decide your orderer release name, as this will be the value you enter in the helm chart for deployment.

    export orderer_release=garrett-ord1
    

    Note: The value you enter in the csr.hosts will add -orderer to the desired release name.

    echo $orderer_release-orderer
    
    echo $NODE_IP
    

    Here’s an example entering csr.hosts for the orderer: csr.hosts orderer

  17. Create the orderer secret based on configuration file:

    cd $ibp4icp_install_dir/connection
    
    kubectl create secret generic garrett-ord1 --from-file=secret.json=orderer.json
    

    Checkpoint: Ensure that your current directory structure looks like the one below before continuing to Orderer Deployment.

    cd $ibp4icp_install_dir
    
    tree
    .
    ├── connection
    │   ├── connection.json
    │   └── orderer.json
    ├── fabric-binaries
    │   ├── bin
    │   │   ├── configtxgen
    │   │   ├── configtxlator
    │   │   ├── cryptogen
    │   │   ├── discover
    │   │   ├── fabric-ca-client
    │   │   ├── get-docker-images.sh
    │   │   ├── idemixgen
    │   │   ├── orderer
    │   │   └── peer
    │   └── config
    │       ├── configtx.yaml
    │       ├── core.yaml
    │       └── orderer.yaml
    ├── ordca-admin
    │   ├── fabric-ca-client-config.yaml
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-OrdererCA.pem
    │       ├── keystore
    │       │   └── f42ca80121375d3398e4663613128683c968e639def3e83c1674f4562f36f512_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── ordcatls
    │   └── tls.pem
    ├── orderer-admin
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-OrdererCA.pem
    │       ├── keystore
    │       │   └── 8ac962d836c0ee0e7458e1deed6e1609da5e473572e292b45da63049e768af19_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    └── ordtlsca-admin
     ├── fabric-ca-client-config.yaml
     └── msp
         ├── cacerts
         │   └── 9-12-19-115-31491-orderer-tlsca.pem
         ├── keystore
         │   └── 060ce4229cc590663813861ba01a6f28bc058f27c65a039c1453dbc87d349840_sk
         ├── signcerts
         │   └── cert.pem
         └── user
    
    23 directories, 26 files
    

Deploy Orderer

Note: See more information on deploying an Orderer.

Here’s an example of an Orderer being deployed: OrdererDeploy Ex

Before moving on, check that your Orderer deployment is available with kubectl get deploy $orderer_release-orderer.

Note: It is normal for this to take a few minutes to become available. If it’s not available after a while, check the logs in Kibana or with kubectl logs.

Deploy PeerOrgCA

Note: See more information on deploying PeerOrgCA.

  1. Create the CA Secret.

    Set a user and password for your secret, which you will use later with CA register:

    export peerca_user=Siler
    export peerca_password=secure_password
    
  2. Create the Secret.

    kubectl create secret generic garrett-peer-ca --from-literal=ca-admin-name=$peerca_user --from-literal=ca-admin-password=$peerca_password
    

    Here’s an example of what you should see: secret/garrett-ca created

    Note: Special characters such as $, \, *, and ! require escaping. If the password you’re using has special characters, you need to escape them by using the \\\ characters (i.e., \\\\!*).

  3. Get the Proxy IP for the CA deployment with:

    kubectl get nodes -l "proxy=true" -o jsonpath="{.items[0].status.addresses[0].address}"
    

    Here’s a sample output:

    9.12.19.115
    

    Note: In order to run this command, you need admin access to the cluster. If you don’t have admin access, please get someone with admin access to run this command for you.

    peer-ca_deploy

  4. Show the release values:

    export release2=garrett-peer-ca
    
    helm get values $release2 --tls
    
    ca:
     name: PeerOrg1CA
     enabled: true
     proxyIP: 9.12.19.115
     app:
       arch: s390x
     tlsca:
       name: garrett-peer-tlsca
       cname: garrett-peer-tlsca-common
     dataPVC:
       storageClassName: nfs-recycle
     ca:
       caAdminSecret: garrett-peer-ca
    license: accept
    
  5. Set environment variables with values from the helm chart:

    export NODE_IP=9.12.19.115
    
    export peer_caname=PeerOrg1CA
    
    export peer_tlscaname=garrett-peer-tlsca
    
  6. Look at the recently deployed helm release and its instructions:

    helm status $release2 --tls
    
    LAST DEPLOYED: Wed Dec 12 18:48:57 2018
    NAMESPACE: bcaas-usa
    STATUS: DEPLOYED
    
    RESOURCES:
    ==> v1/Pod(related)
    NAME                                                  READY  STATUS   RESTARTS  AGE
    garrett-peer-ca-fabric-ca-deployment-cb5954cdd-z2xqd  1/1    Running  0         8m
    
    ==> v1/ConfigMap
    NAME                          DATA  AGE
    garrett-peer-ca-tlsca-config  1     8m
    garrett-peer-ca-ca-configmap  2     8m
    
    ==> v1/PersistentVolumeClaim
    NAME                           STATUS  VOLUME                                    CAPACITY  ACCESS MODES  STORAGECLASS  AGE
    garrett-peer-ca-fabric-ca-pvc  Bound   pvc-a37c731e-fe81-11e8-bcdf-525400442dbc  2Gi       RWX           nfs-recycle   8m
    
    ==> v1/Service
    NAME                TYPE      CLUSTER-IP     EXTERNAL-IP  PORT(S)         AGE
    garrett-peer-ca-ca  NodePort  192.168.1.176  <none>       7054:32179/TCP  8m
    
    ==> v1/Deployment
    NAME                                  DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
    garrett-peer-ca-fabric-ca-deployment  1        1        1           1          8m
    
    NOTES:
    1. Get the application URL by running these commands:
     export SERVICE_NAME=$(kubectl get services --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-peer-ca" -o jsonpath="{.items[0].metadata.name}")
     export NODE_PORT=$(kubectl get --namespace bcaas-usa -o jsonpath="{.spec.ports[0].nodePort}" services $SERVICE_NAME)
     export NODE_IP=$(kubectl get nodes --namespace bcaas-usa -l "proxy=true" -o jsonpath="{.items[0].status.addresses[0].address}")
     echo http://$NODE_IP:$NODE_PORT
    2. Get the endpoint URL by running these commands:
     export APISERVER=$(kubectl config view --minify | grep server | cut -f 2- -d ":" | tr -d " ")
     export POD_NAME=$(kubectl get pods --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-peer-ca" -o jsonpath="{.items[0].metadata.name}")
     export NAMESPACE=bcaas-usa
     export SERVICE_NAME=$(kubectl get services --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-peer-ca" -o jsonpath="{.items[0].metadata.name}")
     echo $APISERVER/api/v1/namespaces/$NAMESPACE/services/$SERVICE_NAME:7054/proxy
    3. Get TLS certificate
     export POD_NAME=$(kubectl get pods --namespace bcaas-usa -l "app=ibm-ibp, release=garrett-peer-ca" -o jsonpath="{.items[0].metadata.name}")
     kubectl exec $POD_NAME -- cat /etc/hyperledger/fabric-ca-server/ca-cert.pem > tls.pem && cat tls.pem | base64
    4. Enroll the bootstrap admin identity:
     Enroll with TLS CA:
     fabric-ca-client enroll -d -u http://$CA_ADMIN:$CA_PASSWORD@$SERVICE_NAME:7054 --tls.certfiles tls-cert.pem --caname tlsca --enrollment.profile tls
     Enroll with default CA:
     fabric-ca-client enroll -d -u http://$CA_ADMIN:$CA_PASSWORD@$SERVICE_NAME:7054 --tls.certfiles tls-cert.pem
    

    Before moving on, check that your PeerCA deployment is available with kubectl get deploy $release2-fabric-ca-deployment.

Note: It’s normal for this to take a few minutes to become available. If it’s not available after a while, check logs in Kibana or with kubectl logs.

Operate PeerOrg CA

Note: See more information on operating the CA.

  1. Set PeerOrgCA environment variables.

    Note: These instructions are available in the helm chart release notes, step 2. In our case, they are modified to use our set environment variables.

    export SERVICE_NAME2=$(kubectl get services -l "app=ibm-ibp, release=$release2" -o jsonpath="{.items[0].metadata.name}")
    
    export NODE_PORT2=$(kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services $SERVICE_NAME2)
    
    export PEERCAURL=$NODE_IP:$NODE_PORT2
    
    echo http://$PEERCAURL
    
    export POD_NAME2=$(kubectl get pods -l "app=ibm-ibp, release=$release2" -o jsonpath="{.items[0].metadata.name}")
    
  2. Make sure the Fabric binaries path is set:

    export PATH=$PATH:$ibp4icp_install_dir/fabric-binaries/bin
    
  3. Set the path where you want to store admin certs for peerca:

    mkdir $ibp4icp_install_dir/peerca-admin
    export FABRIC_CA_CLIENT_HOME=$ibp4icp_install_dir/peerca-admin
    
  4. Get the certs and make the base configuration file for deploying components with CA (i.e., orderer and peer).

    export PEERCACERT=$ibp4icp_install_dir/peercatls/tls.pem
    
    kubectl cp $POD_NAME2:/etc/hyperledger/fabric-ca-server/ca-cert.pem $PEERCACERT
    

    Note: Don’t worry if you see tar: removing leading '/' from member names. It’s a normal message w/ exit code 0.

  5. Make certificates by using the deployed CA:

    fabric-ca-client enroll -u https://$peerca_user:$peerca_password@$PEERCAURL --caname $peer_caname --tls.certfiles $PEERCACERT
    
  6. Go back to the connection directory where connection.json base file was created earlier:

    cd $ibp4icp_install_dir/connection
    
  7. Use the connection template for peer1.json:

    cp connection.json peer1.json
    
  8. Get the base values for your file. (Use the same process as detailed above for orderer, but with values from the following commands placed into the peer1.json file.)

    echo $NODE_IP
    

    Here’s a sample output:

    9.12.19.115
    

    Note: This goes in enrollment.component.cahost and enrollment.tls.cahost, where enrollment.component.cahost is the field inside of the component section of the enrollment section of the peer1.json file we’re making.

    echo $NODE_PORT2
    

    Here’s a sample output:

    32179
    

    Note: This goes in enrollment.component.caport and enrollment.tls.caport.

    echo $peer_caname
    

    Here’s a sample output:

    PeerOrg1CA
    

    Note: This goes in enrollment.component.caname.

    echo $peer_tlscaname
    

    Here’s a sample output:

    garrett-peer-tlsca
    

    This goes in enrollment.tls.caname.

    Note: For Linux, the following command needs to have -w 0 (i.e., cat $PEERCACERT | base64 -w 0).

    cat $PEERCACERT | base64
    

    Here’s a sample output:

    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUNGakNDQWIyZ0F3SUJBZ0lVRnZKTjA2RGhPVnRHWitReFArOEVvOUttTTEwd0NnWUlLb1pJemowRUF3SXcKYURFTE1Ba0dBMVVFQmhNQ1ZWTXhGekFWQmdOVkJBZ1REazV2Y25Sb0lFTmhjbTlzYVc1aE1SUXdFZ1lEVlFRSwpFd3RJZVhCbGNteGxaR2RsY2pFUE1BMEdBMVVFQ3hNR1JtRmljbWxqTVJrd0Z3WURWUVFERXhCbVlXSnlhV010ClkyRXRjMlZ5ZG1WeU1CNFhEVEU0TVRJeE16QXlORFF3TUZvWERUTXpNVEl3T1RBeU5EUXdNRm93YURFTE1Ba0cKQTFVRUJoTUNWVk14RnpBVkJnTlZCQWdURGs1dmNuUm9JRU5oY205c2FXNWhNUlF3RWdZRFZRUUtFd3RJZVhCbApjbXhsWkdkbGNqRVBNQTBHQTFVRUN4TUdSbUZpY21sak1Sa3dGd1lEVlFRREV4Qm1ZV0p5YVdNdFkyRXRjMlZ5CmRtVnlNRmt3RXdZSEtvWkl6ajBDQVFZSUtvWkl6ajBEQVFjRFFnQUViQ20wNDJZQ0Nmd0dzbTZJYklISDZVaUQKZ2cxdFhGNHJMS1cvUVRRQ0Y4VXQwK2RkOEwzOWRPVnVGZGV4YkpZdldFZ3dWK2hpbzJmYW9WcmRWdVNyNWFORgpNRU13RGdZRFZSMFBBUUgvQkFRREFnRUdNQklHQTFVZEV3RUIvd1FJTUFZQkFmOENBUUV3SFFZRFZSME9CQllFCkZKMjZBbUQ1clVGQUY0VytrVHFCZUZnbURqUmNNQW9HQ0NxR1NNNDlCQU1DQTBjQU1FUUNJSG9QTURpMC9sNloKYUg0bXdNd2xiZ1hWRWNkYlRoWXFyRndqWmZMM2VqdnhBaUJhak1rMk1Fa0NNd3B5MHc1YUIxTWFXMXhGYmtqYgptQ3pJbERWL3E1MmZxdz09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
    

    Note: This goes in enrollment.component.catls.cacert and enrollment.tls.catls.cacert.

  9. Create the affiliation for PeerOrg1:

    fabric-ca-client affiliation add PeerOrg1 --caname $peer_caname --tls.certfiles $PEERCACERT
    

    Here’s a sample output:

    2018/12/12 19:19:42 [INFO] Configuration file location: /Users/garrettwoodworth/fabric-ca-client/peerca-admin/fabric-ca-client-config.yaml
    2018/12/12 19:19:42 [INFO] TLS Enabled
    2018/12/12 19:19:42 [INFO] TLS Enabled
    Successfully added affiliation: PeerOrg1
    

    Then:

    fabric-ca-client affiliation list --caname $peer_caname --tls.certfiles $PEERCACERT
    

    Here’s a sample output:

    affiliation: .
     affiliation: org1
         affiliation: org1.department1
         affiliation: org1.department2
     affiliation: org2
         affiliation: org2.department1
     affiliation: PeerOrg1
    
  10. Register the Peer with PeerCA.

    Use the newly created affiliation from the above fabric-ca-client affiliation list command.

    fabric-ca-client register --caname $peer_caname --id.affiliation PeerOrg1 --id.type peer --id.name peer1 --id.secret peer1_pwd --tls.certfiles $PEERCACERT
    

    Here’s a sample output:

    2018/12/12 19:22:05 [INFO] Configuration file location: /Users/garrettwoodworth/fabric-ca-client/peerca-admin/fabric-ca-client-config.yaml
    2018/12/12 19:22:05 [INFO] TLS Enabled
    2018/12/12 19:22:05 [INFO] TLS Enabled
    Password: peer1_pwd
    

    Note: The id.name value in the above register command (i.e. peer1) goes in enrollment.component.enrollid in the peer1.json file.* The id.secret value in the above register command (i.e., peer1_pwd) goes in enrollment.component.enrollsecret in the peer1.json file.

  11. Make the peer-admin MSP:

    fabric-ca-client register --caname $peer_caname --id.name PeerOrg1admin --id.affiliation PeerOrg1 --id.type user --id.secret secure_password --tls.certfiles $PEERCACERT
    

    Then:

    fabric-ca-client enroll -u https://PeerOrg1admin:secure_password@$PEERCAURL --caname $peer_caname -M $ibp4icp_install_dir/peer-admin/msp --tls.certfiles $PEERCACERT
    
  12. Add the Cert to the file.

    Note: For Linux, the following command needs to have -w 0 (i.e., cat $ibp4icp_install_dir/peer-admin/msp/signcerts/cert.pem | base64 -w 0).

    cat $ibp4icp_install_dir/peer-admin/msp/signcerts/cert.pem | base64
    
    LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN6ekNDQW5XZ0F3SUJBZ0lVT2UwZElzcytYajZpL3VqVlFRQnJLanNqam5Fd0NnWUlLb1pJemowRUF3SXcKYURFTE1Ba0dBMVVFQmhNQ1ZWTXhGekFWQmdOVkJBZ1REazV2Y25Sb0lFTmhjbTlzYVc1aE1SUXdFZ1lEVlFRSwpFd3RJZVhCbGNteGxaR2RsY2pFUE1BMEdBMVVFQ3hNR1JtRmljbWxqTVJrd0Z3WURWUVFERXhCbVlXSnlhV010ClkyRXRjMlZ5ZG1WeU1CNFhEVEU0TVRJeE16QXpNak13TUZvWERURTVNVEl4TXpBek1qZ3dNRm93ZERFTE1Ba0cKQTFVRUJoTUNWVk14RnpBVkJnTlZCQWdURGs1dmNuUm9JRU5oY205c2FXNWhNUlF3RWdZRFZRUUtFd3RJZVhCbApjbXhsWkdkbGNqRWVNQXNHQTFVRUN4TUVkWE5sY2pBUEJnTlZCQXNUQ0ZCbFpYSlBjbWN4TVJZd0ZBWURWUVFECkV3MVFaV1Z5VDNKbk1XRmtiV2x1TUZrd0V3WUhLb1pJemowQ0FRWUlLb1pJemowREFRY0RRZ0FFckhuL1F6dWUKRWplb0lRNXNpdmdJL0hNRTdMMmVLNWdSQnByTS9OYnh2OFFqOExtcHNjSFdIU0pNU1JRa3EvajBVVWpnUGF6OQo0ZjNmUHl2d0JPYngxS09COERDQjdUQU9CZ05WSFE4QkFmOEVCQU1DQjRBd0RBWURWUjBUQVFIL0JBSXdBREFkCkJnTlZIUTRFRmdRVTdVVTMzMUppUkpVREdqVjcxL2lqNFFmbS9uNHdId1lEVlIwakJCZ3dGb0FVbmJvQ1lQbXQKUVVBWGhiNlJPb0Y0V0NZT05Gd3dKUVlEVlIwUkJCNHdISUlhUjJGeWNtVjBkSE10VFdGalFtOXZheTFRY204dQpiRzlqWVd3d1pnWUlLZ01FQlFZSENBRUVXbnNpWVhSMGNuTWlPbnNpYUdZdVFXWm1hV3hwWVhScGIyNGlPaUpRClpXVnlUM0puTVNJc0ltaG1Ma1Z1Y205c2JHMWxiblJKUkNJNklsQmxaWEpQY21jeFlXUnRhVzRpTENKb1ppNVUKZVhCbElqb2lkWE5sY2lKOWZUQUtCZ2dxaGtqT1BRUURBZ05JQURCRkFpRUEvbGl5dU5CRmZOeElZMlp2aHo0OQpOZVlaTWVmQ0ppNTRYMEY4ZlFBMTZKb0NJR0lyUS9MV2NCaFZoSS9EZzBOTHBJRVJLT01VNDVjZEhObnFxb0N1CkNPbUMKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    

    This goes in enrollment.component.admincert in the peer1.json file.

  13. Set up Peer TLS CA admin:

    mkdir $ibp4icp_install_dir/peertlsca-admin
    
    export FABRIC_CA_CLIENT_HOME=$ibp4icp_install_dir/peertlsca-admin
    
    fabric-ca-client enroll -u https://$peerca_user:$peerca_password@$PEERCAURL --caname $peer_tlscaname --tls.certfiles $PEERCACERT
    
  14. Create the affiliation for PeerOrg1:

    fabric-ca-client affiliation add PeerOrg1 --caname $peer_tlscaname --tls.certfiles $PEERCACERT
    
    fabric-ca-client affiliation list --caname $peer_tlscaname --tls.certfiles $PEERCACERT
    

    Here’s a sample output:

    affiliation: .
    affiliation: PeerOrg1
    affiliation: org2
       affiliation: org2.department1
    affiliation: org1
       affiliation: org1.department1
       affiliation: org1.department2
    
  15. Register the Peer with Peer TLS CA:

    fabric-ca-client register --caname $peer_tlscaname --id.name peer1tls --id.affiliation PeerOrg1 --id.type peer --id.secret peer1tls_pw --tls.certfiles $PEERCACERT
    

    The id.name value in the above register command (i.e., peer1tls) goes in enrollment.tls.enrollid in the peer1.json file. The id.secret value in the above register command (i.e., peer1tls_pw) goes in enrollment.tls.enrollsecret in the peer1.json file.

  16. Finish the Configuration File with csr.hosts entry.

    Decide the peer release name, as this will be the value you enter in the helm chart for deployment.

    export peer_release=garrett-peer1
    

    Note: The value you enter in the csr.hosts will be the desired release name.

    echo $peer_release
    
    echo $NODE_IP
    

    Here’s an example of csr.hosts:

    "csr": {
              "hosts": [
                "9.12.19.115",
                "garrett-peer1",
                "127.0.0.1"
              ]
          }
    
  17. Use a CouchDB user and password for the CouchDB peer deployment:

    kubectl create secret generic garrett-peer1 --from-file=secret.json=peer1.json --from-literal=couchdbusr=Garrett --from-literal=couchdbpwd=Woodworth
    

    Note: If you’re using leveldb as your ledger for your network, don’t add couchdbusr and couchdbpwd fields.

    Checkpoint: Here’s what your current directory structure before peer1 deploy should look like:

    cd $ibp4icp_install_dir
    
    tree
    .
    ├── connection
    │   ├── connection.json
    │   ├── orderer.json
    │   └── peer1.json
    ├── fabric-binaries
    │   ├── bin
    │   │   ├── configtxgen
    │   │   ├── configtxlator
    │   │   ├── cryptogen
    │   │   ├── discover
    │   │   ├── fabric-ca-client
    │   │   ├── get-docker-images.sh
    │   │   ├── idemixgen
    │   │   ├── orderer
    │   │   └── peer
    │   └── config
    │       ├── configtx.yaml
    │       ├── core.yaml
    │       └── orderer.yaml
    ├── ordca-admin
    │   ├── fabric-ca-client-config.yaml
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-OrdererCA.pem
    │       ├── keystore
    │       │   └── f42ca80121375d3398e4663613128683c968e639def3e83c1674f4562f36f512_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── ordcatls
    │   └── tls.pem
    ├── orderer-admin
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-OrdererCA.pem
    │       ├── keystore
    │       │   └── 8ac962d836c0ee0e7458e1deed6e1609da5e473572e292b45da63049e768af19_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── ordtlsca-admin
    │   ├── fabric-ca-client-config.yaml
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-orderer-tlsca.pem
    │       ├── keystore
    │       │   └── 060ce4229cc590663813861ba01a6f28bc058f27c65a039c1453dbc87d349840_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── peer-admin
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-32179-PeerOrg1CA.pem
    │       ├── keystore
    │       │   └── ec56817088e65def569add9a76755257468e1b09d4590182cbf9dd46578a7792_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── peerca-admin
    │   ├── fabric-ca-client-config.yaml
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-32179-PeerOrg1CA.pem
    │       ├── keystore
    │       │   └── 9775a604fe2a52236f5c5a4246ea9948bc1fd0b7c7604f83d0a070fe74d68789_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── peercatls
    │   └── tls.pem
    └── peertlsca-admin
      ├── fabric-ca-client-config.yaml
      └── msp
          ├── cacerts
          │   └── 9-12-19-115-32179-garrett-peer-tlsca.pem
          ├── keystore
          │   └── 8fa6dbe20129a7566ea3ae664c6ac9285897648bb3a9f4817af6b4febbc6fc7e_sk
          ├── signcerts
          │   └── cert.pem
          └── user
    
    42 directories, 39 files
    

Deploy Peer

Note: See more information on deploying the peer.

Here’s an example of deploying a peer: Peer1 Deploy

Before moving on, check that your peer deployment is available with kubectl get deploy $peer_release.

Note: It is normal for this to take a few minutes to become available. If it’s not available after a while, check the logs in Kibana or with kubectl logs.

Operate Orderer and Peer

Orderer

Note: See more information on Operating Orderer

  1. Set the orderer env values.

    Check if orderer_release is still set with echo $orderer_release. If it’s still not set, do so with export orderer_release=garrett-ord1 (or whatever your orderer release name was).

    export ORDERER_PORT=$(kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services $orderer_release-orderer)
    export ORDERER_URL=$NODE_IP:$ORDERER_PORT
    
  2. Check that the Orderer URL was set correctly:

    echo $ORDERER_URL
    

    Here’s a sample output:

    9.12.19.115:31953
    

    Then:

    export ORDER_POD_NAME=$(kubectl get pods -l "release=$orderer_release" -o jsonpath="{.items[0].metadata.name}")
    
  3. Set up the Orderer MSP:

    cd $ibp4icp_install_dir/orderer-admin/msp
    mkdir admincerts
    cp signcerts/cert.pem admincerts/cert.pem
    
  4. Get the Orderer tls cert:

    mkdir $ibp4icp_install_dir/orderer-tls
    
    kubectl cp $ORDER_POD_NAME:/certs/tls/signcerts/cert.pem $ibp4icp_install_dir/orderer-tls/orderertls.pem
    

Peer

Note: See more information on Operating Peer

  1. Set the peer env variables:

    peer_release=garrett-peer1
    export PEER_PORT=$(kubectl get -o jsonpath="{.spec.ports[0].nodePort}" services $peer_release)
    PEER_URL=$NODE_IP:$PEER_PORT
    
  2. Check that the Peer URL is set correctly:

    echo $PEER_URL
    
  3. Set up the peer certs:

    cd $ibp4icp_install_dir/peer-admin/msp
    mkdir admincerts
    cp signcerts/cert.pem admincerts/cert.pem
    mkdir tlscacerts
    

    The following cert will need a tab after typing out $ibp4icp_install_dir/peertlsca-admin/msp/cacerts/ and then tlscacerts/, as it won’t be the same as my9-12-19-115-32225-org1tlsca.pem.

    cp $ibp4icp_install_dir/peertlsca-admin/msp/cacerts/9-12-19-115-32225-org1tlsca.pem tlscacerts/
    
    cd $ibp4icp_install_dir
    
  4. Retrieve the peer tls cert:

    mkdir $ibp4icp_install_dir/peer-tls
    
    export PEER_POD_NAME=$(kubectl get pods -l "release=$peer_release" -o jsonpath="{.items[0].metadata.name}")
    
    kubectl cp $PEER_POD_NAME:/certs/tls/cacerts/cacert.pem -c peer $ibp4icp_install_dir/peer-tls/peertls.pem
    
  5. Check that your current directory tree looks like the following after setting up your peer and orderer certs.

    cd $ibp4icp_install_dir
    

    Here’s what your tree should look like:

    tree
    .
    ├── connection
    │   ├── connection.json
    │   ├── orderer.json
    │   └── peer1.json
    ├── fabric-binaries
    │   ├── bin
    │   │   ├── configtxgen
    │   │   ├── configtxlator
    │   │   ├── cryptogen
    │   │   ├── discover
    │   │   ├── fabric-ca-client
    │   │   ├── get-docker-images.sh
    │   │   ├── idemixgen
    │   │   ├── orderer
    │   │   └── peer
    │   └── config
    │       ├── configtx.yaml
    │       ├── core.yaml
    │       └── orderer.yaml
    ├── ordca-admin
    │   ├── fabric-ca-client-config.yaml
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-OrdererCA.pem
    │       ├── keystore
    │       │   └── f42ca80121375d3398e4663613128683c968e639def3e83c1674f4562f36f512_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── ordcatls
    │   └── tls.pem
    ├── orderer-admin
    │   └── msp
    │       ├── admincerts
    │       │   └── cert.pem
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-OrdererCA.pem
    │       ├── keystore
    │       │   └── 8ac962d836c0ee0e7458e1deed6e1609da5e473572e292b45da63049e768af19_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── orderer-tls
    │   └── orderertls.pem
    ├── ordtlsca-admin
    │   ├── fabric-ca-client-config.yaml
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-31491-orderer-tlsca.pem
    │       ├── keystore
    │       │   └── 060ce4229cc590663813861ba01a6f28bc058f27c65a039c1453dbc87d349840_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── peer-admin
    │   └── msp
    │       ├── admincerts
    │       │   └── cert.pem
    │       ├── cacerts
    │       │   └── 9-12-19-115-32179-PeerOrg1CA.pem
    │       ├── keystore
    │       │   └── ec56817088e65def569add9a76755257468e1b09d4590182cbf9dd46578a7792_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       ├── tlscacerts
    │       │   └── 9-12-19-115-32179-garrett-peer-tlsca.pem
    │       └── user
    ├── peer-tls
    │   └── peertls.pem
    ├── peerca-admin
    │   ├── fabric-ca-client-config.yaml
    │   └── msp
    │       ├── cacerts
    │       │   └── 9-12-19-115-32179-PeerOrg1CA.pem
    │       ├── keystore
    │       │   └── 9775a604fe2a52236f5c5a4246ea9948bc1fd0b7c7604f83d0a070fe74d68789_sk
    │       ├── signcerts
    │       │   └── cert.pem
    │       └── user
    ├── peercatls
    │   └── tls.pem
    └── peertlsca-admin
       ├── fabric-ca-client-config.yaml
       └── msp
           ├── cacerts
           │   └── 9-12-19-115-32179-garrett-peer-tlsca.pem
           ├── keystore
           │   └── 8fa6dbe20129a7566ea3ae664c6ac9285897648bb3a9f4817af6b4febbc6fc7e_sk
           ├── signcerts
           │   └── cert.pem
           └── user
    
    47 directories, 44 files
    
  6. Set up the configtx.yaml file to add the peer org to the new consortium:

    export FABRIC_CFG_PATH=$ibp4icp_install_dir/fabric-binaries/config/
    
    cd $FABRIC_CFG_PATH
    
    kubectl exec -it $PEER_POD_NAME -c peer -- sh -c "env | grep CORE_PEER_LOCALMSPID"
    

    Here’s a sample output:

    CORE_PEER_LOCALMSPID=PeerOrg1
    

    Then:

    rm -rf configtx.yaml
    
    echo $PEER_URL
    
    echo $ibp4icp_install_dir/peer-admin/msp
    

    Note: You need to enter the above admin with what is used in the echo, since entering $HOME in the configtx.yaml file will result in errors.

  7. Copy the text below into the new configtx.yaml file and then edit:

    ---
    ################################################################################
    #
    #   Section: Organizations
    #
    #   - This section defines the different organizational identities which will
    #   be referenced later in the configuration.
    #
    ################################################################################
    Organizations:
       - &org1
           # DefaultOrg defines the organization which is used in the sampleconfig
           # of the fabric.git development environment
           Name: org1
    
           # ID to load the MSP definition as
           ID: org1
    
           MSPDir: <path_to_peer-admin>/msp
    
           AnchorPeers:
               # AnchorPeers defines the location of peers which can be used
               # for cross org gossip communication.  Note, this value is only
               # encoded in the genesis block in the Application section context
               - Host: 9.30.250.70
                 Port: 30481
    
  8. Change the configtx file and create the PeerOrg1 organization definition.

    Here’s an example: Org Definition json

  9. Make the organization definition from the yaml file:

    mkdir $FABRIC_CFG_PATH/configupdate
    
    configtxgen -printOrg PeerOrg1 > $FABRIC_CFG_PATH/configupdate/orgdefinition.json
    

Orderer

Note: See more information on Operating Orderer

Add the Peer Org to the Consortium with a system channel update.
  1. Set the environment variables:

    kubectl exec -it $ORDER_POD_NAME -c orderer -- sh -c "env | grep ORDERER_GENERAL_LOCALMSPID"
    

    Here’s a sample output:

    ORDERER_GENERAL_LOCALMSPID=OrdererOrg
    

    Then:

    export CORE_PEER_LOCALMSPID=OrdererOrg
    
    export CORE_PEER_MSPCONFIGPATH=$ibp4icp_install_dir/orderer-admin/msp/
    
    export CORE_PEER_TLS_ROOTCERT_FILE=$ibp4icp_install_dir/orderer-tls/orderertls.pem
    
    export CORE_PEER_TLS_ENABLED=true
    
    export CHANNEL_NAME=test-system-channel-name
    
    export ORDERER_CA=$ibp4icp_install_dir/orderer-tls/orderertls.pem
    
  2. Fetch the channel block and update it:

    cd $FABRIC_CFG_PATH
    
    peer channel fetch config ./configupdate/genesis.pb -o $NODE_IP:$ORDERER_PORT -c $CHANNEL_NAME --tls --cafile $ORDERER_CA
    
    cd $FABRIC_CFG_PATH/configupdate
    
    configtxlator proto_decode --input genesis.pb --type common.Block --output ./config_block.json
    
    jq .data.data[0].payload.data.config ./config_block.json  > config.json
    
    jq -s '.[0] * {"channel_group":{"groups":{"Consortiums":{"groups":{"SampleConsortium":{"groups": {"PeerOrg1":.[1]}}}}}}}' config.json ./orgdefinition.json > modified_config.json
    
    configtxlator proto_encode --input config.json --type common.Config --output config.pb
    
    configtxlator proto_encode --input modified_config.json --type common.Config --output modified_config.pb
    
    configtxlator compute_update --channel_id $CHANNEL_NAME --original config.pb --updated modified_config.pb --output config_update.pb
    
    configtxlator proto_decode --input config_update.pb --type common.ConfigUpdate --output config_update.json
    
    echo '{"payload":{"header":{"channel_header":{"channel_id":"'$CHANNEL_NAME'", "type":2}},"data":{"config_update":'$(cat config_update.json)'}}}' | jq . > config_update_in_envelope.json
    
    configtxlator proto_encode --input config_update_in_envelope.json --type common.Envelope --output ./config_update_in_envelope.pb
    
  3. Update the peer channel with the new consortium and PeerOrg1:

    peer channel update -f config_update_in_envelope.pb -c $CHANNEL_NAME -o $NODE_IP:$ORDERER_PORT --tls --cafile $ORDERER_CA
    

Peer

Note: See more information on Operating the Peer

  1. Create a new channel:

    cd $FABRIC_CFG_PATH
    
    cp configtx.yaml oldconfigtx.yaml
    
  2. Here’s the file to use for channel configuration. Please add this to previous configtx.yaml file. This file adds in extra parameters associated to channels such as the capabilities, application, and profiles sections:

    ################################################################################
    #
    #   SECTION: Capabilities
    #
    ################################################################################
    Capabilities:
       # Channel capabilities apply to both the orderers and the peers and must be
       # supported by both.  Set the value of the capability to true to require it.
       Global: &ChannelCapabilities
    
           V1_1: true
    
       # Orderer capabilities apply only to the orderers, and may be safely
       # manipulated without concern for upgrading peers.  Set the value of the
       # capability to true to require it.
       Orderer: &OrdererCapabilities
    
           V1_1: true
    
       # Application capabilities apply only to the peer network, and may be safely
       # manipulated without concern for upgrading orderers.  Set the value of the
       # capability to true to require it.
       Application: &ApplicationCapabilities
    
           V1_2: true
    
    ################################################################################
    #
    #   SECTION: Application
    #
    #   - This section defines the values to encode into a config transaction or
    #   genesis block for application related parameters
    #
    ################################################################################
    
    Application: &ApplicationDefaults
    
       # Organizations is the list of orgs which are defined as participants on
       # the application side of the network
       Organizations:
    
       Capabilities:
           <<: *ApplicationCapabilities
    
    ################################################################################
    #
    #   Profile
    #
    #   - Different configuration profiles may be encoded here to be specified
    #   as parameters to the configtxgen tool
    #
    ################################################################################
    
    Profiles:
    
       mychannel:
           Consortium: SampleConsortium
           Application:
               <<: *ApplicationDefaults
               Organizations:
                   - *org1
               Capabilities:
                   <<: *ApplicationCapabilities
    
  3. Here’s a Changed File example:

    ---
    ################################################################################
    #
    #   Section: Organizations
    #
    #   - This section defines the different organizational identities which will
    #   be referenced later in the configuration.
    #
    ################################################################################
    Organizations:
       - &PeerOrg1
           # DefaultOrg defines the organization which is used in the sampleconfig
           # of the fabric.git development environment
           Name: PeerOrg1
    
           # ID to load the MSP definition as
           ID: PeerOrg1
    
           MSPDir: /Users/garrettwoodworth/fabric-ca-client/peer-admin/msp
    
           AnchorPeers:
               # AnchorPeers defines the location of peers which can be used
               # for cross org gossip communication.  Note, this value is only
               # encoded in the genesis block in the Application section context
               - Host: 9.12.19.115
                 Port: 30967
    
    ################################################################################
    #
    #   SECTION: Capabilities
    #
    ################################################################################
    Capabilities:
       # Channel capabilities apply to both the orderers and the peers and must be
       # supported by both.  Set the value of the capability to true to require it.
       Global: &ChannelCapabilities
    
           V1_1: true
    
       # Orderer capabilities apply only to the orderers, and may be safely
       # manipulated without concern for upgrading peers.  Set the value of the
       # capability to true to require it.
       Orderer: &OrdererCapabilities
    
           V1_1: true
    
       # Application capabilities apply only to the peer network, and may be safely
       # manipulated without concern for upgrading orderers.  Set the value of the
       # capability to true to require it.
       Application: &ApplicationCapabilities
    
           V1_2: true
    
    ################################################################################
    #
    #   SECTION: Application
    #
    #   - This section defines the values to encode into a config transaction or
    #   genesis block for application related parameters
    #
    ################################################################################
    
    Application: &ApplicationDefaults
    
       # Organizations is the list of orgs which are defined as participants on
       # the application side of the network
       Organizations:
    
       Capabilities:
           <<: *ApplicationCapabilities
    
    ################################################################################
    #
    #   Profile
    #
    #   - Different configuration profiles may be encoded here to be specified
    #   as parameters to the configtxgen tool
    #
    ################################################################################
    
    Profiles:
    
       mychannel:
           Consortium: SampleConsortium
           Application:
               <<: *ApplicationDefaults
               Organizations:
                   - *PeerOrg1
               Capabilities:
                   <<: *ApplicationCapabilities
    
Use configtx.yaml to create a channel

Sample config through channel creation

  1. Set the environment variables:

    export CHANNEL=gchannel
    export CC_NAME=gcc
    export CORE_PEER_ADDRESS=$PEER_URL
    export ORDERER_1=$ORDERER_URL
    export CORE_PEER_MSPCONFIGPATH=$ibp4icp_install_dir/peer-admin/msp
    export CORE_PEER_LOCALMSPID=PeerOrg1
    export CORE_PEER_TLS_ENABLED=true
    
  2. Create the channel:

    configtxgen -profile mychannel -outputCreateChannelTx ./$CHANNEL.tx -channelID $CHANNEL
    
    peer channel create -o $ORDERER_1 --tls --cafile $ORDERER_CA -c $CHANNEL -f ./$CHANNEL.tx
    
  3. Join the Peer to the new channel.

    Note: See more information on Operating Peer

    peer channel fetch 0 -o ${ORDERER_1} -c ${CHANNEL} --tls --cafile $ORDERER_CA
    
    export CORE_PEER_TLS_ROOTCERT_FILE=$ibp4icp_install_dir/peer-tls/peertls.pem
    
    peer channel join -b ${CHANNEL}_0.block --tls
    
  4. Install chaincode on the Peer.

    Note: Make sure GOPATH is set.

    cd $GOPATH/src
    
    git clone https://github.com/IBM-Blockchain/marbles
    
    export CC_VERSION=0
    
    peer chaincode install -n ${CC_NAME} -v ${CC_VERSION} -p marbles/chaincode/src/marbles
    

    Here’s a sample output:

    2018-12-12 23:46:58.385 PST [chaincodeCmd] checkChaincodeCmdParams -> INFO 001 Using default escc
    2018-12-12 23:46:58.385 PST [chaincodeCmd] checkChaincodeCmdParams -> INFO 002 Using default vscc
    2018-12-12 23:46:59.544 PST [chaincodeCmd] install -> INFO 003 Installed remotely response:<status:200 payload:"OK" >
    

    Note: It can take several minutes for chaincode to install on peer. If the chaincode is not yet installed, the instantiate command will likely give an error similar to the following:

    Error: could not assemble transaction, err Proposal response was not successful, error code 500, msg failed to execute transaction 55753428355313aae2b26d28fd1ed2305a6837ac58b92c49863e06e596b672c0: error starting container: error starting container: Post http://localhost:2375/containers/create?name=dev-jdoe-gcc-0: dial tcp [::1]:2375: connect: cannot assign requested address

  5. Check that chaincode is installed on peer:

    peer chaincode list --installed
    

    Here’s a sample output:

    Get installed chaincodes on peer:
    Name: gcc, Version: 0, Path: marbles/chaincode/src/marbles, Id:     7a4bf4771ca762929abab1836033e01c09f7cf018ed773ca9aa6a9936f3496af
    

Operate the Peer on a new application channel

  1. Instantiate the chaincode on the channel:

    peer chaincode instantiate -o ${ORDERER_1} -C ${CHANNEL} -n ${CC_NAME} -v ${CC_VERSION} -c '{"Args":[""]}' --tls --cafile $ORDERER_CA -P ""
    

    Here’s a sample output:

    2018-12-12 23:48:56.385 PST [chaincodeCmd] checkChaincodeCmdParams -> INFO 001 Using default escc
    2018-12-12 23:48:56.385 PST [chaincodeCmd] checkChaincodeCmdParams -> INFO 002 Using default vscc
    

    Note: It can take several minutes for chaincode to instantiate on channel. When the command completes, it will return your terminal prompt. You may still have to wait a minute or so before you’re able to successfully invoke the chaincode.

  2. Check for the instantiated Marbles Chaincode, the hello world of the Hyperledger Fabric world:

    peer chaincode list --instantiated -C $CHANNEL
    

    Here’s a sample output:

    Get instantiated chaincodes on channel gchannel:
    Name: gcc, Version: 0, Path: marbles/chaincode/src/marbles, Escc: escc, Vscc: vscc
    
  3. Invoke the instantiated Marbles Chaincode.

    First, create an owner:

    peer chaincode invoke  -C ${CHANNEL} -n ${CC_NAME} -c '{"Args":["init_owner","o0000000000001","Garrett","PeerOrg1"]}' --tls --cafile $ORDERER_CA -o ${ORDERER_1}
    

    Here’s a sample output:

    2019-01-18 18:22:08.304 PST [chaincodeCmd] chaincodeInvokeOrQuery -> INFO 001 Chaincode invoke successful. result: status:200
    

    Then, create a marble for your new owner:

    peer chaincode invoke  -C ${CHANNEL} -n ${CC_NAME} -c '{"Args":["init_marble","m0000000000001","black","200","o0000000000001","PeerOrg1"]}' --tls --cafile $ORDERER_CA -o ${ORDERER_1}
    

    Here’s a sample output:

    2019-01-18 18:22:38.323 PST [chaincodeCmd] chaincodeInvokeOrQuery -> INFO 001 Chaincode invoke successful. result: status:200
    
  4. See the results of your previous invokes with the following command:

    peer chaincode invoke  -C ${CHANNEL} -n ${CC_NAME} -c '{"Args":["read_everything"]}' --tls --cafile $ORDERER_CA -o ${ORDERER_1}
    

    Here’s a sample output:

    2019-01-18 18:22:46.658 PST [chaincodeCmd] chaincodeInvokeOrQuery -> INFO 001 Chaincode invoke successful. result: status:200 payload:"{\"owners\":[{\"docType\":\"marble_owner\",\"id\":\"o0000000000001\",\"username\":\"garrett\",\"company\":\"PeerOrg1\",\"enabled\":true}],\"marbles\":[{\"docType\":\"marble\",\"id\":\"m0000000000001\",\"color\":\"black\",\"size\":200,\"owner\":{\"id\":\"o0000000000001\",\"username\":\"garrett\",\"company\":\"PeerOrg1\"}}]}"
    

Finally, we have transacted! 🙂

Final checkpoint: Check that your tree has all the files from this tutorial:

cd $ibp4icp_install_dir
tree
.
├── connection
│   ├── connection.json
│   ├── orderer.json
│   └── peer1.json
├── fabric-binaries
│   ├── bin
│   │   ├── configtxgen
│   │   ├── configtxlator
│   │   ├── cryptogen
│   │   ├── discover
│   │   ├── fabric-ca-client
│   │   ├── get-docker-images.sh
│   │   ├── idemixgen
│   │   ├── orderer
│   │   └── peer
│   └── config
│       ├── configtx.yaml
│       ├── configupdate
│       │   ├── config.json
│       │   ├── config.pb
│       │   ├── config_block.json
│       │   ├── config_update.json
│       │   ├── config_update.pb
│       │   ├── config_update_in_envelope.json
│       │   ├── config_update_in_envelope.pb
│       │   ├── genesis.pb
│       │   ├── modified_config.json
│       │   ├── modified_config.pb
│       │   └── orgdefinition.json
│       ├── core.yaml
│       ├── gchannel.block
│       ├── gchannel.tx
│       ├── gchannel_0.block
│       ├── oldconfigtx.yaml
│       └── orderer.yaml
├── ordca-admin
│   ├── fabric-ca-client-config.yaml
│   └── msp
│       ├── cacerts
│       │   └── 192-168-22-81-31523-OrdererCA.pem
│       ├── keystore
│       │   └── fd44854e528850f83f8b6eefa0bee15d890f27f407128ec5bb5261cee4e60b45_sk
│       ├── signcerts
│       │   └── cert.pem
│       └── user
├── ordcatls
│   └── tls.pem
├── orderer-admin
│   └── msp
│       ├── admincerts
│       │   └── cert.pem
│       ├── cacerts
│       │   └── 192-168-22-81-31523-OrdererCA.pem
│       ├── keystore
│       │   └── 3061c76f8d8dee50918543c95a4c6a10da584267ce8f7970ddc094c58f695791_sk
│       ├── signcerts
│       │   └── cert.pem
│       └── user
├── orderer-tls
│   └── orderertls.pem
├── ordtlsca-admin
│   ├── fabric-ca-client-config.yaml
│   └── msp
│       ├── cacerts
│       │   └── 192-168-22-81-31523-orderer-tlsca.pem
│       ├── keystore
│       │   └── 3ae30750866b712adc185221328b2ea2a6781de85035a16c2aae7aeffc3fb620_sk
│       ├── signcerts
│       │   └── cert.pem
│       └── user
├── peer-admin
│   └── msp
│       ├── admincerts
│       │   └── cert.pem
│       ├── cacerts
│       │   └── 192-168-22-81-31757-PeerOrg1CA.pem
│       ├── keystore
│       │   └── 38d20e5b40f9a03c443cb38faea78386dce1c5b93a893c573b9c6e31e2636464_sk
│       ├── signcerts
│       │   └── cert.pem
│       ├── tlscacerts
│       │   └── 192-168-22-81-31757-garrett-peer-tlsca.pem
│       └── user
├── peer-tls
│   └── peertls.pem
├── peerca-admin
│   ├── fabric-ca-client-config.yaml
│   └── msp
│       ├── cacerts
│       │   └── 192-168-22-81-31757-PeerOrg1CA.pem
│       ├── keystore
│       │   └── 822404c058880c3f8fae7bbe5a010a24d788fc39d6e62d2d4176489dae4b3452_sk
│       ├── signcerts
│       │   └── cert.pem
│       └── user
├── peercatls
│   └── tls.pem
└── peertlsca-admin
    ├── fabric-ca-client-config.yaml
    └── msp
        ├── cacerts
        │   └── 192-168-22-81-31757-garrett-peer-tlsca.pem
        ├── keystore
        │   └── f57179235ceb76a3c091b7a236c738a73e63e68e4f037c0d4bcbb7cb58073e7e_sk
        ├── signcerts
        │   └── cert.pem
        └── user

48 directories, 59 files

Congratulations! You successfully implemented the IBM Blockchain Platform, which allows you to deploy a full fabric network (CA, Orderer, and Peer) within your own data center by using IBM Cloud Private.