Building a decentralized marketplace using Hyperledger Fabric
In our previous post, we described how a private blockchain could be a good fit to implement a decentralized marketplace. In this post, we expose the key points of our experience building a prototype using Hyperledger Fabric v1.4.
In our previous post Can we put our trust in a decentralized marketplace?, we proposed the idea of a decentralized marketplace built using a private blockchain and smart contracts. We detailed the guarantees that it needs to provide to its members and detailed how to implement them using the functionalities provided by a private blockchain. The conclusion was that a private blockchain can provide the required guarantees but that implementing such a solution requires caution.
To further explore this topic, we decided to implement a prototype decentralized marketplace using Hyperledger Fabric. Overall, this project helped us realize that the successful implementation of our envisioned decentralized marketplace requires more than just smart contract development. In this blog post, we list key requirements and provide details regarding the reasoning behind them and how they affected our decentralized marketplace prototype.
At the lower level, the deployment and maintenance of the distributed ledger peers require the commitment of all the participants in terms of resources and manpower, but also competence development. Looking at the smart contracts, they require deep technical and business knowledge to be correctly implemented and administrated. Going further, the ledger participants need to agree on how to deal with the consequences of a smart contract failure. Additionally, the blockchain does not stand in a vacuum – it needs to be integrated with the infrastructure of each participant, generating more costs for them to bear.
At the time of this work, the latest Hyperledger Fabric release was 1.4. As a result, our implementation does not leverage the improvements that were brought in version 2. However, the points discussed here are, to the best of our knowledge, still valid.
Private blockchain platform
Based on the design that was detailed in the previous post, we had a pretty good idea of what we needed to implement using the blockchain. The first step to realize it was to choose which platform to use.
We identified three major blockchain platforms as having all the features we needed to implement our decentralized marketplace: JP Morgan Quorum, R3 Corda and Hyperledger Fabric. We chose Hyperledger Fabric due to its large and active community and because some project members already had some experience working with it.
After deciding on a ledger to use, the next step was to understand its architecture and how to apply it to our use case. This step was critical to make sure that we were getting the properties that we expected from the private ledger. Indeed, an incorrectly deployed private ledger brings no more guarantees than a centralized database.
Looking specifically at Hyperledger Fabric, the system is composed of two types of nodes: peers and orderers. The peers are responsible for hosting the ledger and smart contracts. While the orderers are responsible for ordering the transactions and doing some administrative bookkeeping.
As a user of Hyperledger Fabric, some of the guarantees are only achieved when hosting a peer. Taking smart contract execution as an example, hosting a peer is the only way to be sure that the execution of a smart contract is faithful to its definition. This means that, in most cases, all the organizations participating in the ledger will need to host at least one peer. In addition to increasing their cost, this requires every organization to have a decent competence level in Hyperledger Fabric.
For the orderers, the problem is different. Running an orderer provides its host with the possibility to read the content of all the transactions, as well as influence or even prevent their ordering. As a result, the organizations hosting the orderers must be chosen carefully so as not to weaken the system. Moreover, running an orderer is also another cost that some organizations must pay, for the blockchain to operate.
For our decentralized marketplace use case, because we assume that each organization does not trust any other to properly execute the smart contracts, it means that they all need to host at least one peer. In turn, this implies that they need to have the competences, resources and manpower to do so. These requirements put a high barrier to the entry in the marketplace, making it suitable only for larger organizations. In practice, this limits the scope of our decentralized marketplace to business to business interactions.
Having understood how to deploy the ledger, the next step was to start implementing the business logic. This is the role of the smart contracts, or chaincodes as they are called in Hyperledger Fabric. They ensure that the state of the ledger is only mutated following pre-defined rules. How to specify and agree on these rules is out of the scope of the blockchain. It is codified in the agreement binding the participants. In Hyperledger Fabric, these restrictions regarding the allowed ledger mutations are enforced using the endorsement mechanism. This mechanism ensures that the smart contracts have been independently executed by multiple organizations, and that they agree on the outcome. To do so, the chaincodes are executed by peers from different organizations and the results of the independent executions (called the endorsements) are then collected and bundled before being recorded to the ledger. For the execution to be deemed valid, all the results must be identical, and the set of peers that have performed the execution must fulfill the endorsement policy.
The endorsement policy is defined by the blockchain administrators and dictates what endorsements are needed for a chaincode invocation to be valid. These requirements can target both the number of endorsements needed, as well as the organizations these endorsements must come from. This policy is key in guaranteeing that the chaincodes are executed properly. If the policy is too weak, it makes it possible for a subset of the participants to execute a modified version of the chaincode and still collect enough endorsements for the execution to be deemed valid. As a result, the endorsement policy needs to be strong enough to guarantee the proper execution of the chaincodes. However, it should also avoid involving too many organizations, as this would negatively impact the performance of the ledger. Balancing these constraints requires the administrators to have deep knowledge of both the platform and the business logic.
The previous sections assume the smart contract perfectly encode the business logic. However, it can happen that, due to a bug for instance, the smart contract behavior differs from the expected business logic. In such a case, it is possible that multiple invalid transactions are accepted, before the issue is identified and fixed. Moreover, even after the issue has been fixed, it might not be straightforward to identify all the transactions that have been impacted and all the members that have been wronged. In such a case, it is crucial for the blockchain participants to have a clear agreement regarding how to handle the situation.
If the code is considered the absolute law, the members that have been wronged are at fault for not auditing the code carefully enough and should not be compensated. On the opposite, if the smart contract developer is considered at fault for not following the specification, they should be liable for any damage due to their mistake. Without such an agreement between the participants, the different blockchain members could end up having different opinions regarding what is to be considered the proper “history”, defeating the purpose of the blockchain.
To summarize, based on the considerations discussed above, we designed an architecture allowing us to leverage the benefits from Hyperledger Fabric (similar to the one presented in figure 1) and implemented the smart contracts regulating the exchange in our marketplace (as presented in our previous blog post). The next step was then to deploy our solution in a production environment. In Hyperledger Fabric, it is straightforward to setup a development system; the project provides many examples of how to do so. A production system is however much more complex. It needs to span multiple organizations and integrate into their existing infrastructure. This requires each organization to perform some integration and administration work both internally and in collaboration with the other blockchain members.
Regarding such administrative tasks, one of the issues we faced with Hyperledger Fabric version 1.4 is that they were not fully covered by the existing tools. For some procedures, it is up to the participants to decide and agree on how they should be performed. For instance, the administration of the blockchain often requires sharing data between the different organizations (for instance cryptographic materials, chaincode packages, and so on). How to securely perform such data exchanges is not specified by Hyperledger Fabric and therefore needs to be devised by the participants. Moreover, if manually performing these administrative tasks is fine for smaller systems, it will become necessary to automate them when the number of participants grows.
As for the integration work, this project made us realize that it is not to be underestimated. For a system to be able to interface with the smart contract hosted on the ledger, it needs to interact with a peer. To that effect, Hyperledger Fabric already provides SDKs for the Node.js and Java languages, and they are further working on Python and Go versions. As a result, it is straightforward to perform this integration when using a language supported by the SDKs. However, when dealing with unsupported languages, the implementation can become much more complicated. The solution often relies on an intermediate component, for instance a shared library or a standalone proxy, to perform the interaction with the peer. That intermediate component is typically written using one of the available SDKs and exposes an interface that is easier for the target system to interact with. However, beyond just adding complexity to the implementation, such a component might also introduce vulnerabilities. Indeed, being on the path between the client and the ledger, it is able to alter the data read from the ledger, modify the client transactions or simply impact availability.
For our use case, the functionality that we felt lacked most in Hyperledger Fabric was a way for the end users to interact with the smart contracts from a web browser, similar to what Metamask provides for Ethereum. Without such a component, it is not possible to provide an end-to-end experience for the end user when solely using a web application. The end user has no choice but to trust the server side to handle the blockchain transactions on their behalf. In practice, this requires them to be in the same security domain, forcing each organization to host their own instance of any web application.
Looking back, this project made us realize that even though the heart of the marketplace functionality is provided by the smart contracts, a lot of work is required to build the environment needed to execute them.
All the participants need to be actively involved to deploy and maintain the distributed ledger that’s needed by the smart contracts, and to integrate them with the rest of their infrastructure.
As a result, at least for now, properly designing and implementing a solution using Hyperledger Fabric is possible, but it requires a lot of competences and resources from all the participants. However, we believe that, as the ecosystem matures, the Hyperledger projects will continue to improve and be supported by an increasing number of tools. This will help make Hyperledger Fabric easier to work with for the developers, administrators and end users.
Read the first blog post of this series: Can we put our trust in a decentralized marketplace?
Find out more about our research into the future of cloud and computing.