This is GOSH’s answer to the blog post: “sigstore, blockchain vs transparency logs” co-authored by Luke Hinds (Red Hat) and Prof Santiago Torres-Arias (Purdue University).
We will address some limitations of the decision they discuss and how it affects solutions built on top of Sigstore, like Chainguard.
At GOSH we chose the blockchain. As we will see further on, the reasons for this decision are similar to the reasons cited by Sigstore in arguing against the blockchain. Indeed, we agree 100% with every point discussed in the Sigstore blog post. We completely understand all the reasons behind their decision. Moreover, if we were in Sigstore’s place, having the same circumstances, the same experience and background, we too may have decided not to use the blockchain. That would not make it the right decision overall, but we are not here to denounce the decision that works for them.
Below we will be analysing point-by-point Sigstore’s argumentation, their decision process, and adding some perspective to their claims. We hope that, towards the end, you better understand our solution. One where the blockchain is placed front and centre of Software Supply Chain Security.
The main reason we reach opposite conclusions, despite the same facts, lies in the expertise of the people behind our respective projects. While brilliant engineers at Sigstore are great at what they are doing, they are not blockchain engineers or architects. They look at the blockchain in its current state and inevitably come to the conclusion that it is not at all suited to their undertaking, and what they need to accomplish. Which, again, is completely true.
“While our assessment is that blockchains do not fit the bill (now), we wouldn’t discard a constantly-evolving and constantly-innovating field forever” — concludes their post.
Here at GOSH we come from a different background. Most of GOSH’s core team are coming from EverX, where the last 5 years were spent building Everscale, one of the most scalable and innovative blockchains to date. GOSH core development team members have led the design and implementation of radical blockchain architecture, including consensus protocols, governance models, tokenomics, and complex smart contract system research. We are trained to solve problems on the blockchain. We know how to build architecture to allow for challenging use cases. And securing the software supply chain starting from the source code is probably the most interesting real-world blockchain use case since it began. Tackling this challenge is why we are here.
Tokens & Fees
“The first issue is the need for a token and its related fees.
For a public blockchain a token of some sort is required to transact on the chain. A majority of the time these tokens are acquired as part of an initial ‘giveaway’, conversion of fiat money via some sort of exchange or via the use of compute power (in the case of proof-of-work).
With sigstore the key tenet of the service is that it is ‘free’ at point of use.
Typically fees are required and a general trend exists where the more popular the chain ecosystem becomes, the higher the token value inflates.”
There are a few separate issues the Authors discuss here, all of which we completely agree with. But before addressing them, let’s put them in order. All public blockchains require some sort of token (usually called “native token”) which one needs, even if only to pay for gas (actually, as we later discuss, the main reason for native tokens is quite different). There are two reasons blockchains need gas:
To pay validators (or miners, in proof of work blockchains) – people who run specialised hardware that create and validate blocks of transactions and other meta information
To prevent spam (the gas price in this respect is a payment for the time one can expect the blockchain to work exclusively on processing this user’s transaction)
It is certainly true that native tokens should have value to ensure the security of the blockchain. This is because security here is based on the fact that someone may lose considerable value if they produce something faulty or otherwise incorrect on the blockchain. A guarantee measured in its value. However, this does not at all mean that transactions need necessarily cost something as well, as the Authors suggested. There is no security guarantee in the fact that a user pays for transactions. These are disconnected.
Let’s return to that point later and concentrate on the notion of fees themselves.
Most blockchains require fees to process transactions. They therefore cannot accommodate “free” services. All while there is a common agreement that a free service is a highly desirable feature for a mass adoption of any digital product.
There are several blockchains which have attempted to tackle this issue before. EOS pioneered the “bond” idea where developers would put a bond of native tokens and use blockchain for free. This did not work out well, as it required developers to incur quite heavy upfront costs. However, the thought, hatched by EOS creator Dan Larimer, that a blockchain does not necessarily require constant fees for processing transactions is, in itself, liberating. Later, several blockchains tried to lower their fees to the point where payments would be negligible. The most notable of these is Solana. Yet even their results are unsatisfactory. This is for two reasons: the fees are still present, thereby preventing the “free model”, while simultaneously being too low to prevent spam (which became apparent after many DDOS attacks on Solana).
One of the requirements of the GOSH blockchain’s tokenomics was to:
Allow for a completely free service
While preventing spam
Describing the solution in detail here would take too long (those interested can take a look at our tokenomics instead), but the general concept is as follows:
There is a smart contract address space in GOSH. It is determined by the hash of the smart contract code (let’s just point out that GOSH is a content addressable blockchain), and it is called the “Free Service Area.” In it, all smart contract executions are subsidised completely by the network validators. Rather, instead of fees, collected on the network, going to validators, they go to a ‘Giver’ which subsidises these smart contract executions. However, these contracts have some restrictions: one of which is that they can not transact value outside of the Area. Meaning they can only transact for free between themselves. This allows for the creation of a completely free user experience without generating spam, because the execution of these contracts are only available within the service they provide, i.e. if a smart contract is a Git commit, it can only transact with commits of the particular repository. Much like Github commits are free, but the spam attack on infrastructure using it is very costly, and without an obvious gain.
The last point about the inflation of token price, despite being a generally correct observation, is no longer relevant because it’s free, and free is not inflationary. Likewise, GOSH has two “native” tokens, instead of one. The “GOSH” token is a value-capture token. The token which pays for gas in a smart contract outside the “Free Service Area” is called “SHELL” and it is an algorithmic stable coin, and the value against the dollar does not inflate in the case of a stable coin.
The rest of points that the Authors make in this section are related to permissioned blockchains like Hyperledger with which we, again, are in complete agreement: private blockchains are insecure by nature and defeat the purpose for which the blockchain was invented in the first place.
As for the considerations of factors such as a 51% attack: this is very rare. In the whole history of the blockchain industry, and even when successfully conducted, they are usually successfully reverted (see Ethereum Classic attack). Vitalik tweeted about 51% on POS here.
✅ Free service on the blockchain — DONE
✅ Non inflationary token on the blockchain — DONE
Private Key (Wallet) management
“The truth is the majority do not want to deal with the headache and cost of storing private keys securely. This is why cryptographic signing adoption is so low in open source communities (typically lower than 5%). With blockchain a key pair is required (a wallet). This is a problem for the following reasons:
OSS contributors are often unfunded and not employed to develop open source. Quite often they could be from a developing nation or somehow disadvantaged financially (most students right?!). It is not fair to expect them to spend upwards of $100 on buying a hardware wallet.
Likewise we cannot expect users to store their wallet and passphrase in a cloud service.”
Blockchain cryptography is very strong and very well developed. One can just look at the amazing advances in Zero Knowledge Proofs for perspective. The reason for this is precisely because blockchain provides resources for such advanced research. Something which, unfortunately, cannot otherwise be said of open source. It is because of this that we believe blockchain tokenization is the only valuable business model for the whole of OSS, not only Software Supply Chain Security.
However, OSS developers don’t need to wait for solutions to the open source business model. They can buy a hardware wallet for very cheap even now. EverX produces one for less than $10. A single chip secures the hardware element which has not only ed25519 support inside for GOSH blockchain cryptography, but also supports end-to-end supply chain protection against man-in-the-middle and other threats starting at the manufacturing facility.
At the blockchain level there is a formally verified multisignature wallet smart contract which handles up to 32 signatures, combined with a mobile application, managing the access and recovery of a lost key.
Contrary to this there’s: “use Facebook to log in and sign the supply chain.” This just doesn’t sound right to us. The signatures one uses for Software Supply Chain security should be, well, secure.
✅ Formally verified multisignature wallet — DONE
✅ Negligibly cheap hardware keystore — DONE
Controversy
The truth is that blockchain makes for a controversial topic. Post to any forum about blockchain (save the ones catering to those holding crypto-currency) and you will find a very flammable mix of views. One group feels it is the answer to everything, the other feels it’s no more than a Ponzi scheme. Those in the middle are typically undecided and often drowned out.
We agree. The blockchain is inundated with controversy. And, though much of it is unfounded, some of it does ring true. As a result of this, we believe it important to try and look at it in a sober way.
Blockchain may not be the answer to everything, but we are making the case here that blockchain is the best solution to securing the Software Supply Chain. And if a solution works, we ask that people not be so quick to dismiss it simply because it is controversial.
Production Ready
The truth is we would have been in a ‘prototype’ stage for a lot longer if using blockchain, while we gained the confidence to have sigstore run in production.
As we have said, production readiness is all down to the expertise of a particular team, and not to the technology itself. Seeing as the blockchain technology is a mature one by now, a team with years of experience in developing it, debugging it, and adapting it to different tasks should naturally be able to achieve production readiness sooner. GOSH has an additional advantage: it is an evolution of what is already the most scalable blockchain architecture in history, Everscale.
The underlying architecture of Everscale had to be adapted for the specific purpose of storing git on-chain. Meaning GOSH is the only blockchain capable of storing git on-chain, with potentially hundreds of millions of objects in repositories.
✅ Production ready scalable blockchain tech — DONE
Decentralisation
One clear advantage a blockchain has is its distributed / decentralised architecture. However when we dig into the current developer landscape we often find it’s not decentralised. As mentioned under the Token heading, permissioned / token-less blockchains require gateways for access-control , canonicalization etc. dApps use centralised API access from providers
After the DAO hack in ethereum, a hard fork by developers created wide controversy about the other “angles” of centralisation that may exist: in this case, the fact that a series of developers can — and have — effectively thrown away the protocol-level decentralisation.
Many posts, including some by Vitalik Buterin, have discussed the issue of blockchain decentralisation. While in general a validity point (i.e. if we consider use cases for a public blockchain, it must be decentralised) it has an embedded deception. We’d like to note that decentralisation is not a binary state. The network is not either decentralised or not. It’s a process that has never reached 100% completion, i.e. decentralisation is not a finite state.
When considering the Authors’ argument as to why they chose transparent logs, we would like to give an argument in favour of the blockchain in terms only of security:
In its 13 years of existence, no major blockchain network has been hacked. The DAO hack mentioned by the Authors is:
not a blockchain hack, but rather a smart contract implementation hack. This is the reason why all system smart contracts in GOSH are formally verified.
Ethereum at that point was not a major cryptocurrency network by today’s standards. And what happened is a quite democratic and interesting result: there were now two networks. Participants decided which to support. Forked Ethereum clearly won.
But let’s apply this case to the Secure Software Supply Chain. In the case of today’s software supply chains the damage is immensely hard to reverse. The recent attacks on logistic providers show the damage takes months to be patched. Perhaps then, a blockchain fork agreed by all token holders would be a much easier and cleaner solution in such a case?
We agree, yet again, with the Authors’ point of view regarding the centralization of blockchain APIs. Moreover the more performant the blockchain becomes the harder it is to run an access node. Yet, on GOSH, techniques exist that may allow an application developer to run a blockchain node without incurring huge costs. In GOSH in order to prove a state one needs only to hold a few key blocks and blocks from the last (short) epoch of a particular shard, where the subject smart contracts are located. Now every transaction can be proved cryptographically without a need to hold or process all blocks. The proof itself can be performed quite inexpensively on a smartphone, making it a non issue how centralised the access infrastructure is by itself. In general though, blockchains are just better at providing fault tolerance.
✅ Secure recoverable secret — DONE
✅ Sufficiently Decentralised Blockchain — DONE
Why We Didn’t Choose Transparent Logs
Fault tolerance problems
Logs stored on servers are centralised by nature. This means that, by inherent design, all their critical operational infrastructure relies on many potential points of failure.
Let’s take a look at Github itself. In a Github outage, users are unable to verify their certificates. This is one of the key reasons why GOSH is an implementation of Git on the blockchain and not just a hash storage service.
Fucio is another point of failure. As it is a certificate rotating tree: if the service halts — there is no access to OpenID secrets and users cannot sign certificates.
Problem of trust
The transparency log cannot be altered unless the root is altered. But there is no trustless infrastructure to independently verify this. As a user, you need to trust the certificate.
In turn, if the registry is out of service — users are unable to even download the manifest.
Without the blockchain present there are just too many critical points of failure. After all, this is what blockchain and distributed computing was invented to solve.
Record ordering
However, there is one point of paramount importance: only on the blockchain is it possible to prove ‘order of time.’ This makes transparency logs, as well as any other technology built for this purpose, limited in their use case and much less secure. So much so that once we solved all the issues mentioned above, blockchain became the most plausible ready-to-use solution to securing the Software Supply Chain. That’s what GOSH is doing right now.
There are a few things that need correcting here:
"Contrary to this there’s: “use Facebook to log in and sign the supply chain.” This just doesn’t sound right to us"
You're not using facebook to sign the supply chain. The signing operation uses ecdsa. You're exchanging an ID-Token for a OIDC scope where OIDC is simply a form of identity and is one of many supported. . sigstore also supports multiple key mgmt approaches. Keys can be stashed in a cloud KMS (azure, GCP, AWS, Vault, k8s secrets) or created within a PCKS11 hw dongle, you could even stick them on the blockchain if you want.
"Logs stored on servers are centralised by nature. This means that, by inherent design, all their critical operational infrastructure relies on many potential points of failure."
Not really true, there are multiple approaches to dealing with points of failure, rekor has a sharding API should a log become corrupted. Rekor uses trillian , which is the same backend as used by CTL's and those things are critical infrastructure and there is plenty of operational prior art. CTL's are run by cloudflare, google and facebook : https://certificate.transparency.dev/howctworks/
"Fucio is another point of failure. As it is a certificate rotating tree: if the service halts — there is no access to OpenID secrets and users cannot sign certificates."
Fulcio has no data store and does not persist any certs. There is no certificate rotating tree is, certs are short lived and stored in the tlog.
"However, there is one point of paramount importance: only on the blockchain is it possible to prove ‘order of time.’"
We have a timestamping service
"In turn, if the registry is out of service — users are unable to even download the manifest."
Not true, sigstore provides offline bundles which folks already use in airgapped environments. The DoD are leveraging this approach via Rancher Government Solutions
There is a lot more wrong here, but those are the most obvious to me.
Like we said in the original article, if someone can come up with a compelling blockchain PoC, we would love to see it. That's the great thing about OSS, anyone can contribute code and share it with others.