<aside> 💡 This is copied from an internal document I wrote in June 2021 amongst a focus on scaling the blockchain’s capacity to account for committed storage.
There were some comments on my original document, mainly from @ZX Zhang, which I have not automatically transferred into public view.
</aside>
This is a rough proposal to reduce and simplify the per-sector state accounting cost in order to support higher network capacity. The Filecoin chain state includes a lot of metadata about each sector, most of which is only relevant to deals. Committed-capacity sectors could be made nearly fungible, reducing much of their accounting state to a constant size.
Unfortunately, the need to retain the sealed CID (CommR) for each sector limits the gains. CommR today accounts for just over half of the state, so this proposal gets <2x improvement.
However, this proposal makes those gains by simplifying the state and calculations that we do today, discarding CC sector identities and calculating aggregates instead. These simplifications, or similar ideas, may unlock subsequent improvements that make a bigger difference. Simplifications also simplify reasoning about the current and future protocol, ease migrations to new policies, etc.
Thus, this isn’t a proposal for immediate development, but a sharing of ideas.
See also Filecoin Plus premium and TODO link to Neutron.
The Filecoin network’s growth rate and absolute network size are limited by many linearities involved in onboarding and maintaining storage. In order to unlock very high or exponential growth in committed storage, we need to remove network consensus costs that are linear in storage.
The Filecoin chain state includes information about every sector. A significant amount of per-sector state is relevant only to sectors with deals. A vast majority of sectors, including all incremental network growth today, have no deals. These committed-capacity (i.e. empty) sectors all have the same power but pay similar state & accounting costs as sectors with deals because they share a data schema and accounting process.
Further, a significant amount of per-sector state is maintained for the primary purpose of computing penalties if the sector later faults or is terminated. Every sector has different parameters based on network state when it was committed, deals included, and details of any committed-capacity sector it replaced.
So deals and sector-specific penalties drive heterogeneous sectors, which prevents aggregation and summarisation.
The core idea behind this proposal is to lose the individual identities of committed-capacity sectors as far as possible, maintaining instead an aggregate committed-capacity state for each miner.
Fungible CC sectors do not have individual committed lifetimes, weights, or pledges. They do not expire automatically. Penalties for faulting or terminating a CC sector depend on the miner’s CC aggregate state, not the identity of the sector lost.
Sectors with deals retain their existing schema, although some of the simplifications described below could also be applied.
Per-sector state is reduced to only a sector number, and sealed CID (CommR), totalling about 40 bytes raw data (a little more with AMT overheads). These two items are required today for Window PoSt computation and verification.
Very roughly, a 100GiB state tree could support ~80EiB worth of 32GiB sectors. TODO: better measurements. (Today, a 14.6 GiB state supports 6EiB (200m sectors) → 78 bytes/sector)