Outline

See Direct data onboarding: impacts & work outline for a high level overview of the goals and expected impacts of this project.

Summary of technical changes

• Instead of PublishStorageDeals, a FIL+ allocation is made by a client directly with the Datacap/Verified Registry actors (already possible today).
• New actor methods for activating a sector or committing new data to an existing sector (SnapDeals) accept additional parameters. These new methods facilitate a flow that skips the built-in market actor entirely.
• At pre-commit, an SP must specify a sector’s data commitment (unsealed CID), but does not need to specify the structure of that data nor any deals or FIL+ allocations.
• At activation time, an SP specifies the structure of data that has been committed to the sector, a sequence of piece CIDs and sizes.
  • The SP specifies the IDs of any FIL+ allocations claimed by a piece of data being committed.
  • The SP specifies the IDs of any built-in market deals (later: any market deals) satisfied by a piece of data being committed.
  • Both of these are optional: no allocation or deal ID is required.
• The miner code verifies that the pieces of data are included in the sector’s data commitment, then notifies the verified registry and built-in market actor to claim/activate the associated records.
• The new method parameters make space for notifying any actor about such a data commitment, but actually sending those notifications is deferred to a future change.

Design principles

This design is informed by the following principles

• The existing onboarding flow keeps working with no necessary change from SPs. The existing flow must and can only use the built-in market, in the same way it does today.
• The new flow supports onboarding FIL+ verified data with no interactions with the built-in market (especially, no deal required).
• The new flow supports onboarding non-FIL data with no interactions with either the built-in market or verified registry actors. Participants only pay costs for the facilities they use.
• The new flow can also support the built-in market, so that the old flow is ready to be deprecated once participants migrate. We’re ready to cut the technical debt of having two flows.
• The new flow is ready to support alternative markets with no API changes, though actually allowing them is deferred.

Design

The schemas described below include consideration for a number of other potential, future features of storage and data onboarding. These are mostly represented by placeholder fields in method parameters which are to be initially set to nil, but allow for future specification of their values without requiring SPs to migrate to new method numbers or parameter schemas. These future features are outlined at the end.

Workflows

A sketch of the current verified data onboarding workflow is: