📃 Note: All PR’s mentioned will be merged in the upcoming Lotus release v1.19.0 and Boost release v1.5.0

Issue Overview:

*Domino effect, fixing 1 issue resolved another.

Highlights

Issue #9478 :

Problem - Unable to snap-up a sector again. If something went wrong during sealing and the upgrade was aborted, the user was unable to use that sector for another snap-upgrade.

Solution: Clear the Creation Time, in case the sector had accepted piece data previously. More info in the PR.

PR: https://github.com/filecoin-project/lotus/pull/9642

Issue #8496 : Problem - Error messages on loop during an open deadline. The reason for the error messages was because they where trying seal/Finalize the upgraded sector in a the open deadline. Which is impossible, so the user experienced a lot of retries and error messages until the deadline closed.

Solution - By creating a new WaitMutable state - now if the deadline is open and the sectors are trying finalize they will wait on the worker until the deadline has closed. Important to note that they will not finalize as soon as the deadline closes, they will wait 1h before continuing.

PR: https://github.com/filecoin-project/lotus/pull/9598

Issue #8491 : Problem - Sector files remains on workers, if a sector upgrade is aborted. If a sector reverts back to Proving during sealing, the files are left on the worker. Lotus-miner sectors remove will only delete the files for the Proving sector.

Solution - Added new sealing state FUS (FinalizeUnSealed) This will remove the unsealed files from all storage, and makes sure sealed/unsealed files only exist in long-term-storage. Note: we're not keeping anything unsealed because we're reverting to CC. So the deal will need to be sent/imported and sealed again.

PR: ‣

Snap Sealing Optimization:

For Snap Upgrades we recommend the following flow:

Example 1: Perfect for SPs with that have over 10k+ CC sectors, want’s onboard data fast, and want a pure snap upgrade sealing pipeline.

• Option 1; Turn off the PC1 workers, convert the PC2/C2 workers to AP / RU / PRU2
• Option 2; PC1 workers are converted to AP workers, and PC2 / C2 to RU / PRU2

**Example 2: ”**The combo” for SPs that want to create new CC sectors and onboard data at the same time.

Example 3:

From our internal testing this setup had the worst throughput, took the the longest, and resulted in a lot of abort upgrades. So for smaller SPs with limited number of workers, our recommendation is to choose Example 1. Either create new sectors or if you have a lot CC’s - convert the PC2/C2 workers to RU / PRU2 workers until all CC’s have deals.

(We know that every sealing pipeline is unique, these examples are just recommendations.)

Benchmark for snap-up

Sealing environments:

FIL_PROOFS_USE_GPU_COLUMN_BUILDER=1
FIL_PROOFS_USE_GPU_TREE_BUILDER=1
RUST_GPU_TOOLS_CUSTOM_GPU="NVIDIA GeForce RTX 4090:16384”
FIL_PROOFS_USE_MULTICORE_SDR=1
FIL_PROOFS_MULTICORE_SDR_PRODUCERS=3
FIL_PROOFS_SDR_PARENTS_CACHE_SIZE=1073741824