Moved to Github

After the initial project planning using Notion and gathering feedback from the community, the project has now moved into the Lotus Github Repo for the rest of the project phases.

Check the 🗺 Mega Lotus-Miner project issue and Github Project tracker:

Lotus-Miner-V2 • filecoin-project

Summary

<aside> 📄 We define enterprise storage providers as storage providers who focus on onboarding data with 10+ PiB raw storage and beyond, or SPs that aspire to have that amount of deal storage capacity.

Taking in storage deals and serving retrievals is a requirement to be considered an enterprise storage provider.

</aside>

Problems we identified:

• Deal/sealing task scheduling inefficiencies. These compound for storage providers wanting to scale fast, and/or that needs to have a high deal/sealing throughput.
• Lotus-Miner does not support redundancy for critical operations like PoSt.
• The Sector storage subsystem is not stable and performant at scale, its also not able to handle mutable sectors (future proof). Under pressure (or crashes/abrupt shutdowns) the sectors storage subsystem can cause failure of reporting what state a sector is actually in, which makes monitoring and troubleshooting hard.
• Managing multiple MinerIDs - A common way of managing funds from investors and lending-platforms (also risk management), is to separate funds/ownership between different MinerIDs. Managing lotus-worker and storage servers between MinerIDs is both time consuming and not very flexible
• No monitoring systems are shipped with the Lotus-Miner to observe the health and condition of their setups.

Community feedback

We gathered all the high-level problem areas we identified and potential solutions we looked into in an open Github discussion. We engaged with the Storage Provider Working Groups and Decentralised Storage Alliance to get their feedback and to surface any other areas that are problematic for storage providers. We promoted the post to the broader community through relevant Slack-channels / SP-Newsletters / Lotus-Twitter:

Link to Github discussion: Architectural/System-level problems for enterprise-level storage providers

Summary of feedback:

• 2022.11.27 FIL-SP-STACK SitRep:
• 2022.12.05 FIL-SP-STACK SitRep:
• 2022.12.12 FIL-SP-STACK SitRep