6/30/22

• What retrieval team is - Hannah, Rod, Kyle (make retrieval work in SP network)
  • Evergreen, Saturn
  • Initial short set of projects - finish existing work in data transfer (stability)
• Evergreen
  • Incentive to continue extending slingshot deals
  • Slingshot evergreen
    • Evergreen deal engine (pull based deal engine) - I have this CID, give me a deal for it
    • Here’s a list of CID that we have metadata for (track their current replication on the network)
    • If the number of CIDs is close to 0, make that CID eligible for renewal
      • SP attempts to retrieve data from an existing deal
      • 72 hours to seal
    • Also checks for regional replication
    • Want to build up to 10 replicas per CID
  • Retrieval testing has always been an important part of the program (reward ppl differently based on quality of service)
    • Retrieval success rate w/ Dealbot
    • Continually increased index of weight of how it factors into reward
    • Sample over an extended period of time, ability to get retrievals from an SP
    • In built incentive mechanisms to work with SPs who have good retrieval mechanisms
  • Slingshot v3 design
    • Eligibility of SPs based on retrieval performance (retrieval sampling, Sp to SP retrieval)
      • Embed logging into filclient (SP to SP retrieval)
      • Dealbot for sampling
    • Suspending ppl for periods of time from the deal engine
• One of 2 ways that replicas will be built
  • SP to SP retrievals
  • Slingshot v3
• Issues being reported
  • Likes HTTP retrieval
  • Autoretrieve is a separate workstream
  • Dealbot -> low number of parallel deals (but 5 individual instances)
    • AI: dealbot use case
• SP to SP retrieval

  • Whole piece, should always be free

  • Will be Fil+ data (evergreen goes through KYC flow)

  • Problems that happen in moving of data, and then miner who is actually doing retrievals

    • HTTP retrievals - transferring pieces

  • Some of the top issues we see today 👇

    • SP’s index is actually not configured correctly or unable to report that the CID is actually available to them (”piece not found” and similar issues)
      • SP doesn’t have a record of where the deal is being stored
      • even though it’s on disk, deals are sealed, we see PoST, it cannot be retrieved

  • Scanning + unseal copies → longer term solution?

  • Add mapping for why retrievals have failed / add meaningful error codes in autoretrieve?

    • SPs can take information and correct the problem

  • short term solution → http sidecar service - doesn’t go through deal filtering

    • implement deal filtering so restricted data is not open for all ppl to retrieve
    • SPs can open a new port to serve http/car files (with other files filtered out)

  • Need SPs to be informed on when they need to fix their things (storing unsealed copies etc.) → need visibility into which SPs should be on this list