Meta:

• Where is this getting posted: New post on https://discuss.ipfs.tech/
  • ✅ https://discuss.ipfs.tech/t/unanticipated-shutdown-of-hydra-network-bridge-nodes-analyzing-network-impact-and-implementing-direct-ipni-path-for-recovery/16408
• What is the name of the post: “DHT Hydras for Network Bridging have been fully Shut down”
• Post status: Unposted in review
  • Ensure we do a followup to https://discuss.ipfs.tech/t/dht-hydra-peers-dialling-down-non-bridging-functionality-on-2022-12-01/15567/7 pointing to the new post

TL;DR

Recently, it was found that Hydra peers were declining in quantity, and ultimately on 2023-04-20 the last Hydra node stopped providing network bridging to IPNI (the InterPlanetary Network Indexer). Internal studies show that Hydras provided less a ~10% performance gain in the IPFS DHT network. Given the substantial cost of Hydra's operation, Protocol Labs conducted a "Hydra Dial Down" experiment in December, 2022. The result of this experiment showed low risk and impact, and therefore, we have decided to leave the Hydras non functional replacing their bridging function with a direct path to IPNI on 2023-05-04 from popular gateways like ipfs.io/dweb.link. The exciting news is that allowing the Hydras to drawdown was largely unrecognized by the majority of network participants due to the effective redundant content routing systems now in place. In fact in some areas network performance may have even improved!

Pointers

• Results of the ProbeLab Hydra network performance impact study: https://github.com/protocol/network-measurements/blob/master/results/rfm21-hydras-performance-contribution.md
• IPFS Camp 2022 Talk on “The IPFS Network From The Hydra's PoV”: https://youtu.be/zhzxJGoLTg0
• Hydra dialdown discussion from Content Routing WG April 16, 2023 Content Routing WG #9 Brussels Hydra drawdown discussion

Details

Hydras have been serving three primary purposes, but when compared with the cost of operation, they do not provide enough performance advantage to justify continued operation:

1. Faster access to the Provider Records, i.e., reduced latency.
   1. As observed by the hydra dialdown study by ProbeLab, p95 traffic impact is ~12% slower in terms of TTFB.
2. Availability of Provider Records, given that Hydras are generally stable and online, whereas normal DHT nodes churn (i.e., if all 20 peers where the Provider Record is stored go offline, the content becomes unreachable).
   1. According to another one of ProbeLab’s studies on the Liveness of Provider Records in the IPFS network, it turns out that provider records stay available in the network regardless of Hydra’s presence for longer than the “reprovide interval” (see Section 4.3 of the relevant report).
3. A “bridge” to services such as InterPlanetary Network Indexers (IPNIs) like cid.contact which advertise content from large providers like web3.storage and Filecoin nodes.
   1. The necessity for this bridge has been mitigated now that IPFS implementations like Kubo have defaulted to querying cid.contact in parallel to the DHT since 0.18 [1]. Popular HTTP gateways like ipfs.io/dweb.link have enabled this dual querying of the DHT and cid.contact.

Timeline

Source

On March 18, 2023, we encountered a problem where replacement Hydra nodes could not start due to a Docker credential issue. This resulted in a steady decrease in the total number of Hydra nodes available on our network.