The InterPlanetary Network Indexer enables searching for content-addressable data available from storage providers available on the Filecoin and IPFS networks.
The Filecoin IPFS network is vast and depends on connecting a client seeking a file with the location the file is hosted using a CID. To be usable this needs to be fast. The IPNI makes these lookups fast enough to compete with web2 content lookups.
IPFS traditionally has relied on content routing tables which connect peers to content which are stored in a distributed hash table known as the DHT. Because no single node knows which node has which CID’s the DHT leverages Kademlia which helps to manage queries across peers to locate content.
Network Indexers are network nodes that store mappings of content multi-hashes to provider data records.
A client that wants to know where a piece of information is stored can query the indexer, using the CID, or multi-hash of the content, and receive a provider record that tells the client where it can retrieve the content and via which data transfer protocols.
Content can be queried at cid.contact. (Github Repo) or one of the 7 partner indexer node instances listed on the cid.contact page.
Filecoin stores a great deal of data, but without proper indexing of that data, clients cannot perform efficient retrieval. To improve Filecoin content discoverability, Indexer nodes are developed to store mappings of CIDs to content providers for content lookup upon retrieval request.
In order to populate that mapping of content providers and CID’s to an indexer node an Indexer Provider service runs alongside the Storage Provider, which tells the Indexer what content a storage provider has.
The Indexer Provider Service sends updates to the Indexer via a series of Advertisement messages. Each message references a previous advertisement so that as a whole it forms an advertisement chain. The Network Indexer performs the function of consuming this chain from the initial Advertisement to the most recent.
Indexer node enables content discovery within Filecoin network today, it is also a fundamental component toward the end goal of universal retrieval across IPFS/Filecoin, i.e. to be able to efficiently retrieve Filecoin content from IPFS. Combining Indexer work for content discovery with IPFS/Filecoin interoperability work for content transfer, we aim to reach the end state of efficient and universal retrieval across IPFS and Filecoin network.
Project Pages: