We want to be able to perform breaking protocol upgrades to the DHT without having to create a new DHT and to duplicate the content and the network traffic.
The rationale of the design is that there is a minimal set of features that all protocol versions must implement in order to communicate with each other. Protocol versions can then implement additional features. Each peer advertise its version, or feature set, and will prioritise nodes with the same version or feature set in its own routing table.
FindPeer RPC (find closest peer(s) to key)
Mutable/flexible peerID format
mutable/flexible provider records/peer records/data. The peer records could contain information about the supported dht protocols, and provider records information about data transfer protocols
The scenario assumes that the features are linear, namely the protocol versions can simply be sorted by protocol version, or by release date.
When a peer is aware of more remote nodes whose identity fits inside a bucket, than the size of this bucket, the node will keep in priority remote nodes whose version is the same as itself.
This implies that nodes running the same protocol version will be strongly connected with each other, while being able to reach all the nodes in the network (running different protocol versions).
The vanilla Kademlia keyspace can be represented as a k-dimensional space, with k=256. The linear upgradable DHT keyspace can be represented as a (k+1)-dimensional space, where the last dimension is a distance tie-breaker.
Initial assumption: Kademlia DHT’s routing is sound
After a node has joined a well-behaving network running Kademlia DHT protocol, for instance using bootstrap nodes, it is able to lookup and find any other peer participating in the network using its name (Kademlia 256-bit address).
Global Routing Soundness
The Upgradable DHT Global Routing Soundness is defined as the capability of all peers to lookup and find any other peer in the network, using a globally supported RPC (such as FindPeer).
All peers can reach every other peer in the network, because the linear Upgradable DHT is a special case of the Kademlia DHT routing. Each nodes’ bucket has exactly the same number of peers as the Kademlia DHT, but when the buckets have too many candidates, the Upgradable DHT indicates which peers are selected to be inside the bucket. Hence, if Kademlia DHT routing is sound for all possible buckets combinations, then the Upgradable DHT Global Routing is also sound.
Overlay Routing Soundness