Gradual Network Upgrades

[aschmahmann]

A number of large changes to the DHT (e.g. AutoNAT based automatic switching between client and server modes) have occurred in recent months and there are many more ahead (e.g. #558 #559 #584). We need to figure out how to do protocol upgrades in a smooth way, and ideally one with the following properties

1. Small groups can attempt an upgrade without disconnecting from the larger network (i.e. no large coordination costs to propose a network upgrade).
2. Can use this strategy to establish an open testnet that can be upgraded without waiting on an official release
3. Minimal latency addition to queries
4. Minimal machine/network resource utilization to be part of a testnet

Proposal

Nest DHTs inside of each other. For two DHTs A and B insist that every member of A is a member of B (i.e. A is nested inside of B, such as A = /testnet/kad/2.0.0, B = /mainnet/1.0.0). Then we can go through the following procedure (using GetClosestPeers as an example, but applies to the other DHT query types as well)

1. Query /testnet for the closest nodes.
2. Query using the /mainnet protocol where the lookup procedure is seeded from the results of the /testnet query (instead of/in addition to the /mainnet routing table)
   • Optional: It is possible to have this setup such that servers of /testnet are only clients of /mainnet, if so then there needs to be a transfer step where we ask the closest nodes from the /testnet who the closest nodes they know about in /mainnet are. If /testnet servers are also /mainnet servers then this happens automatically.

Puts

Depending on what we want we could do Puts in a couple different ways, such as:

1. Put to both networks (good for search performance in the new network) (my preference)
2. Put to only the main network (assuming that servers in /testnet are servers in /mainnet)
   • Note: We will only start putting to /testnet once it is big enough to automatically get data by virtue of being a server in the main network (good for reducing storage on the network)

Bonuses

Having this setup also gives us some freebies like paving the way for allowing for multiple DHTs with different validators so that a network can have a shared base DHT (e.g. for finding other peers) and then application-specific DHTs layered on top (e.g. for finding/storing application specific content).

[makew0rld]

ipfs/kubo#7169 is related.

[aschmahmann]

@makeworld-the-better-one there may be some relationship between these two issues.

Note though that this proposal assumes that one network is a subset of another (e.g. I first check Yggdrasil for data, then the regular network) and the proposal will cause problems if this is not true (e.g. some nodes use Yggdrasil, some use CJDNS, some use both).

[makew0rld]

Ah okay. Yeah, in the issue I linked it's referring to separate DHTs and networks, not nested ones.