libp2p.kad_dht package
Subpackages
Submodules
libp2p.kad_dht.kad_dht module
Kademlia DHT implementation for py-libp2p.
This module provides a complete Distributed Hash Table (DHT) implementation based on the Kademlia algorithm and protocol.
- class libp2p.kad_dht.kad_dht.DHTMode(value)
Bases:
EnumDHT operation modes.
- CLIENT = 'CLIENT'
- SERVER = 'SERVER'
- class libp2p.kad_dht.kad_dht.KadDHT(host: IHost, mode: DHTMode, enable_random_walk: bool = False, validator: NamespacedValidator | None = None, validator_changed: bool = False, protocol_prefix: TProtocol = '/ipfs', enable_providers: bool = True, enable_values: bool = True)
Bases:
ServiceKademlia DHT implementation for libp2p.
This class provides a DHT implementation that combines routing table management, peer discovery, content routing, and value storage.
Optional Random Walk feature enhances peer discovery by automatically performing periodic random queries to discover new peers and maintain routing table health.
- Example:
# Basic DHT without random walk (default) dht = KadDHT(host, DHTMode.SERVER)
# DHT with random walk enabled for enhanced peer discovery dht = KadDHT(host, DHTMode.SERVER, enable_random_walk=True)
- async add_peer(peer_id: ID) bool
Add a peer to the routing table.
params: peer_id: The peer ID to add.
Returns
- bool
True if peer was added or updated, False otherwise.
- apply_fallbacks() None
Apply fallback validators if not explicitely changed by the user
This sets default validators like ‘pk’ and ‘ipns’ if they are missing and the default validator set hasn’t been overridden.
- async find_providers(key: str, count: int = 20) list[PeerInfo]
Reference to provider_store.find_providers for convenience.
- get_routing_table_size() int
Get the number of peers in the routing table.
Returns
- int
Number of peers.
- async get_value(key: str, quorum: int = 0) bytes | None
Retrieve a value from the DHT.
- Args:
key: String key (will be converted to bytes for lookup) quorum: Minimum number of valid peer responses required for confidence. If quorum > 0 and not met, the function still returns the best value found (if any) but logs a warning. Set to 0 to disable quorum checking.
- Returns:
The value if found (best value even if quorum not met), None otherwise. Note: When quorum is not met, a warning is logged but the best available value is still returned. This allows graceful degradation when the network has insufficient peers.
- get_value_store_size() int
Get the number of items in the value store.
Returns
- int
Number of items.
- async handle_stream(stream: INetStream) None
Handle an incoming DHT stream using varint length prefixes.
- is_random_walk_enabled() bool
Check if random walk peer discovery is enabled.
Returns
- bool
True if random walk is enabled, False otherwise.
- async put_value(key: str, value: bytes) None
Store a value in the DHT.
- Args:
key: String key (will be converted to bytes for storage) value: Binary value to store
- Raises:
InvalidRecordType: If no validator is registered for the key’s namespace ValueError: If trying to replace a newer value with an older one
- register_validator(namespace: str, validator: Validator) None
Register a custom validator for a specific namespace.
This allows storing and retrieving values with custom namespaced keys (e.g., /myapp/key). The validator will be used to validate values before storing and after retrieval.
- Args:
namespace: The namespace string (e.g., “myapp” for keys like /myapp/key) validator: A Validator instance with validate() and select() methods
- Example:
- class MyValidator(Validator):
- def validate(self, key: str, value: bytes) -> None:
# Custom validation logic pass
- def select(self, key: str, values: list[bytes]) -> int:
return 0 # Return index of best value
dht.register_validator(“myapp”, MyValidator()) await dht.put_value(“/myapp/my-key”, b”my-value”)
- set_namespace_validator(ns: str, val: Validator) None
Adds a validator under a specofic namespace to the current DHT config.
Raises an error if the current validator is not a NamespacedValidator
- set_validator(val: NamespacedValidator) None
Set a custom validator for the DHT config.
This marks the validator as explicitly changed, so the default validators (pk and ipns) will not be automatically applied later.
libp2p.kad_dht.peer_routing module
Peer routing implementation for Kademlia DHT.
This module implements the peer routing interface using Kademlia’s algorithm to efficiently locate peers in a distributed network.
- class libp2p.kad_dht.peer_routing.PeerRouting(host: IHost, routing_table: RoutingTable)
Bases:
IPeerRoutingImplementation of peer routing using the Kademlia algorithm.
This class provides methods to find peers in the DHT network and helps maintain the routing table.
- async find_closest_peers_network(target_key: bytes, count: int = 20) list[ID]
Find the closest peers to a target key in the entire network.
Performs an iterative lookup by querying peers for their closest peers. If the routing table has fewer peers than MIN_PEERS_THRESHOLD, it falls back to using connected peers first, then peers from the peerstore if needed, to gather up to ‘count’ initial query targets.
Returns
- list[ID]
Closest peer IDs
libp2p.kad_dht.provider_store module
Provider record storage for Kademlia DHT.
This module implements the storage for content provider records in the Kademlia DHT.
- class libp2p.kad_dht.provider_store.ProviderRecord(provider_info: PeerInfo, timestamp: float | None = None)
Bases:
objectA record for a content provider in the DHT.
Contains the peer information and timestamp.
- class libp2p.kad_dht.provider_store.ProviderStore(host: IHost, peer_routing: Any | None = None)
Bases:
objectStore for content provider records in the Kademlia DHT.
Maps content keys to provider records, with support for expiration.
- add_provider(key: bytes, provider: PeerInfo) None
Add a provider for a given content key.
- Parameters:
key – The content key
provider – The provider’s peer information
Returns
None
- async find_providers(key: bytes, count: int = 20) list[PeerInfo]
Find content providers for a given key.
- Parameters:
key – The content key to look for
count – Maximum number of providers to return
Returns
- List[PeerInfo]
List of content providers
- get_provided_keys(peer_id: ID) list[bytes]
Get all content keys provided by a specific peer.
- Parameters:
peer_id – The peer ID to look for
Returns
- List[bytes]
List of content keys provided by the peer
- get_providers(key: bytes) list[PeerInfo]
Get all providers for a given content key.
- Parameters:
key – The content key
Returns
- List[PeerInfo]
List of providers for the key
libp2p.kad_dht.routing_table module
Kademlia DHT routing table implementation.
- class libp2p.kad_dht.routing_table.KBucket(host: IHost, bucket_size: int = 20, min_range: int = 0, max_range: int = 115792089237316195423570985008687907853269984665640564039457584007913129639936)
Bases:
objectA k-bucket implementation for the Kademlia DHT.
Each k-bucket stores up to k (BUCKET_SIZE) peers, sorted by least-recently seen.
- async add_peer(peer_info: PeerInfo) bool
Add a peer to the bucket. Returns True if the peer was added or updated, False if the bucket is full.
- get_peer_info(peer_id: ID) PeerInfo | None
Get the PeerInfo for a given peer ID if it exists in the bucket.
- get_stale_peers(stale_threshold_seconds: int = 3600) list[ID]
Get peers that haven’t been pinged recently.
params: stale_threshold_seconds: Time in seconds params: after which a peer is considered stale
Returns
- list[ID]
List of peer IDs that need to be refreshed
- key_in_range(key: bytes) bool
Check if a key is in the range of this bucket.
params: key: The key to check (bytes)
Returns
- bool
True if the key is in range, False otherwise
- peer_id_in_range(peer_id: ID) bool
Check if a peer ID is in the range of this bucket.
params: peer_id: The peer ID to check
Returns
- bool
True if the peer ID is in range, False otherwise
- refresh_peer_last_seen(peer_id: ID) bool
Update the last-seen timestamp for a peer in the bucket.
params: peer_id: The ID of the peer to refresh
Returns
- bool
True if the peer was found and refreshed, False otherwise
- class libp2p.kad_dht.routing_table.RoutingTable(local_id: ID, host: IHost)
Bases:
objectThe Kademlia routing table maintains information on which peers to contact for any given peer ID in the network.
- async add_peer(peer_obj: PeerInfo | ID) bool
Add a peer to the routing table.
- Parameters:
peer_obj – Either PeerInfo object or peer ID to add
Returns
bool: True if the peer was added or updated, False otherwise
- cleanup_routing_table() None
Cleanup the routing table by removing all data. This is useful for resetting the routing table during tests or reinitialization.
- find_bucket(peer_id: ID) KBucket
Find the bucket that would contain the given peer ID.
- Parameters:
peer_id – The peer ID to find a bucket for
Returns
KBucket: The bucket for this peer
- find_local_closest_peers(key: bytes, count: int = 20) list[ID]
Find the closest peers to a given key.
- Parameters:
key – The key to find closest peers to (bytes)
count – Maximum number of peers to return
Returns
List[ID]: List of peer IDs closest to the key
- get_peer_ids() list[ID]
Get all peer IDs in the routing table.
Returns
- param List[ID]:
List of all peer IDs
- get_peer_info(peer_id: ID) PeerInfo | None
Get the peer info for a specific peer.
- Parameters:
peer_id – The ID of the peer to get info for
Returns
PeerInfo: The peer info, or None if not found
- get_peer_infos() list[PeerInfo]
Get all PeerInfo objects in the routing table.
Returns
List[PeerInfo]: List of all PeerInfo objects
- get_stale_peers(stale_threshold_seconds: int = 3600) list[ID]
Get all stale peers from all buckets
- params: stale_threshold_seconds:
Time in seconds after which a peer is considered stale
Returns
- list[ID]
List of stale peer IDs
- peer_in_table(peer_id: ID) bool
Check if a peer is in the routing table.
- Parameters:
peer_id – The ID of the peer to check
Returns
bool: True if the peer is in the routing table, False otherwise
- libp2p.kad_dht.routing_table.key_to_int(key: bytes) int
Convert a 256-bit key to an integer for range calculations.
- libp2p.kad_dht.routing_table.peer_id_to_key(peer_id: ID) bytes
Convert a peer ID to a 256-bit key for routing table operations. This normalizes all peer IDs to exactly 256 bits by hashing them with SHA-256.
- Parameters:
peer_id – The peer ID to convert
- Returns:
32-byte (256-bit) key for routing table operations
libp2p.kad_dht.utils module
Utility functions for Kademlia DHT implementation.
- libp2p.kad_dht.utils.bytes_to_multibase(data: bytes, encoding: str | None = None) str
Convert bytes to multibase-encoded string.
- Parameters:
data – Bytes to encode
encoding – Encoding to use. When None the process-wide default from
libp2p.encoding_configis used.
- Returns:
Multibase-encoded string
- libp2p.kad_dht.utils.create_key_from_binary(binary_data: bytes) bytes
Creates a key for the DHT by hashing binary data with SHA-256.
params: binary_data: The binary data to hash.
Returns
bytes: The resulting key.
- libp2p.kad_dht.utils.maybe_consume_signed_record(msg: Message | Peer, host: IHost, peer_id: ID | None = None) bool
Attempt to parse and store a signed-peer-record (Envelope) received during DHT communication. If the record is invalid, the peer-id does not match, or updating the peerstore fails, the function logs an error and returns False.
Parameters
- msgMessage | Message.Peer
The protobuf message received during DHT communication. Can either be a top-level Message containing senderRecord or a Message.Peer containing signedRecord.
- hostIHost
The local host instance, providing access to the peerstore for storing verified peer records.
- peer_idID | None, optional
The expected peer ID for record validation. If provided, the peer ID inside the record must match this value.
Returns
- bool
True if a valid signed peer record was successfully consumed and stored, False otherwise.
- libp2p.kad_dht.utils.multibase_to_bytes(multibase_str: str) bytes
Convert multibase-encoded string to bytes.
- Args:
multibase_str: Multibase-encoded string
- Returns:
Decoded bytes
- Raises:
multibase.InvalidMultibaseStringError: If string is not valid multibase multibase.DecodingError: If decoding fails
Calculate the number of prefix bits shared by two byte sequences.
params: first: First byte sequence params: second: Second byte sequence
Returns
int: Number of shared prefix bits
- libp2p.kad_dht.utils.sort_peer_ids_by_distance(target_key: bytes, peer_ids: list[ID]) list[ID]
Sort a list of peer IDs by their distance to the target key.
params: target_key: The target key to measure distance from params: peer_ids: List of peer IDs to sort
Returns
List[ID]: Sorted list of peer IDs from closest to furthest
libp2p.kad_dht.value_store module
Value store implementation for Kademlia DHT.
Provides a way to store and retrieve key-value pairs with optional expiration.
- class libp2p.kad_dht.value_store.ValueStore(host: IHost, local_peer_id: ID)
Bases:
objectStore for key-value pairs in a Kademlia DHT.
Values are stored with a timestamp and optional expiration time.
- cleanup_expired() int
Remove all expired values from the store.
Returns
- int
The number of expired values that were removed
- get(key: bytes) Record | None
Retrieve a value from the DHT.
Parameters
- keybytes
The key to look up
Returns
- Optional[bytes]
The stored value, or None if not found or expired
- has(key: bytes) bool
Check if a key exists in the store and hasn’t expired.
Parameters
- keybytes
The key to check
Returns
- bool
True if the key exists and hasn’t expired, False otherwise
- put(key: bytes, value: bytes, validity: float = 0.0) None
Store a value in the DHT.
- Parameters:
key – The key to store the value under
value – The value to store
validity – validity in seconds before the value expires. Defaults to DEFAULT_TTL if set to 0.0.
Returns
None
libp2p.kad_dht.pb
Module contents
Kademlia DHT implementation for py-libp2p.
This module provides a Distributed Hash Table (DHT) implementation based on the Kademlia protocol.
- class libp2p.kad_dht.KadDHT(host: IHost, mode: DHTMode, enable_random_walk: bool = False, validator: NamespacedValidator | None = None, validator_changed: bool = False, protocol_prefix: TProtocol = '/ipfs', enable_providers: bool = True, enable_values: bool = True)
Bases:
ServiceKademlia DHT implementation for libp2p.
This class provides a DHT implementation that combines routing table management, peer discovery, content routing, and value storage.
Optional Random Walk feature enhances peer discovery by automatically performing periodic random queries to discover new peers and maintain routing table health.
- Example:
# Basic DHT without random walk (default) dht = KadDHT(host, DHTMode.SERVER)
# DHT with random walk enabled for enhanced peer discovery dht = KadDHT(host, DHTMode.SERVER, enable_random_walk=True)
- async add_peer(peer_id: ID) bool
Add a peer to the routing table.
params: peer_id: The peer ID to add.
Returns
- bool
True if peer was added or updated, False otherwise.
- apply_fallbacks() None
Apply fallback validators if not explicitely changed by the user
This sets default validators like ‘pk’ and ‘ipns’ if they are missing and the default validator set hasn’t been overridden.
- async find_providers(key: str, count: int = 20) list[PeerInfo]
Reference to provider_store.find_providers for convenience.
- get_routing_table_size() int
Get the number of peers in the routing table.
Returns
- int
Number of peers.
- async get_value(key: str, quorum: int = 0) bytes | None
Retrieve a value from the DHT.
- Args:
key: String key (will be converted to bytes for lookup) quorum: Minimum number of valid peer responses required for confidence. If quorum > 0 and not met, the function still returns the best value found (if any) but logs a warning. Set to 0 to disable quorum checking.
- Returns:
The value if found (best value even if quorum not met), None otherwise. Note: When quorum is not met, a warning is logged but the best available value is still returned. This allows graceful degradation when the network has insufficient peers.
- get_value_store_size() int
Get the number of items in the value store.
Returns
- int
Number of items.
- async handle_stream(stream: INetStream) None
Handle an incoming DHT stream using varint length prefixes.
- is_random_walk_enabled() bool
Check if random walk peer discovery is enabled.
Returns
- bool
True if random walk is enabled, False otherwise.
- async put_value(key: str, value: bytes) None
Store a value in the DHT.
- Args:
key: String key (will be converted to bytes for storage) value: Binary value to store
- Raises:
InvalidRecordType: If no validator is registered for the key’s namespace ValueError: If trying to replace a newer value with an older one
- register_validator(namespace: str, validator: Validator) None
Register a custom validator for a specific namespace.
This allows storing and retrieving values with custom namespaced keys (e.g., /myapp/key). The validator will be used to validate values before storing and after retrieval.
- Args:
namespace: The namespace string (e.g., “myapp” for keys like /myapp/key) validator: A Validator instance with validate() and select() methods
- Example:
- class MyValidator(Validator):
- def validate(self, key: str, value: bytes) -> None:
# Custom validation logic pass
- def select(self, key: str, values: list[bytes]) -> int:
return 0 # Return index of best value
dht.register_validator(“myapp”, MyValidator()) await dht.put_value(“/myapp/my-key”, b”my-value”)
- set_namespace_validator(ns: str, val: Validator) None
Adds a validator under a specofic namespace to the current DHT config.
Raises an error if the current validator is not a NamespacedValidator
- set_validator(val: NamespacedValidator) None
Set a custom validator for the DHT config.
This marks the validator as explicitly changed, so the default validators (pk and ipns) will not be automatically applied later.
- class libp2p.kad_dht.PeerRouting(host: IHost, routing_table: RoutingTable)
Bases:
IPeerRoutingImplementation of peer routing using the Kademlia algorithm.
This class provides methods to find peers in the DHT network and helps maintain the routing table.
- async find_closest_peers_network(target_key: bytes, count: int = 20) list[ID]
Find the closest peers to a target key in the entire network.
Performs an iterative lookup by querying peers for their closest peers. If the routing table has fewer peers than MIN_PEERS_THRESHOLD, it falls back to using connected peers first, then peers from the peerstore if needed, to gather up to ‘count’ initial query targets.
Returns
- list[ID]
Closest peer IDs
- class libp2p.kad_dht.RoutingTable(local_id: ID, host: IHost)
Bases:
objectThe Kademlia routing table maintains information on which peers to contact for any given peer ID in the network.
- async add_peer(peer_obj: PeerInfo | ID) bool
Add a peer to the routing table.
- Parameters:
peer_obj – Either PeerInfo object or peer ID to add
Returns
bool: True if the peer was added or updated, False otherwise
- cleanup_routing_table() None
Cleanup the routing table by removing all data. This is useful for resetting the routing table during tests or reinitialization.
- find_bucket(peer_id: ID) KBucket
Find the bucket that would contain the given peer ID.
- Parameters:
peer_id – The peer ID to find a bucket for
Returns
KBucket: The bucket for this peer
- find_local_closest_peers(key: bytes, count: int = 20) list[ID]
Find the closest peers to a given key.
- Parameters:
key – The key to find closest peers to (bytes)
count – Maximum number of peers to return
Returns
List[ID]: List of peer IDs closest to the key
- get_peer_ids() list[ID]
Get all peer IDs in the routing table.
Returns
- param List[ID]:
List of all peer IDs
- get_peer_info(peer_id: ID) PeerInfo | None
Get the peer info for a specific peer.
- Parameters:
peer_id – The ID of the peer to get info for
Returns
PeerInfo: The peer info, or None if not found
- get_peer_infos() list[PeerInfo]
Get all PeerInfo objects in the routing table.
Returns
List[PeerInfo]: List of all PeerInfo objects
- get_stale_peers(stale_threshold_seconds: int = 3600) list[ID]
Get all stale peers from all buckets
- params: stale_threshold_seconds:
Time in seconds after which a peer is considered stale
Returns
- list[ID]
List of stale peer IDs
- peer_in_table(peer_id: ID) bool
Check if a peer is in the routing table.
- Parameters:
peer_id – The ID of the peer to check
Returns
bool: True if the peer is in the routing table, False otherwise
- class libp2p.kad_dht.ValueStore(host: IHost, local_peer_id: ID)
Bases:
objectStore for key-value pairs in a Kademlia DHT.
Values are stored with a timestamp and optional expiration time.
- cleanup_expired() int
Remove all expired values from the store.
Returns
- int
The number of expired values that were removed
- get(key: bytes) Record | None
Retrieve a value from the DHT.
Parameters
- keybytes
The key to look up
Returns
- Optional[bytes]
The stored value, or None if not found or expired
- has(key: bytes) bool
Check if a key exists in the store and hasn’t expired.
Parameters
- keybytes
The key to check
Returns
- bool
True if the key exists and hasn’t expired, False otherwise
- put(key: bytes, value: bytes, validity: float = 0.0) None
Store a value in the DHT.
- Parameters:
key – The key to store the value under
value – The value to store
validity – validity in seconds before the value expires. Defaults to DEFAULT_TTL if set to 0.0.
Returns
None