Bitswap

The Bitswap protocol enables content discovery and file sharing in py-libp2p networks.

Overview

Bitswap is a message-based protocol for exchanging content-addressed blocks between peers. It is a core component of IPFS (InterPlanetary File System) and enables efficient peer-to-peer file sharing.

Key Features

Content-addressed block storage and retrieval
Wantlist-based block exchange
Priority-based request handling
Bidirectional peer communication
Cancellation support for requests

Protocol Version

This implementation supports Bitswap 1.0.0 (/ipfs/bitswap/1.0.0).

Quick Start

Basic Usage

import trio
from libp2p import new_host
from libp2p.bitswap import BitswapClient
import multiaddr

async def main():
    # Create a host
    listen_addr = multiaddr.Multiaddr("/ip4/0.0.0.0/tcp/0")
    host = new_host()

    async with host.run([listen_addr]), trio.open_nursery() as nursery:
        # Create Bitswap client
        bitswap = BitswapClient(host)
        bitswap.set_nursery(nursery)
        await bitswap.start()

        # Add a block
        cid = compute_cid(b"Hello, Bitswap!")
        await bitswap.add_block(cid, b"Hello, Bitswap!")

        # Request a block from a peer
        data = await bitswap.get_block(cid, peer_id, timeout=30)
        print(f"Received: {data}")

        await bitswap.stop()

trio.run(main)

Running a Provider

async def run_provider():
    host = new_host()
    listen_addr = multiaddr.Multiaddr("/ip4/0.0.0.0/tcp/4001")

    async with host.run([listen_addr]), trio.open_nursery() as nursery:
        bitswap = BitswapClient(host)
        bitswap.set_nursery(nursery)
        await bitswap.start()

        # Add blocks to serve
        blocks = [b"Block 1", b"Block 2", b"Block 3"]
        for data in blocks:
            cid = compute_cid(data)
            await bitswap.add_block(cid, data)

        print(f"Provider listening on {host.get_addrs()}")
        await trio.sleep_forever()

Running a Client

async def run_client(provider_multiaddr):
    host = new_host()
    listen_addr = multiaddr.Multiaddr("/ip4/0.0.0.0/tcp/0")

    # Parse provider address
    provider_info = info_from_p2p_addr(provider_multiaddr)

    async with host.run([listen_addr]), trio.open_nursery() as nursery:
        bitswap = BitswapClient(host)
        bitswap.set_nursery(nursery)
        await bitswap.start()

        # Connect to provider
        await host.connect(provider_info)

        # Request blocks
        wanted_cids = [...]  # List of CIDs to request
        for cid in wanted_cids:
            try:
                data = await bitswap.get_block(cid, provider_info.peer_id)
                print(f"Got block: {data}")
            except Exception as e:
                print(f"Failed to get block: {e}")

API Reference

BitswapClient

class libp2p.bitswap.BitswapClient(host: IHost, block_store: BlockStore | None = None, protocol_version: str = '/ipfs/bitswap/1.2.0', provider_query_manager: ProviderQueryManager | None = None)

Bases: object

Bitswap client for exchanging blocks with other peers.

Supports Bitswap protocol versions 1.0.0, 1.1.0, 1.2.0, and 1.3.0 for content discovery and file sharing in a peer-to-peer network.

For 1.3.0 payment support, register a PaymentExtension.

async add_block(cid: bytes | str | CIDv0 | CIDv1, data: bytes) → None

Add a block to the local store.

Args:: cid: The CID of the block data: The block data
Raises:: BlockTooLargeError: If the block exceeds maximum size

async cancel_want(cid: bytes | str | CIDv0 | CIDv1) → None

Cancel a previous want for a block.

Args:: cid: The CID to cancel

async get_block(cid: bytes | str | CIDv0 | CIDv1, peer_id: ID | None = None, timeout: float = 90) → bytes

Get a block, fetching from peers if not available locally.

If a ProviderQueryManager was supplied at construction time and no explicit peer_id is given, the manager is consulted first to discover which peers have the block via the DHT. The first discovered provider is used; if none is found the request falls back to broadcasting to all connected peers.

Args:: cid: The CID of the block to fetch peer_id: Optional peer to request from; DHT discovery is skipped when set. timeout: Timeout in seconds
Returns:: The block data
Raises:: BlockNotFoundError: If the block cannot be found BitswapTimeoutError: If the request times out

async get_blocks_batch(cids: list[bytes | str | CIDv0 | CIDv1], peer_id: ID | None = None, timeout: float = 90, batch_size: int = 32) → dict[bytes, bytes]

Fetch multiple blocks in batches using a single wantlist per batch.

Sends all CIDs in one wantlist message, waits for all responses on the same stream. This avoids opening hundreds of individual streams which causes Kubo to send GO_AWAY.

Args:: cids: List of CIDs to fetch peer_id: Optional specific peer to request from timeout: Timeout per batch in seconds batch_size: How many CIDs to request per wantlist message
Returns:: Dict mapping cid_bytes -> block_data for all successfully fetched blocks

async have_block(cid: bytes | str | CIDv0 | CIDv1, peer_id: ID | None = None) → bool

Check if a peer has a block (v1.2.0 feature).

Args:: cid: The CID of the block to check peer_id: Optional specific peer to query
Returns:: True if peer has the block, False otherwise

register_extension(protocol: str, extension: IBitswapExtension) → None: Register an extension for a specific protocol.

set_nursery(nursery: Nursery) → None: Set the nursery for background tasks.

async start() → None: Start the Bitswap client.

async stop() → None: Stop the Bitswap client.

async want_block(cid: bytes | str | CIDv0 | CIDv1, priority: int = 1, want_type: int = 0, send_dont_have: bool = False) → None

Add a block to the wantlist without blocking.

Args:: cid: The CID of the block to want priority: Priority of the request want_type: 0 for Block (full block), 1 for Have (just check) - v1.2.0 send_dont_have: Whether to request DontHave response if not found - v1.2.0

BlockStore

class libp2p.bitswap.BlockStore

Bases: ABC

Abstract interface for storing and retrieving blocks.

Implementations should provide persistent or in-memory storage for content-addressed blocks.

abstract async delete_block(cid: bytes | str | CIDv0 | CIDv1) → None

Delete a block.

Args:: cid: The CID of the block to delete

abstract get_all_cids() → list[bytes]

Get all CIDs in the block store.

Returns:: List of all CIDs as bytes

abstract async get_block(cid: bytes | str | CIDv0 | CIDv1) → bytes | None

Get a block by its CID.

Args:: cid: The CID of the block to retrieve
Returns:: The block data if found, None otherwise

abstract async has_block(cid: bytes | str | CIDv0 | CIDv1) → bool

Check if a block exists.

Args:: cid: The CID of the block
Returns:: True if the block exists, False otherwise

abstract async put_block(cid: bytes | str | CIDv0 | CIDv1, data: bytes) → None

Store a block.

Args:: cid: The CID of the block data: The block data

MemoryBlockStore

class libp2p.bitswap.MemoryBlockStore

Bases: BlockStore

In-memory block store implementation using CID objects as keys.

async delete_block(cid: bytes | str | CIDv0 | CIDv1) → None: Delete a block.

get_all_cids() → list[bytes]: Get all CIDs in the store as bytes (for caller compatibility).

async get_block(cid: bytes | str | CIDv0 | CIDv1) → bytes | None: Get a block by its CID.

async has_block(cid: bytes | str | CIDv0 | CIDv1) → bool: Check if a block exists.

async put_block(cid: bytes | str | CIDv0 | CIDv1, data: bytes) → None: Store a block.

size() → int: Get the number of blocks in the store.

Configuration

Bitswap configuration constants and defaults.

Errors

Bitswap protocol errors.

exception libp2p.bitswap.errors.BitswapError

Bases: Exception

Base exception for Bitswap errors.

exception libp2p.bitswap.errors.BlockNotFoundError

Bases: BitswapError

Raised when a requested block is not found.

exception libp2p.bitswap.errors.BlockTooLargeError

Bases: BitswapError

Raised when a block exceeds the maximum size.

exception libp2p.bitswap.errors.BlockUnavailableError

Bases: BitswapError

Raised when a peer confirms they don’t have a requested block (DontHave).

exception libp2p.bitswap.errors.InvalidBlockError

Bases: BitswapError

Raised when a block is invalid or malformed.

exception libp2p.bitswap.errors.InvalidCIDError

Bases: BitswapError

Raised when a CID is invalid.

exception libp2p.bitswap.errors.MessageTooLargeError

Bases: BitswapError

Raised when a message exceeds the maximum size.

exception libp2p.bitswap.errors.TimeoutError

Bases: BitswapError

Raised when an operation times out.

Protocol Details

Message Format

Bitswap messages use Protocol Buffers with the following structure:

message Message {
  message Wantlist {
    message Entry {
      bytes block = 1;      // CID of the block
      int32 priority = 2;   // Priority (higher = more important)
      bool cancel = 3;      // Cancel a previous request
    }
    repeated Entry entries = 1;
    bool full = 2;          // Full wantlist or update
  }

  Wantlist wantlist = 1;
  repeated bytes blocks = 2;  // Block data
}

Wire Protocol

Messages are sent as length-prefixed protobuf messages:

Message is serialized to protobuf bytes
Length is encoded as unsigned varint
Format: [varint length][protobuf message]

Maximum message size: 4 MiB

Maximum block size: 2 MiB

Block Exchange Flow

Client requests block:
- Check local block store
- If not found, add to wantlist
- Send wantlist to peer(s)
- Wait for block response
Server receives wantlist:
- Parse wantlist entries
- Check local block store for requested blocks
- Send available blocks to client
Client receives blocks:
- Store blocks locally
- Notify pending requests
- Send cancel messages for received blocks

Wantlist Management

Bitswap maintains two types of wantlists:

Local wantlist: Blocks this peer wants
Peer wantlists: Blocks each connected peer wants

Wantlist updates can be:

Full (full = true): Replace entire wantlist
Incremental (full = false): Apply specific changes

Advanced Usage

Custom Block Store

Implement a custom block store by subclassing BlockStore:

from libp2p.bitswap import BlockStore
from pathlib import Path
from typing import Optional

class FileBlockStore(BlockStore):
    def __init__(self, base_path: str):
        self.base_path = Path(base_path)
        self.base_path.mkdir(exist_ok=True)

    async def get_block(self, cid: bytes) -> Optional[bytes]:
        file_path = self.base_path / cid.hex()
        if file_path.exists():
            return file_path.read_bytes()
        return None

    async def put_block(self, cid: bytes, data: bytes) -> None:
        file_path = self.base_path / cid.hex()
        file_path.write_bytes(data)

    async def has_block(self, cid: bytes) -> bool:
        file_path = self.base_path / cid.hex()
        return file_path.exists()

    async def delete_block(self, cid: bytes) -> None:
        file_path = self.base_path / cid.hex()
        if file_path.exists():
            file_path.unlink()

# Use the custom store
bitswap = BitswapClient(host, FileBlockStore("/path/to/blocks"))

Priority-Based Requests

Request blocks with different priorities:

# High priority block
await bitswap.want_block(critical_cid, priority=10)

# Normal priority
await bitswap.want_block(normal_cid, priority=1)

# Low priority
await bitswap.want_block(optional_cid, priority=0)

Blocks with higher priority values are served first.

Cancel Requests

Cancel a block request:

# Add to wantlist
await bitswap.want_block(cid)

# Later, if no longer needed
await bitswap.cancel_want(cid)

This sends a cancel message to all peers that haven’t responded yet.

Examples

The examples/bitswap directory contains complete examples:

Provider mode: Serve blocks to other peers
Client mode: Request blocks from providers
Demo mode: Automated demonstration

Run examples:

# Provider
python examples/bitswap/bitswap.py --mode provider

# Provider with file
python examples/bitswap/bitswap.py --mode provider --file myfile.txt

# Client
python examples/bitswap/bitswap.py --mode client --provider <multiaddr>

# Demo
python examples/bitswap/bitswap.py --mode demo

Limitations

Current Version Limitations

CID Encoding: Uses simplified SHA-256 hashing instead of proper CIDv0/CIDv1
Protocol Version: Only supports Bitswap 1.0.0
Block Store: Default implementation is in-memory only
No Peer Scoring: No accounting or peer reputation system

Future Enhancements

Planned improvements:

Support for Bitswap 1.1.0 (CIDv1 with prefix encoding)
Support for Bitswap 1.2.0 (Have/DontHave responses)
Proper CID library integration
Persistent block storage
Peer scoring and accounting
Content routing integration with DHT

Bitswap

Subpackages

Overview

Key Features

Protocol Version

Quick Start

Basic Usage

Running a Provider

Running a Client

API Reference

BitswapClient

BlockStore

MemoryBlockStore

Configuration

Errors

Protocol Details

Message Format

Wire Protocol

Block Exchange Flow

Wantlist Management

Advanced Usage

Custom Block Store

Priority-Based Requests

Cancel Requests

Examples

Limitations

Current Version Limitations

Future Enhancements

See Also