Overlay Network Lookup Services

Abstract

This document proposes a solution for querying the state of UTXO-based overlay networks by specifying a lookup-service mechanism. In current interoperable implementations, clients send a { service, query } object to a /lookup endpoint and receive an output-list answer describing relevant outputs using BEEF-backed transaction data. This standard defines the parameters and processing steps needed to query for and access topical UTXOs.

Motivation

As introduced in BRC-22, UTXO-based overlay networks provide a secure and scalable solution to managing states derived from the Bitcoin network. In order to enable users to query the state of these overlay networks and retrieve relevant UTXOs, a standardized method for lookup services is required. This document aims to address this need by outlining a clear and well-defined process for submitting queries to overlay network nodes and retrieving the resultant UTXOs.

Specification

We build on the node first described in BRC-22 and hook into the output admittance and spend events that naturally occur as transactions are submitted. When transactions are received, topical logic determines which outputs will be part of which overlay networks, and lookup services respond to queries over the resulting indexed state.

Status Note

Older drafts of this document described provider-based lookup and BRC-36-style JSON UTXO responses. Current interoperable behavior in bsv-blockchain/ts-sdk instead uses:

• service instead of provider,
• an output-list response type,
• BEEF payloads rather than hydrated BRC-36 JSON objects,
• and optional compact application/octet-stream responses for aggregation efficiency.

The older provider/BRC-36 framing SHOULD be treated as deprecated unless required for legacy compatibility.

API Endpoint and Parameters

The overlay node hosts a POST /lookup API endpoint which accepts a JSON request body of the form:

{
  "service": "ls_example",
  "query": {
    "...": "service-specific query fields"
  }
}

Service names used with current SDK tooling are expected to begin with ls_.

Overlay Network Node Processing Steps

Upon receiving a query, the overlay network node performs the following steps:

1. Check that the requested service is supported on this node. If not, return an error.

2. Pass the query object to the selected lookup service implementation.

3. The service produces a responsive list of currently admitted outputs and any optional service-specific context bytes associated with each output.

4. The node returns a Lookup Answer. Current interoperable answers use the following structure:

{
  "type": "output-list",
  "outputs": [
    {
      "beef": [/* BEEF bytes */],
      "outputIndex": 0,
      "context": [/* optional bytes */]
    }
  ]
}

5. For transport efficiency, the node MAY instead return application/octet-stream encoding the same information as:

   • count of outpoints,
   • for each output: txid, output index, context length, optional context bytes,
   • followed by a shared BEEF payload sufficient to reconstruct the returned outputs.

Lookup Services

Each lookup service is assigned a service identifier by the overlay network node so that multiple can be installed simultaneously. For each service, the overlay network node sends events when outputs are admitted and when they later become spent.

Each service maintains its own data storage and retrieval mechanism independently. This allows each service to use the most appropriate solution for the specific data being managed. A service is under no obligation to process events for all topics and may ignore events irrelevant to its indexing model.

Authentication, Payment, and Policy

Authentication, payment, and additional policy checks MAY be layered on top of /lookup by deployment-specific facilitators. Current baseline SDK interoperability does not require BRC-31 authentication or BRC-41 monetization for every lookup.

Example Requests and Responses

Below are examples of an HTTP request and response for the /lookup route.

Request

POST /lookup HTTP/1.1
Host: example-overlay-node.com
Content-Type: application/json

{
  "service": "ls_example",
  "query": {
    "topic": "tm_example",
    "search": "example_search_criteria"
  }
}

Response

HTTP/1.1 200 OK
Content-Type: application/json

{
  "type": "output-list",
  "outputs": [
    {
      "outputIndex": 0,
      "beef": [1, 2, 3]
    }
  ]
}

In this example, a client submits a query to the /lookup endpoint. The node returns an output-list answer containing BEEF sufficient to reconstruct the responsive output.

Implementation

Developers should expose a POST /lookup endpoint that accepts { service, query } requests and returns output-list answers in JSON or compact octet-stream form. Services that support historical traversal should also preserve enough ancestry information to satisfy BRC-64.