User:Justmoon/BIP Draft: Custom Services

From Bitcoin Wiki
Jump to: navigation, search

This page describes a BIP (Bitcoin Improvement Proposal).
Please see BIP 2 for more information about BIPs and creating them. Please do not just create a wiki page.

  BIP: Draft
  Title: Custom Services
  Author: Stefan Thomas <justmoon@members.fsf.org>
  Status: Draft
  Type: Standards Track
  Created: 03-08-2012

Abstract

This BIP adds new fields to the version message which clients can use to announce custom services without polluting the limited 64-bit services field. It also makes some non-binding recommendations regarding the implementation of custom services.

Motivation

We would like to encourage experimentation with custom services that extend the Bitcoin protocol with useful functionality. Examples include Distributed Hash Tables (DHT), distributed pools, lightweight client support protocols, directed message routing and support for custom transports. However, without a general, extensible framework for protocol extensions, these custom services are likely to collide in various ways. This BIP provides such a framework.

Specification

Two new fields are added to the version command, after extra_height:

Field Size Description Data type Comments
1+ service_count var_int Number of extra services
 ? service_list service[] List of service definitions

The service definitions are given in the following format:

Field Size Description Data type Comments
 ? service_name var_str Unique service identifier
4 service_version uint32_t Identifies service version being used by the node
 ? service_data var_str Additional service-specific data

A node MUST NOT announce two services with the same service_name. If a remote node sends such a version message the client MAY disconnect.

The service_version is service-specific and can be any integer. Higher versions SHOULD be higher integers. When a service is standardized, it is assigned a NODE_* constant for use with the services field and future iterations of the protocol depend on the Bitcoin protocol version. Both the NODE_* flag and the custom service entry MAY be provided for the duration of a transitional period.

Services SHOULD pass an empty string (0x00) as service_data and use a custom handshake to initialize their protocol, exchange information about capabilities etc. Note that to become a standardized service, a service MUST NOT rely on service_data since there is no corresponding mechanism for the standard services defined in the services field.

However, services MAY use service_data if they do not intend to become standard services and need a simple way to transmit a small amount of initialization data. For example, a node offering a custom transport like UDP or WebSocket, may choose to announce this as a service and include the port number in service_data. The format for service_data is service-specific and may be any binary or ASCII data.

Service identifier

Each service SHOULD choose a new identifier that is not used by any other service.

Service identifiers that are reserved or used by an accepted BIP MUST NOT be used except in the way specified by that BIP.

Service identifiers MUST be between five (5) and eleven (11) characters long. Service identifiers MUST use only ASCII characters, excluding: / * _ :

Valid examples:

  • MySampleSvc
  • smartserv
  • P-Pool

Valid, but discouraged examples:

  • MySVC 1.0 (use service_version to differentiate versions)
  • @@---. (identifiers should be pronounceable)
  • lightweight (avoid too generic names)

Invalid examples:

  • Pppc (too short)
  • SuperService (too long)
  • Cool_Svc (invalid character)

Optional: Service messages

All messages of a custom service SHOULD be represented by a single "command" on the Bitcoin network, consisting of an underscore, followed by the service identifier. For example: _MySampleSvc

To distinguish different messages within the service, the following format SHOULD be used:

Field Size Description Data type Comments
12 subcommand char[12] ASCII string identifying the service command, NULL padded (non-NULL padding results in packet rejected)
 ? subpayload uchar[] The actual data

The length of subpayload is derived from the length of the total payload minus twelve (12) bytes for the subcommand. Implementations MUST NOT rely on this format to be used by unknown services. Clients SHOULD ignore any services or subcommands they don't explicitly understand.

The recommended way to refer to messages following this format in documentation is by the service identifier, followed by a colon, followed by the subcommand. For example, the subcommand search for the MySampleSvc service would be referred to as: MySampleSvc:search

Full hexdump of an example MySampleSvc:search message:

0000   F9 BE B4 D9 5F 4D 79 53  61 6D 70 6C 65 53 76 63   ...._MySampleSvc
0010   14 00 00 00 73 D5 56 77  73 65 61 72 63 68 00 00   ....s.Vwsearch..
0020   00 00 00 00 12 34 56 78  9A BC DE F0               .....4Vx....

Message header:
 F9 BE B4 D9                                                                   - Main network magic bytes
 5F 4D 79 53 61 6D 70 6C 65 53 76 63                                           - "_MySampleSvc" command
 14 00 00 00                                                                   - Payload is 20 bytes long
                                                                                 (includes 12 bytes for subcommand)
 73 D5 56 77                                                                   - Checksum

Service header:
 73 65 61 72 63 68 00 00 00 00 00 00                                           - "search" subcommand

Search message:
 12 34 56 78 9A BC DE F0                                                       - Payload

Optional: Announcing known nodes

Bitcoin includes the services for each node in the addr message. In order to provide the same functionality, a custom service X MAY define a message X:addr with the following format:

  • If X:addr is empty or starts with TYPE_ALL (0x00), then all addresses in the next addr message support the service.
  • If X:addr starts with TYPE_BITS (0x01), then the rest of the data is a bit array marking which nodes in the next addr message support the service. Any nodes not covered by the bitarray don't support the service.
  • If X:addr starts with TYPE_RLE (0x02), then the next byte determines whether the first address supports the service and the remaining data is a series of var_ints determining the lengths of runs of nodes that support or don't support the service, not counting the first node in the run. Any nodes not covered by the RLE have the opposite value of the last node covered.
  • If there was no X:addr message since the last addr message or the beginning of the connection, then none of the nodes in the next addr message support the service.

Examples:

 00         - TYPE_ALL - all nodes in the next addr message support the service
 01         - TYPE_BITS
 FC         - 11111100 - First six nodes support the service, next two don't
 1F         - 00011111 - Next three don't, next five do
            - Remaining nodes don't
 02         - TYPE_RLE
 01         - First set of nodes are nodes that support the service
 05         - First six nodes support the service
 FD 42 03   - Next 835 nodes don't
 00         - Next node does
 08         - Next nine nodes don't
            - Remaining nodes do

Multiple X:addr messages for different services MUST all be applied to the next addr message. If there are multiple X:addr messages for the same service, only the most recent one is used.

If a node knows about many different services, the recommendation is to store a bitmask of all the recognized services for each node, order by that field and then use X:addr messages using TYPE_RLE.

Rationale

This BIP aims to fulfill the following goals:

  • Minimize the risk of namespace collisions, ambiguities or other issues arising from conflicting custom services
  • Provide an easy upgrade path for custom services to become standardized services with their own NODE_* flag
  • Place minimum restrictions on custom service authors
  • Allow custom services to be created with minimum effort
  • Allow clients to support multiple/many custom services at once

To achieve these goals this BIP adds two new fields to the version message. It would have been possible to avoid changes to version by adding a new message instead. However, it makes sense to keep both types of service announcements in the same message so that the life cycle of standardized services and custom services remains exactly the same. This also simplifies detecting a service which is in the transition from a custom to a standardized service (and being announced using both methods.)

While writing this BIP it became apparent that twelve characters were not sufficient to contain both the service identifier and the subcommand. This lead to the definition of a recommended encoding for service-specific messages. Services where this encoding is not appropriate are able to supercede it.

For announcing other nodes supporting the service, it would have been possible to extend the addr message, however some services may not require this type of announcement whereas other services may wish to include additional information per node. Therefore we provide a recommended scheme that implements efficiently the same functionality provided by the services bitmask. Services for which this scheme is not appropriate can supercede these recommendations as needed.

Finally, this BIP defines both explicitly and implicitly some useful common nomenclature that can be used when discussing custom services, e.g. "subcommand", "subpayload", "service identifier" and the colon format for referring to subcommands.

Copyright

This document is placed in the public domain.