|
|
| (11 intermediate revisions by the same user not shown) |
| Line 1: |
Line 1: |
| − | {{bip}}
| + | #REDIRECT [[BIP 0036]] |
| − | | |
| − | <pre>
| |
| − | BIP: Draft
| |
| − | Title: Custom Services
| |
| − | Author: Stefan Thomas <justmoon@members.fsf.org>
| |
| − | Status: Draft
| |
| − | Type: Standards Track
| |
| − | Created: 03-08-2012
| |
| − | </pre>
| |
| − | | |
| − | ==Abstract==
| |
| − | | |
| − | This BIP adds new fields to the <code>version</code> message which clients can use to announce custom services without polluting the limited 64-bit <code>services</code> field. It also makes some non-binding recommendations regarding the implementation of custom services.
| |
| − | | |
| − | ==Motivation==
| |
| − | | |
| − | We would like to encourage experimentation with custom services that extend the Bitcoin protocol with useful functionality. Examples include Distributed Hash Tables (DHT), distributed pools, lightweight client support protocols, directed message routing and support for custom transports. However, without a general, extensible framework for protocol extensions, these custom services are likely to collide in various ways. This BIP provides such a framework.
| |
| − | | |
| − | ==Specification==
| |
| − | | |
| − | Two new fields are added to the <code>version</code> command, after <code>extra_height</code>:
| |
| − | | |
| − | {|class="wikitable"
| |
| − | ! Field Size !! Description !! Data type !! Comments
| |
| − | |-
| |
| − | | 1+ || service_count || [[Protocol_specification#Variable_length_integer|var_int]] || Number of extra services
| |
| − | |-
| |
| − | | ? || service_list || service[] || List of service definitions
| |
| − | |}
| |
| − | | |
| − | The service definitions are given in the following format:
| |
| − | | |
| − | {|class="wikitable"
| |
| − | ! Field Size !! Description !! Data type !! Comments
| |
| − | |-
| |
| − | | ? || service_name || [[#Variable length string|var_str]] || Unique service identifier
| |
| − | |-
| |
| − | | 4 || service_version || uint32_t || Identifies service version being used by the node
| |
| − | |-
| |
| − | | ? || service_data || [[#Variable length string|var_str]] || Additional service-specific data
| |
| − | |}
| |
| − | | |
| − | A node MUST NOT announce two services with the same <code>service_name</code>. If a remote node sends such a <code>version</code> message the client MAY disconnect.
| |
| − | | |
| − | The <code>service_version</code> is service-specific and can be any integer. Higher versions SHOULD be higher integers. When a service is standardized, it is assigned a <code>NODE_*</code> constant for use with the <code>services</code> field and future iterations of the protocol depend on the Bitcoin protocol version. Both the <code>NODE_*</code> flag and the custom service entry MAY be provided for the duration of a transitional period.
| |
| − | | |
| − | Services SHOULD pass an empty string (0x00) as <code>service_data</code> and use a custom handshake to initialize their protocol, exchange information about capabilities etc. Note that to become a standardized service, a service MUST NOT rely on <code>service_data</code> since there is no corresponding mechanism for the standard services defined in the <code>services</code> field.
| |
| − | | |
| − | However, services MAY use <code>service_data</code> if they do not intend to become standard services and need a simple way to transmit a small amount of initialization data. For example, a node offering a custom transport like UDP or WebSocket, may choose to announce this as a service and include the port number in <code>service_data</code>. The format for <code>service_data</code> is service-specific and may be any binary or ASCII data. For ease of debugging/introspection, a human-readable (ASCII) format is generally recommended.
| |
| − | | |
| − | ===Service identifier===
| |
| − | | |
| − | Each service SHOULD choose a new identifier that is not used by any other service. To register a new identifier, add it to the [[Service identifiers]] wiki page along with the name of the maintainer and a way to contact them. Please do not register identifiers unless you are actually using them.
| |
| − | | |
| − | Service identifiers that are reserved or used by an accepted BIP MUST NOT be used except in the way specified by that BIP.
| |
| − | | |
| − | Service identifiers MUST be between five (5) and eleven (11) characters long. Service identifiers MUST use only ASCII characters, excluding: / * _ :
| |
| − | | |
| − | Valid examples:
| |
| − | * <code>MySampleSvc</code>
| |
| − | * <code>smartserv</code>
| |
| − | * <code>P-Pool</code>
| |
| − | | |
| − | Valid, but discouraged examples:
| |
| − | | |
| − | * <code>MySVC 1.0</code> (use <code>service_version</code> to differentiate versions)
| |
| − | * <code>@@---.</code> (identifiers should be pronounceable)
| |
| − | * <code>lightweight</code> (avoid too generic names)
| |
| − | | |
| − | Invalid examples:
| |
| − | | |
| − | * <code>Pppc</code> (too short)
| |
| − | * <code>SuperService</code> (too long)
| |
| − | * <code>Cool_Svc</code> (invalid character)
| |
| − | | |
| − | ===Optional: Custom commands===
| |
| − | | |
| − | Bitcoin command names are limited to 12 characters. That doesn't leave a lot of space for both the service identifier and the service command. Therefore we recommend that all service commands SHOULD be represented by a single "command" on the Bitcoin network. To avoid collision with other services the exact service identifier SHOULD be used, but to avoid collisions with current or future Bitcoin messages the service identifier SHOULD be prefixed with an underscore. For example: <code>_MySampleSvc</code>
| |
| − | | |
| − | The service-specific command name SHOULD then be specified in an extra header in the payload:
| |
| − | | |
| − | {|class="wikitable"
| |
| − | ! Field Size !! Description !! Data type !! Comments
| |
| − | |-
| |
| − | | 12 || subcommand || char[12] || ASCII string identifying the service command, NULL padded (non-NULL padding results in packet rejected)
| |
| − | |-
| |
| − | | ? || subpayload || uchar[] || The actual data
| |
| − | |}
| |
| − | | |
| − | The length of <code>subpayload</code> is derived from the length of the total payload minus twelve (12) bytes for the <code>subcommand</code>. Implementations MUST NOT rely on this format to be used by unknown services. Clients SHOULD ignore any services or subcommands they don't explicitly understand.
| |
| − | | |
| − | The recommended way to refer to messages following this format in documentation is by the service identifier, followed by a colon, followed by the subcommand. For example, the subcommand <code>search</code> for the <code>MySampleSvc</code> service would be referred to as: <code>MySampleSvc:search</code>
| |
| − | | |
| − | Full hexdump of an example <code>MySampleSvc:search</code> message:
| |
| − | | |
| − | <pre>
| |
| − | 0000 F9 BE B4 D9 5F 4D 79 53 61 6D 70 6C 65 53 76 63 ...._MySampleSvc
| |
| − | 0010 14 00 00 00 73 D5 56 77 73 65 61 72 63 68 00 00 ....s.Vwsearch..
| |
| − | 0020 00 00 00 00 12 34 56 78 9A BC DE F0 .....4Vx....
| |
| − | | |
| − | Message header:
| |
| − | F9 BE B4 D9 - Main network magic bytes
| |
| − | 5F 4D 79 53 61 6D 70 6C 65 53 76 63 - "_MySampleSvc" command
| |
| − | 14 00 00 00 - Payload is 20 bytes long
| |
| − | (includes 12 bytes for subcommand)
| |
| − | 73 D5 56 77 - Checksum
| |
| − | | |
| − | Service header:
| |
| − | 73 65 61 72 63 68 00 00 00 00 00 00 - "search" subcommand
| |
| − | | |
| − | Search message:
| |
| − | 12 34 56 78 9A BC DE F0 - Payload
| |
| − | </pre>
| |
| − | | |
| − | ===Optional: Announcing known nodes===
| |
| − | | |
| − | Bitcoin includes the <code>services</code> field for each node in the <code>addr</code> message. This is very useful for discovering other nodes that support a specific service. For example a lightweight client may be interested in finding all the nodes capable of serving it.
| |
| − | | |
| − | However, some services may not need this functionality whereas other services may need to communicate extra information per node. So rather than modifying the <code>addr</code> message, we instead send a service-specific message ahead of it. This message SHOULD use the addr subcommand, so for the <code>MySampleSvc</code> service it would be called <code>MySampleSvc:addr</code>. We'll refer to it here generically as the <code>*:addr</code> message.
| |
| − | | |
| − | The <code>*:addr</code> of a service specifies which nodes in the next following <code>addr</code> list support that service. It uses the following format:
| |
| − | | |
| − | * If <code>*:addr</code> is empty or starts with TYPE_ALL (0x00), then all addresses in the next <code>addr</code> message support the service.
| |
| − | * If <code>*:addr</code> starts with TYPE_BITS (0x01), then the rest of the data is a bit array marking which nodes in the next addr message support the service. Any nodes not covered by the bitarray don't support the service.
| |
| − | * If <code>*:addr</code> starts with TYPE_RLE (0x02), then the next byte determines whether the first address supports the service and the remaining data is a series of [[Protocol_specification#Variable_length_integer|var_ints]] determining the lengths of runs of nodes that support or don't support the service, not counting the first node in the run. Any nodes not covered by the RLE have the opposite value of the last node covered.
| |
| − | * If there was no <code>*:addr</code> message since the last <code>addr</code> message or the beginning of the connection, then none of the nodes in the next <code>addr</code> message support the service.
| |
| − | | |
| − | Examples:
| |
| − | | |
| − | 00 - TYPE_ALL - all nodes in the next addr message support the service
| |
| − | | |
| − | 01 - TYPE_BITS
| |
| − | FC - 11111100 - First six nodes support the service, next two don't
| |
| − | 1F - 00011111 - Next three don't, next five do
| |
| − | - Remaining nodes don't
| |
| − | | |
| − | 02 - TYPE_RLE
| |
| − | 01 - First set of nodes are nodes that support the service
| |
| − | 05 - First six nodes support the service
| |
| − | FD 42 03 - Next 835 nodes don't
| |
| − | 00 - Next node does
| |
| − | 08 - Next nine nodes don't
| |
| − | - Remaining nodes do
| |
| − | | |
| − | Multiple <code>X:addr</code> messages for different services MUST all be applied to the next <code>addr</code> message. If there are multiple <code>X:addr</code> messages for the same service, only the most recent one is used.
| |
| − | | |
| − | If a node knows about many different services, the recommendation is to store a bitmask of all the recognized services for each node, order by that field and then use <code>X:addr</code> messages using TYPE_RLE.
| |
| − | | |
| − | ==Standardization==
| |
| − | | |
| − | Custom services may become standard parts of the protocol. Services which wish to become part of the Bitcoin protocol MUST fulfill the following criteria:
| |
| − | | |
| − | * MUST NOT use <code>service_data</code>; Standard services have no corresponding field
| |
| − | * MUST use the *:addr mechanism; Standard services can specify one bit per node
| |
| − | * MUST NOT use any subcommands that conflict with current or planned Bitcoin protocol commands
| |
| − | | |
| − | The standardization process will usually take place as follows:
| |
| − | | |
| − | # The service is implemented and tested.
| |
| − | # Once the API is known to be relatively stable it is formalized and submitted as a BIP.
| |
| − | # Once the BIP is accepted, the service is assigned a <code>NODE_*</code> constant and the transitional period starts:
| |
| − | #* Clients MUST understand both the announcement of the service via the <code>services</code> field and via <code>service_list</code> and include both methods in their own <code>version</code> message.
| |
| − | #* Clients MUST accept both the wrapped form messages like <code>MySampleSvc:search</code> as well as the corresponding non-namespaced messages like <code>search</code>. Clients MUST only send wrapped messages.
| |
| − | #* During the transitional period the API of the service MUST NOT change.
| |
| − | # After the transitional period:
| |
| − | #* Clients MUST only announce the service via the <code>services</code> field.
| |
| − | #* Clients MUST only send unwrapped messages.
| |
| − | # Future changes to the service API now require a BIP and an increase in the Bitcoin protocol version.
| |
| − | | |
| − | This process of adding a service to the Bitcoin protocol should only be undertaken for services where there is a strong rationale for doing so. Services MAY also be standardized as custom services via a BIP while maintaining the custom service format.
| |
| − | | |
| − | ==Rationale==
| |
| − | | |
| − | This BIP aims to fulfill the following goals:
| |
| − | | |
| − | * Minimize the risk of namespace collisions, ambiguities or other issues arising from conflicting custom services
| |
| − | * Provide an easy upgrade path for custom services to become standardized services with their own <code>NODE_*</code> flag
| |
| − | * Place minimum restrictions on custom service authors
| |
| − | * Allow custom services to be created with minimum effort
| |
| − | * Allow clients to support multiple/many custom services at once
| |
| − | | |
| − | To achieve these goals this BIP adds two new fields to the <code>version</code> message. It would have been possible to avoid changes to <code>version</code> by adding a new message instead. However, it makes sense to keep both types of service announcements in the same message so that the life cycle of standardized services and custom services remains exactly the same. This also simplifies detecting a service which is in the transition from a custom to a standardized service (and being announced using both methods.)
| |
| − | | |
| − | While writing this BIP it became apparent that twelve characters were not sufficient to contain both the service identifier and the subcommand. This lead to the definition of a recommended encoding for service-specific messages. Services where this encoding is not appropriate are able to supercede it.
| |
| − | | |
| − | For announcing other nodes supporting the service, it would have been possible to extend the <code>addr</code> message, however some services may not require this type of announcement whereas other services may wish to include additional information per node. Therefore we provide a recommended scheme that implements efficiently the same functionality provided by the <code>services</code> bitmask. Services for which this scheme is not appropriate can supercede these recommendations as needed.
| |
| − | | |
| − | Finally, this BIP defines both explicitly and implicitly some useful common nomenclature that can be used when discussing custom services, e.g. "subcommand", "subpayload", "service identifier" and the colon format for referring to subcommands.
| |
| − | | |
| − | ==Copyright==
| |
| − | | |
| − | This document is placed in the public domain.
| |
