Protocol Specification

This is an overview of specification of the eXtensible Binary Universal Protocol as it is defined in prototype release version 0.2.0.

This should be usable for example to get idea how is the internal structure of protocol defined.

This document is not final and it's also incomplete.

For full specification, please see other parts of protocol specification in documentation.

Tree Structure (Level 0)

Lowest protocol's level defines basic tree structure using two primitive types.

Binary blob (sequence of bytes)
Natural non-negative integer number

See complete specification of tree structure.

UBNumber Encoding

UBNumber is encoding used for representation of instance of countable infinite set stored as one or more bytes (similar to UTF-8 encoding).

First, non-zero bits are counted for length and then rest of bits is used as value while value is also incremented so that there is only one code for each number.

Most native encoding using UBNumber is UBNatural for representation of a natural non-negative integer number.

Examples of the UBNatural codes (sequence of bits = represented value):

 00000000                             = 0
 00000001                             = 1
 00000010                             = 2
 00000011                             = 3
 ...
 01111111                             = 7Fh = 127
 10000000 00000000                    = 80h = 128
 10000000 00000001                    = 81h = 129
 ...
 10111111 11111111                    = 407Fh = 16511
 11000000 00000000 00000000           = 4080h = 16512
 ...

Various interpretations can be mapped on UBNumber encoding. For this level there are defined two:

UBNatural encoding using directly value from UBNumber
UBENatural where value 7Fh is reserved for infinity constant and higher values are shifted by one

Document

Document starts with 6 bytes long blob called “Document Header” followed by a single block called “Root Block” and optional blob called “Extended Area”.

Header for current version of protocol is (hexadecimal):

FE 00 58 42 00 02

Block

Each block starts with single value:

UBNatural attributePartSize

If attributePartSize = 0 then this block is called “Terminator” and block ends. Otherwise it is followed by value:

UBENatural dataPartSize

If attributePartSize = count of bytes used by dataPartSize then this block is called “Data Block” and binary blob follows which has length in bytes specified by dataPartSize value and block ends.

If attributePartSize > count of bytes used by dataPartSize then this block is called “Node Block” and a sequence of attributes follows until sum of count of bytes used by attributes = attributePartSize - count of bytes used by dataPartSize.

UBNumber attribute

After attributes, sequence of blocks follows until sum of block sizes = dataPartSize and block ends.

If dataPartSize = infinity for data block then binary blob is ended by a sequence of two zero bytes. A sequence of two bytes where first is zero followed by a non zero byte is considered a sequence of nonstoping zero bytes. The non zero byte defines the length of the sequence.

If dataPartSize = infinity for node block then sequence of data blocks is ended by the terminator.

See following picture for clarification:

Block Structure

Block Types (Level 1)

Level 1 introduces block types and catalog. For this level all non-listed attributes in node blocks are considered zero.

See complete specification of block types.

Block Type

First two attributes in node block are interpreted as follows:

UBNatural - TypeGroup
UBNatural - BlockType

Blocks types are organized into Groups. Type group with value 0 is called Basic Group and it's meaning is build-in and defined in the protocol itself.

TypeGroup value specifies to which group each block belongs. BlockType value specifies particular block in the corresponding group where meaning of single groups and block types is defined in document using basic blocks.

Basic Group

Basic group uses TypeGroup = 0. Basic blocks provides ability to store block type declarations, definitions and links to catalog.

Undefined

This block is used for unknown block types or data padding.

Block: Basic (0) / Unknown (0)

Declaration

Block: Basic (0) / Document Declaration (1)

Declaration block determines the allowed range of groups. This block should be located at the beginning of each file, if the application didn't provide any static/special meaning, but it might be used anywhere inside document as well.

Definition:

+Natural groupsCount - The number of allocated groups
+Natural preserveGroups - The number of groups to keep from previous declarations
FormatDeclaration formatDeclaration - Declaration of format
Any documentRoot - Root node of document

For subblocks of this block there is permitted range of values in the interval group preserveGroups + 1 .. preserveGroups + groupsCount + 1. preservedGroups + groupsCount + 1. If the value reserveGroups = 0, takes the highest not yet reserved group in the current or parental blocks + 1. For all values of zero and the application of rules of cutting the block of zeros coincides with the data block.

Format Declaration

Format declaration allows you use either declaration from catalog or local format definition or both.

Block: Basic (0) / Format Declaration (2)

+CatalogFormatSpecPath catalogFormatSpecPath - Specification of format defined as path in catalog
+Natural formatSpecRevision - Specification's revision number
FormatDefinition formatDefinition

Format Definition

This block allows to specify the basic structure of format specification. Specifies the sequence of groups and their definition.

Block: Basic (0) / Format Definition (3)

Any[] formatParameters - Join or Consist format parameters
+RevisionDefinition[] revisions

Block: Basic (0) / Format Join Parameter (4)

+FormatDeclaration formatDeclaration

Block: Basic (0) / Format Consist Parameter (5)

+GroupDeclaration groupDeclaration

Group Declaration

Group declaration allows you use either declaration from catalog or local group definition or both.

Block: Basic (0) / Group Declaration (6)

+CatalogGroupSpecPath catalogGroupSpecPath - Specification of format defined as path in catalog
+Natural groupSpecRevision - Specification's revision number
GroupDefinition groupDefinition

Group Definition

Block: Basic (0) / Group Definition (7)

Any[] groupParameters - Join or Consist group parameters
+RevisionDefinition[] revisions

Block: Basic (0) / Group Join Parameter (8)

+GroupDeclaration groupDeclaration

Block: Basic (0) / Group Consist Parameter (9)

+BlockDeclaration blockDeclaration

Block Declaration

Block: Basic (0) / Group Declaration (10)

+CatalogBlockSpecPath catalogBlockSpecPath - Specification of format defined as path in catalog
+Natural blockSpecRevision - Specification's revision number
BlockDefinition blockDefinition

Block Definition

Block: Basic (0) / Block Definition (11)

Any[] blockParameters - Join or Consist or List Join or List Consist block parameters
+RevisionDefinition[] revisions

Block: Basic (0) / Block Join Parameter (12)

+BlockDeclaration blockDeclaration

Block: Basic (0) / Block Consist Parameter (13)

+BlockDeclaration blockDeclaration

Block: Basic (0) / Block List Join Parameter (14)

+BlockDeclaration blockDeclaration

Block: Basic (0) / Block List Consist Parameter (15)

+BlockDeclaration blockDeclaration

Revision Definition

Block: Basic (0) / Revision Definition (16)

Block allows to define parameters count for particular specification definition.

+Natural parametersCount

Catalog

XBUP protocol uses unified centralized catalog of type specifications. To access this centralized catalog, there is provided interface for catalog service, available on the xbup.org project server.

TODO

Transformations (Level 2)