User Tools

Site Tools

Site » Documentation » Concept » Type_system 
en:doc:concept:type_system

This is an old revision of the document!


Concept: Type System

With data structure defined using only blocks with attributes or blobs, next step should define a way how to use them and distinguish what certain values represents. First step related to this will be introduced in this concept and it's definition of type system.

Block Type

For all non-data block, first two attributes are reserved for block type:

UBNatural - TypeGroup
UBNatural - BlockType

Blocks are organized by type into groups and the TypeGroup value determines to which group the block belongs to. The value 0 means that it is a basic group is used, which built-in group accesible all the time. The BlockType determines the specific type of block in the group. Allowed ranges of block type values for other than basic group and therefore the meaning of groups of blocks is defined using declaration block, which is in basic group.

Document Type

First value at the beginning of the stream is a sequence of bits that specifies used numerical encoding cluster size.

Byte - ClusterSize = FEh (11111110 in bit stream)

This value is unary encoded, which means ClusterSize = 7, but effectively defines cluster with size 8 bits.

For development purposes the header of the file next contains few identification chars.

UBNatural - ProtocolVersion = 00h
DWord(4xUBNatural) - ProtocolSignature = 58 42 00 XXh ("XB" + development version)

Protocol version 0 is reserved for the protocol development stage. The development version then specify particular structure of the file and any incompatible changes shall be reflected in a change to this value. The header can be interpreted as a 32-bit identification number and 16-bit version number. It is assumed that the final version will have different file headers.

Data is then processed as a single block, and data after that block are called extended area, which is cosidered as general, potentially infinite data blob.

Block Declaration

To define block, there are two kinds of possible operations available. Consist operation defines type of next subblock, while join operation performs inclusion of definition for both attributes and subblocks from given block type. Same two operations can be used for declaration of format and group.

For the block specification, target block types for these two operations are not required and there are also defined two additional operations for finite and infinite list, which means there is 8 operations in total:

  • Consist - Adds single child block having declared type
  • Join - Merges current block with given block appending attributes and child types
  • Any - Adds one general child block (consist operation on undefined block)
  • Attribute - Adds one single general attribute (join operation on undefined block)
  • Infinite list - Adds UBENatural attribute with count and list of of potential infinite subblocks having declared type
  • Finite list - Adds UBNatural attribute and merges list of declared types using join operation.
  • Any list - Adds potentially infinite list of subblocks of undeclared type (infinite list of undefined block)
  • Attributes list - Adds finite list of genera attributes (finite list of undefined block)

FIXME

The block definition allows to define the attributes and parameters.

The definition of a type system are stored in the catalog of types, where it is possible to use your own definitions using the built in basic blocks and later it should be possible to add the definition from any source.

The following chart shows the ER diagram of the type definition in the catalog of types, including the tree hierarchy of categories of the definitions:

Type definition's ER diagram

Diagram source file diagram4.dia

Basic Group

Basic group uses TypeGroup = 0. Basic blocks provides ability to store block type declarations, definitions and links to catalog.

Undefined

This block is used for unknown block types or data padding.

Block: Basic (0) / Unknown (0)

Declaration

Declaration block determines the allowed range of groups. This block should be located at the beginning of each file, if the application didn't provide any static/special meaning, but it might be used anywhere inside document as well.

Block: Basic (0) / Document Declaration (1)

+Natural groupsCount - The number of allocated groups
+Natural preserveGroups - The number of groups to keep from previous declarations
FormatDeclaration formatDeclaration - Declaration of format
Any documentRoot - Root node of document

For subblocks of this block there is permitted range of values in the interval group preserveGroups + 1 .. preserveGroups + groupsCount + 1. preservedGroups + groupsCount + 1. If the value reserveGroups = 0, takes the highest not yet reserved group in the current or parental blocks + 1. For all values of zero and the application of rules of cutting the block of zeros coincides with the data block.

Format Declaration

Format declaration allows you use either declaration from catalog or local format definition or both.

Block: Basic (0) / Format Declaration (2)

+CatalogFormatSpecPath catalogFormatSpecPath - Specification of format defined as path in catalog
+Natural formatSpecRevision - Specification's revision number
FormatDefinition formatDefinition

Format Definition

This block allows to specify the basic structure of format specification. Specifies the sequence of parameters using either join or consist operation.

Block: Basic (0) / Format Definition (3)

Any[] formatParameters - Join or Consist format parameters
+RevisionDefinition[] revisions

Block: Basic (0) / Format Join Parameter (4)

+FormatDeclaration formatDeclaration

Block: Basic (0) / Format Consist Parameter (5)

+GroupDeclaration groupDeclaration

Group Declaration

Group declaration allows you use either declaration from catalog or local group definition or both.

Block: Basic (0) / Group Declaration (6)

+CatalogGroupSpecPath catalogGroupSpecPath - Specification of format defined as path in catalog
+Natural groupSpecRevision - Specification's revision number
GroupDefinition groupDefinition

Group Definition

This block allows to specify the basic structure of group specification. Specifies the sequence of parameters using either join or consist operation.

Block: Basic (0) / Group Definition (7)

Any[] groupParameters - Join or Consist group parameters
+RevisionDefinition[] revisions

Block: Basic (0) / Group Join Parameter (8)

+GroupDeclaration groupDeclaration

Block: Basic (0) / Group Consist Parameter (9)

+BlockDeclaration blockDeclaration

Block Declaration

Block declaration allows you use either declaration from catalog or local block definition or both.

Block: Basic (0) / Group Declaration (10)

+CatalogBlockSpecPath catalogBlockSpecPath - Specification of format defined as path in catalog
+Natural blockSpecRevision - Specification's revision number
BlockDefinition blockDefinition

Block Definition

This block allows to specify the basic structure of block specification. Specifies the sequence of parameters using either join, consist, list join or list consist operation.

Block: Basic (0) / Block Definition (11)

Any[] blockParameters - Join or Consist or List Join or List Consist block parameters
+RevisionDefinition[] revisions

Block: Basic (0) / Block Join Parameter (12)

+BlockDeclaration blockDeclaration

Block: Basic (0) / Block Consist Parameter (13)

+BlockDeclaration blockDeclaration

Block: Basic (0) / Block List Join Parameter (14)

+BlockDeclaration blockDeclaration

Block: Basic (0) / Block List Consist Parameter (15)

+BlockDeclaration blockDeclaration

Revision Definition

Block allows to define parameters count for particular specification definition.

Block: Basic (0) / Revision Definition (16)

+Natural parametersCount

Type Context

For each block, there is type context which provides mapping of particular block type (as defined above) to particular declaration/definition. Context is typically same for block and it's childer, except for declaration blok and it's formatDeclaration parameter, which affects context of documentRoot block and also incrementally affects group and block definitions which can use block types already defined in currently processed declaration.

FIXME

Basic Types

Basic types correspond to the above-mentioned types of attributes.

Compound Block Types

This group of blocks is needed for the construction of more complex blocks, which are consisting of more simpler parts. This is essentially about sequences, and collections. Examples of the use can be found in some already defined document specification for lists of blocks and groups.

Document Specification

Here are described some of the possible ways how to define the type of blocks in the document. (obsolete)

Document Definition

The definition of a document is a separate document determining the permitted ranges of groups and block types. In the case of the specifications it points the values of GroupListPointer and DocumentRootPointer to the same block.

Document Processing

The following text describe the how to deal with the document specifications. This is mainly about the techniques of how to perform control checks and connect specifications into the sequence.

Specification's Processing

Defined specifications should be processed using appropriate method. Although it is possible to store the table for each block, it would be very inefficient. The outline of usable proposed method follows.

Active Specification

Current specification maintains values of the indexes to the catalog for the currently processed element and keeps a list of the existing range of groups up to lower levels. In the case that we want to handle another block of the document, it's possible to travel up in the tree so far as is necessary and delete definition of groups using the table. After that going through the blocks the way to the desired node and process block specifications.

Preprocessed Specifications

Lets walk through the document to depth and prepare a specification table for each specification block. For the current block it is possible to get copy of the specification. In the processing of another lets walk through his ancestries, until we hit on the specification block, which table we can use.

Document Validation

The rules for each level should be checked for compliance with the required limits. The corruption might be caused by a mistake of the applications, or with the file damages. Checking the document is split on the rules for determining the validity and to determine the document compatibility. While the validity determines if the file is properly written and, therefore, is processible for real work, compatibility checks to determine whether document is possible to use in the specific application. In the case of the XBUP protocol validation methods forms similar hierarchy as levels.

Document Validity

The document is valid if it is properly created and all types of blocks and their attributes are properly defined. This precisely means:

  • The document is properly created, it was defined at the level of 0
  • BlockGroup value of each block does not exceed permitted range in its context (WrongGroup)
  • BlockType value of each block does not exceed the permitted range of values in its group (WrongType)
  • The count of the attributes of each block is not larger in its scope than allowed (TooManyAttributes)

Document Compatibility

Compatibility is a property of the document saying that this document is processible by the given applications. The application is compatible if:

  • The main document version is in supported set of values
  • The value of minor version is equal to or greater than the value supported for given block

Todo:

  • Define the type of items including, for example, UBPointer the next level + relevant types of linked subblocks
  • Argumentation for the order of the system blocks
  • Argumentation for the order of attributes
  • Block for definition of auxiliary data with while keeping the meaning
en/doc/concept/type_system.1418665223.txt.gz · Last modified: 2014/12/15 18:40 by hajdam