Follow

# An explanation of the MTi fixed point output formats

The MTi can output data in several precision or output formats. IEEE floating point (32-bit) is the standard format. For most data outputs there is the possibility to use fixed point 12.20 and fixed point 16.32 number formats as well as IEEE double precision floating point (64-bit). This article describes the fixed point 12.20 and fixed point 16.32 number formats.

A fixed point number format maps a real number onto an integer number by applying a fixed scaling to it. If for example you have real value 3.1415 and a fixed scaling of 100, the resulting integer value would be 3.1415 * 100 = 314. Reconstructing the real number from this value means applying the reverse operation: 314 / 100 = 3.14.

In practice the scaling values used are powers of 2 since multiplication and division by such values can be done by shift operations and shift operations are very efficient on most processor architectures. The advantage of fixed point numbers is that using them, under a certain set of rules, they allow for high performance math operations on processors that lack a hardware floating point unit (FPU). So one of the main purposes for our fixed point output is to accommodate users having such a processor to efficiently make use of our data as input without the need for expensive floating point conversions.

## Notation, range and resolution

Xsens' notates fixed point values in the form "m.f" where "m" is the number of integer bits and "f" the number of fractional bits. The "f" determines the scaling value which is equal to 2f. The total size of the integer is m + f. The resolution (the smallest difference between two numbers) is equal to 1/2f.

So for the 12.20 fixed point format the integer part has 12 bits, the scaling value is 220 and the size of the integer is 12 + 20 = 32. The resolution = 1/220 = 0,00000095367431640625

Xsens' fixed point numbers are always signed and the sign bit is included in the integer part. Therefore the range of the fixed point number is [-2(m-1) .. 2(m-1) - (1 / 2f)]

For the 12.20 format the range is therefore [-211 .. 211 - (1 / 220)] ≈ [-2048.0 .. 2047.9999]

## Formats used in MTi products

The following section describes which formats are used, how they are constructed and how they are transmitted.

### Fixed point 12.20 format

The size of the number is (12 + 20) = 32 bits. This 32-bit number is transported as 4 bytes a 12-bit integral part followed by the 20-bit fractional part. Both parts are transmitted most significant byte first (big-endian). Note that this is in a different order than the fixed point 16.32 format. It is calculated from a real number as:

i = round(realNumber * 220)

The range of fixed point 12.20 numbers in MTi products is [-2048.0 .. 2047.9999990].

Example:

Real number = 3.1415

i = round(3.1415 * 220) = 3294102 = 0x324396 = {0x003 integer part, 0x24396 fractional part}

This number is transmitted as the byte sequence: {0x00, 0x32, 0x43, 0x96}

### Fixed point 16.32 format

The size of the number is (16 + 32) = 48 bits. This 48-bit number is transported as a 32-bit fractional part followed by the 16-bit integer part. Both parts are transmitted most significant byte first (big-endian). The total amount of bytes is 6. Note that this is in a different order than the fixed point 12.20 format. It is calculated from a real number as:

i = round(realNumber * 232)

The range of fixed point 16.32 numbers in MTi products is [-32768.0 .. 32767.9999999998].

Example:

realNumber = 3.1415

i = round(3.1415 * 232) = 13492639760 = 0x324395810 = {0x0003 integer part, 0x24395810 fractional part}

This number is transmitted as the byte sequence: {0x24, 0x39, 0x58, 0x10, 0x00, 0x03

## Other reasons for fixed point use in MTi’s

Another reason to use a fixed point format is to have a larger number of significant bits. Although a floating point value has a larger range it can be less precise for certain number ranges. A single precision IEEE floating point has 23 significant bits where for example Xsens' 12.20fp format has 32 and the 16.32fp format 48. A double precision IEEE floating point has 53 significant bits.

In Xsens products, the single precision floating point format is precise enough for almost all outputs. For the latitude and longitude positional outputs however it is not sufficient. To achieve the required 1mm precision, these quantities need to be outputted using at least 8 decimal places. A minimum of 27 fractional bits for that. Besides the 8 decimal places the format also needs 3 digits before the decimal point to represent the integer part of the quantity as longitude lies in the range [-180 .. 180]. For these 3 digits, another 8 significant bits are required. This brings the total to 35 significant bits required. So both single precision floats and 12.20 fixed point values are not precise enough. Therefore it is strongly advised to output these quantities as 16.32 fixed point  or double precision floating point values. The advantage of using 16.32 over doubles is the fewer transmitted data bytes.