Describe a binary record — bin

bin_layout() parses a specification of a fixed-size binary record into a zubin_layout: the type, element count, size, 0-based byte offset and byte order of each field, the record's size, and its alignment. Offsets are 0-based, as in every file format specification.

Usage

bin_layout(spec, endian = c("little", "big", "native"), align = FALSE)

Arguments

spec: A specification string; or a named character vector of type tokens, whose names are the field names (c(id = "u32", "x3")); or a zubin_layout, which is returned unchanged.
endian: The byte order of fields the specification does not order itself: "little", "big", or "native", the host's (see bin_info()).
align: FALSE for packed fields, TRUE for C struct alignment.

Value

A zubin_layout: a list with spec, the normalised specification; fields, a data frame with columns name, type, count (array elements, 1 for a scalar), size, offset and endian (NA for one-byte types); size, the record size in bytes; and align, the record's alignment (1 when packed). length() and names() describe the fields that carry values, which excludes padding; as.data.frame() returns fields; bin_size() returns size.

The specification

One string of fields separated by whitespace or a comma:

layout  := [ endian ] field { sep field }
endian  := "<" | ">" | "="          little, big, host; default `endian`
field   := [ name ":" ] type [ "[" count "]" ] [ "." order ]
         | "x" width                padding: never read, written as zeros
type    := u8 i8 u16 i16 u32 i32 u64 i64 u128 i128
         | f16 bf16 f32 f64 c64 c128 bool
         | "b" width                fixed bytes
         | "s" width                fixed string
order   := "le" | "be"              multi-byte numeric types only
name    := [A-Za-z_] [A-Za-z0-9_.]*
count, width := 1 .. 2^31 - 1

For example "<magic:u32 version:u16 flags:u16 count:u64 name:s32", or "<ts:i64 price:f64 qty:i32 side:u8 x3 crc:u32.be". An endian prefix overrides endian, and a field's .le or .be suffix overrides both. Names are unique; an unnamed field is called V1, V2, ... by its position among the fields that are not padding. t[k] is an array of k elements, for the numeric types and bool only (write b24, not b8[3]). Every malformed specification is a zubin_spec_error carrying position, the 0-based byte position of the problem in the string.

Types

The type model, in both directions (bin_unpack() and bin_decode() read; bin_pack(), bin_encode() and bin_put() write):

Spec	Bytes	Read as	Written from
`u8 i8 u16 i16`	1–2	integer	integer, whole double, logical
`i32`	4	integer; -2^31 is an error unless `na = "allow"`	integer, whole double; `NA` only with `na = "allow"`
`u32`	4	double	integer, whole double
`i64 u64`	8	double, an error above 2^53 in magnitude; or `integer64` (`int64 = "integer64"`), an error for `u64` from 2^63	integer, whole double, `integer64`
`i128 u128`	16	as `i64`: double, an error above 2^53 in magnitude; or `integer64`, an error outside its range	integer, whole double (any whole value in range is exact), `integer64`
`f16 bf16`	2	double	numeric, rounded to nearest even
`f32 f64`	4, 8	double	numeric; `NA` survives `f64` and is a NaN in the others
`c64 c128`	8, 16	complex: two `f32` or two `f64`, real part first	complex, or numeric as a real part; `c64` rounds each part as `f32`
`bool`	1	logical; any non-zero byte is `TRUE`	logical, not `NA`
`b<n>`	n	a list of raw vectors	a list of raw vectors of exactly n bytes
`s<n>`	n	character: the bytes before the first NUL, as UTF-8	character of at most n UTF-8 bytes, NUL-padded
`x<n>`	n	not returned	not given; written as zeros
`t[k]`	k x width	a matrix of k columns (`name.1` ... `name.k` in a data frame)	a matrix of k columns, or one record's k values

A value that does not fit is a zubin_range_error, and an NA with no bytes a zubin_na_error; nothing is truncated, wrapped or rounded into an integer. See zubin-conditions.

Alignment

With align = FALSE fields are packed at consecutive offsets. With align = TRUE each field starts at the next multiple of its element width (1 for bool, b, s and x; a complex type aligns as its part, c64 to 4 and c128 to 8; i128 and u128 to 16) and the record size is rounded up to the largest of them: the rule C compilers apply to a struct on 64-bit targets, so a struct written with fwrite() reads back with align = TRUE. (32-bit x86 aligns 8-byte members to 4 inside a struct; such a struct needs explicit x padding.)

Examples

wav <- bin_layout("<riff:s4 size:u32 wave:s4")
wav
#> <zubin_layout: 12 bytes, 3 fields>
#>  name type count size offset endian
#>  riff s4   1     4    0            
#>  size u32  1     4    4      little
#>  wave s4   1     4    8            
bin_size(wav)
#> [1] 12
wav$fields
#>   name type count size offset endian
#> 1 riff   s4     1    4      0   <NA>
#> 2 size  u32     1    4      4 little
#> 3 wave   s4     1    4      8   <NA>

# The same, built programmatically
bin_layout(c(riff = "s4", size = "u32", wave = "s4"))
#> <zubin_layout: 12 bytes, 3 fields>
#>  name type count size offset endian
#>  riff s4   1     4    0            
#>  size u32  1     4    4      little
#>  wave s4   1     4    8            

# A C struct { uint8_t a; uint32_t b; uint16_t c; }
bin_layout("a:u8 b:u32 c:u16", endian = "native", align = TRUE)$fields
#>   name type count size offset endian
#> 1    a   u8     1    1      0   <NA>
#> 2    b  u32     1    4      4 little
#> 3    c  u16     1    2      8 little