Properties

HDF5 property lists are collections of name-value pairs which can be passed to other HDF5 functions to control features that are typically unimportant or whose default values are usually used. In HDF5.jl, these options are typically handled by keyword arguments to such functions, which will internally create the appropriate Properties objects, and so users will not usually be required to construct them manually.

Not all properties defined by the HDF5 library are currently available in HDF5.jl. If you require additional properties, please open an issue or pull request.

Common functions

HDF5.setproperties!Function
setproperties!(props::Properties...; kwargs...)

For each (key, value) pair in kwargs, set the corresponding properties in each Properties object in props. Returns a Dict of any pairs which didn't match properties in props.

source

Properties types

HDF5.AttributeCreatePropertiesType
AttributeCreateProperties(;kws...)
AttributeCreateProperties(f::Function; kws...)

Properties used when creating attributes.

  • char_encoding: the character enconding, either :ascii or :utf8.

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.FileAccessPropertiesType
FileAccessProperties(;kws...)
FileAccessProperties(f::Function; kws...)

Properties used when accessing files.

  • alignment :: Tuple{Integer, Integer}: a (threshold, alignment) pair: any file object greater than or equal in size to threshold bytes will be aligned on an address which is a multiple of alignment. Default values are 1, implying no alignment.

  • driver: the file driver used to access the file. See Drivers.

  • driver_info (get only)

  • fclose_degree: file close degree property. One of:

    • :weak
    • :semi
    • :strong
    • :default
  • libver_bounds: a (low, high) pair: low sets the earliest possible format versions that the library will use when creating objects in the file; high sets the latest format versions that the library will be allowed to use when creating objects in the file. Values can be a VersionNumber for the hdf5 library, :earliest, or :latest . See H5P_SET_LIBVER_BOUNDS

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.FileCreatePropertiesType
FileCreateProperties(;kws...)
FileCreateProperties(f::Function; kws...)

Properties used when creating a new File. Inherits from ObjectCreateProperties, with additional properties:

  • userblock :: Integer: user block size in bytes. The default user block size is 0; it may be set to any power of 2 equal to 512 or greater (512, 1024, 2048, etc.). See H5P_SET_USERBLOCK.
  • track_order :: Bool: tracks the file creation order.

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.GroupCreatePropertiesType
GroupCreateProperties(;kws...)
GroupCreateProperties(f::Function; kws...)

Properties used when creating a new Group. Inherits from ObjectCreateProperties, with additional options:

  • local_heap_size_hint :: Integer: the anticipated maximum local heap size in bytes. See H5P_SET_LOCAL_HEAP_SIZE_HINT.
  • track_order :: Bool: tracks the group creation order.

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.DatasetCreatePropertiesType
DatasetCreateProperties(;kws...)
DatasetCreateProperties(f::Function; kws...)

Properties used when creating a new Dataset. Inherits from ObjectCreateProperties, with additional properties:

  • alloc_time: the timing for the allocation of storage space for a dataset's raw data; one of:

    • :default

    • :early: allocate all space when the dataset is created

    • :incremental: Allocate space incrementally, as data is written to the dataset

    • :late: Allocate all space when data is first written to the dataset.

    See H5P_SET_ALLOC_TIME.

  • fill_time: the timing of when the dataset should be filled; one of:

    • :alloc: Fill when allocated
    • :never: Never fill
    • :ifset: Fill if a value is set
  • fill_value: the fill value for a dataset. See H5P_SET_FILL_VALUE.

  • chunk: a tuple containing the size of the chunks to store each dimension. See H5P_SET_CHUNK (note that this uses Julia's column-major ordering).

  • external: A tuple of (name,offset,size), See H5P_SET_EXTERNAL.

  • filters (only valid when layout=:chunked): a filter or vector of filters that are applied to applied to each chunk of a dataset, see Filters. When accessed, will return a Filters.FilterPipeline object that can be modified in-place.

  • layout: the type of storage used to store the raw data for a dataset. Can be one of:

    • :compact: Store raw data in the dataset object header in file. This should only be used for datasets with small amounts of raw data.

    • :contiguous: Store raw data separately from the object header in one large chunk in the file.

    • :chunked: Store raw data separately from the object header as chunks of data in separate locations in the file.

    • :virtual: Draw raw data from multiple datasets in different files. See the virtual property below.

    See H5P_SET_LAYOUT.

  • no_attrs_hint: Minimize the space for dataset metadata by hinting that no attributes will be added if set to true. Attributes can still be added but may exist elsewhere within the file. See H5P_SET_DSET_NO_ATTRS_HINT.

  • virtual: when specified, creates a virtual dataset (VDS). The argument should be a "virtuala collection of VirtualMapping objects for describing the mapping from the dataset to the source datasets. When accessed, returns a VirtualLayout object.

The following options are shortcuts for the various filters, and are set-only. They will be appended to the filter pipeline in the order in which they appear

  • blosc = true | level: set the BloscExt.BloscFilter compression filter; argument can be either true, or the compression level.

  • deflate = true | level: set the Filters.Deflate compression filter; argument can be either true, or the compression level.

  • fletcher32 = true: set the Filters.Fletcher32 checksum filter.

  • shuffle = true: set the Filters.Shuffle filter.

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.DatasetAccessPropertiesType
DatasetAccessProperties(;kws...)
DatasetAccessProperties(f::Function; kws...)

Properties that control access to data in external, virtual, and chunked datasets.

  • chunk_cache: Chunk cache parameters as (nslots, nbytes, w0). Default: (521, 0x100000, 0.75)
  • efile_prefix: Path prefix for reading external files. The default is the current working directory.
    • :origin: alias for raw"$ORIGIN" will make the external file relative to the HDF5 file.
  • virtual_prefix: Path prefix for reading virtual datasets.
  • virtual_printf_gap: The maximum number of missing source files and/or datasets with the printf-style names when getting the extent of an unlimited virtual dataset
  • virtual_view: Influences whether the view of the virtual dataset includes or excludes missing mapped elements
    • :first_missing: includes all data before the first missing mapped data
    • :last_available: includes all available mapped data

A function argument passed via do will be given an initialized property list that will be closed.

See Dataset Access Properties

source
HDF5.DatasetTransferPropertiesType
DatasetTransferProperties(;kws...)
DatasetTransferProperties(f::Function; kws...)

Properties used when transferring data to/from datasets

  • dxpl_mpio: MPI transfer mode when using Drivers.MPIO file driver:
    • :independent: use independent I/O access (default),
    • :collective: use collective I/O access.

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.LinkCreatePropertiesType
LinkCreateProperties(;kws...)
LinkCreateProperties(f::Function; kws...)

Properties used when creating links.

  • char_encoding: the character enconding, either :ascii or :utf8.

  • create_intermediate_group :: Bool: if true, will create missing intermediate groups.

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.ObjectCreatePropertiesType
ObjectCreateProperties(;kws...)
ObjectCreateProperties(f::Function; kws...)

Properties used when creating a new object. Available options:

  • obj_track_times :: Bool: governs the recording of times associated with an object. If set to true, time data will be recorded. See H5P_SET_OBJ_TRACK_TIMES.

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.StringCreatePropertiesType
StringCreateProperties(;kws...)
StringCreateProperties(f::Function; kws...)

A function argument passed via do will be given an initialized property list that will be closed.

source
HDF5.DatatypeCreatePropertiesType
DatatypeCreateProperties(;kws...)
DatatypeCreateProperties(f::Function; kws...)

A function argument passed via do will be given an initialized property list that will be closed.

source

Virtual Datasets

HDF5.VirtualMappingType
VirtualMapping(
    vspace::Dataspace,
    srcfile::AbstractString,
    srcdset::AbstractString,
    srcspace::Dataspace
)

Specify a map of elements of the virtual dataset (VDS) described by vspace to the elements of the source dataset described by srcspace. The source dataset is identified by the name of the file where it is located, srcfile, and the name of the dataset, srcdset.

Both srcfile and srcdset support "printf"-style formats with %b being replaced by the block count of the selection.

For more details on how source file resolution works, see H5P_SET_VIRTUAL.

source
HDF5.VirtualLayoutType
VirtualLayout(dcpl::DatasetCreateProperties)

The collection of VirtualMappings associated with dcpl. This is an AbstractVector{VirtualMapping}, supporting length, getindex and push!.

source

Drivers

File drivers determine how the HDF5 is accessed. These can be set as the driver property in FileAccessProperties.

HDF5.Drivers.CoreType
Core([increment::Csize_t, backing_store::Cuint, [write_tracking::Cuint, page_size::Csize_t]])
Core(; increment::Csize_t = 8192, backing_store::Cuint = true, write_tracking::Cuint = false, page_size::Csize_t = 524288)

Arguments

  • increment: specifies the increment by which allocated memory is to be increased each time more memory is required. (default: 8192)
  • backing_store: Boolean flag indicating whether to write the file contents to disk when the file is closed. (default: true)
  • write_tracking: Boolean flag indicating whether write tracking is enabled. (default: false)
  • page_size: Size, in bytes, of write aggregation pages. (default: 524288)
source
HDF5.Drivers.POSIXType
POSIX()

Also referred to as SEC2, this driver uses POSIX file-system functions like read and write to perform I/O to a single, permanent file on local disk with no system buffering. This driver is POSIX-compliant and is the default file driver for all systems.

source
HDF5.Drivers.ROS3Type
ROS3()
ROS3(aws_region::String, secret_id::String, secret_key::String)
ROS3(version::Int32, authenticate::Bool, aws_region::String, secret_id::String, secret_key::String)

This is the read-only virtual driver that enables access to HDF5 objects stored in AWS S3

source

Internals

The following macros are used for defining new properties and property getters/setters.

HDF5.@propertyclassMacro
@propertyclass P classid

Define a new subtype of P <: Properties corresponding to a HDF5 property list with class identifier classid.

Once defined, the following interfaces can be defined:

superclass(::Type{P})

This should return the type from which P inherits. If not defined, it will inherit from GenericProperties.

class_propertynames(::Type{P})

This should return a Tuple of Symbols, being the names of the properties associated with P.

class_getproperty(::Type{P}, p::Properties, name::Symbol)

If name is an associated property of type P, this should return the value of the property, otherwise call class_getproperty(superclass(P), p, name).

class_setproperty!(::Type{P}, p::Properties, name::Symbol, val)

If name is an associated property of type P, this should set the value of the property, otherwise call class_setproperty!(superclass(P), p, name, val).

source
HDF5.@bool_propertyMacro
@bool_property(name)

Wrap property getter/setter API functions that use 0/1 to use Bool values

source
HDF5.@enum_propertyMacro
@enum_property(name, sym1 => enumvalue1, sym2 => enumvalue2, ...)

Wrap property getter/setter API functions that use enum values to use symbol instead.

source