Blosc2.jl Main API

Compressor

Representation of compressor supprted by blosc

available_compressors()::Dict{Symbol, Compressor}

Get dictionary of available compressors

available_compressors_names()::Vector{Symbol}

Get vector of available compressors names

has_compressor(name::Symbol)

Check if compressor name available

compressor(name::Symbol)

Get compressor by name

default_compressor()

Default compressor

default_compressor_name()

Name of default compressor

has_filter(name::Symbol)

Check if filter with name name is exists

available_filter_names()

Get vector with names of available filters

filter_description(name::Symbol)

Get desctription of filter name

filter_pipeline(args...)

Make filter pipeline. Each argument can be filter name (Symbol) or pair name=>meta ('Pair{Symbol, Integer}'). Each argument represents a filter in a chain of filters that are sequentially applied during compression Maximum number of filters is max_filters_count()

Examples

filter_pipeline(:trunc_prec=>23, :shuffle)

sizes(buff::Vector{UInt8}, offset = 1)

Get information about a compressed buffer at offset offset (1-indexed), as Tuple with values:

• the number of uncompressed bytes

• the number of compressed bytes

uncompressed_sizof(buff::Vector{UInt8}, offset = 1)

Uncompressed size of data in buffer buff in bytes`

uncompressed_length(::Type{T}, buff::Vector{UInt8}, offset = 1)

Uncompressed length of Vector{T} then decompressed from buffer buff

compressed_sizeof(buff::Vector{UInt8}, offset = 1)

Compressed size of data in buffer buff in bytes

CompressionParams

Compression params

CompressionParams(;
    compressor ::Symbol = default_compressor_name()
    level ::UInt8 = 5
    typesize ::Int32 = 8
    nthreads ::Int32 = 1
    blocksize ::Int32 = 0
    splitmode ::Bool = false
    filter_pipeline ::FilterPipeline = filter_pipeline(:shuffle)
)

Create compression parameters

Arguments

• compressor::Symbol - The name of compressor to use
• level - The compression level from 0 (no compression) to 9 (maximum compression)
• typesize - The size of type being compressed
• nthreads - The number of threads to use internally
• blocksize - The requested size of the compressed blocks (0 means auto)
• splitmode - Whether the blocks should be split or not.
• filter_pipeline - Filters pipeline

DecompressionParams

Decompression params

DecomplessionParams(;nthreads = 1)

Create decompression parameters

Context{T <: Params}
CompressionContext = Context{CompressionParams}
DecompressionContext = Context{DecompressionParams}

Constructors

Context{T}(params::T) where {T <: Params}

Create context.

• If params is CompressionParams then this is CompressionContext
• If params is DecompressionParams then this is DecompressionContext

CompressionContext(;kwargs...)

Create compression context kwargs passed to CompressionParams constructor

DecompressionContext(;kwargs...)

Create decompression context kwargs passed od DecompressionParams consturctor

CompressionContext(::Type{T}; kwargs...)

Create CompressionContext for compressing vectors with element type T. This is equivalent to CompressionContext(CompressionParams(T; kwargs...))`

Context{T <: Params}
CompressionContext = Context{CompressionParams}
DecompressionContext = Context{DecompressionParams}

Constructors

Context{T}(params::T) where {T <: Params}

Create context.

• If params is CompressionParams then this is CompressionContext
• If params is DecompressionParams then this is DecompressionContext

CompressionContext(;kwargs...)

Create compression context kwargs passed to CompressionParams constructor

DecompressionContext(;kwargs...)

Create decompression context kwargs passed od DecompressionParams consturctor

max_compression_overhead()

The maximum overhead during compression in bytes.

max_compressed_size(src)

The maximum size after compression in bytes.

max_compress_buffer(src)

Creates a buffer that is guaranteed to contain compressed data

unsafe_compress!([ctx = CompressionContext(T; kwargs...)], dest::Ptr{T}, dest_size, src::Ptr{T}, src_size; kwargs...)

Compress src_size bytes from src buffer, into dest buffer (with dest_size bytes size) using context ctx with no checks, return size of the data written to dest

The unsafe prefix on this function indicates that no validation is performed on the pointers dest and src to ensure that they are valid. Incorrect usage may corrupt or segfault your program, in the same manner as C.

compress!([ctx = CompressionContext(T; kwargs...)],
     dest::Vector{UInt8}, src::Vector{T},
     [dest_offset = 1], [src_range = 1:length(src)]
    ; kwargs...
    )

Compress elements from src_range of src into dest buffer starting from dest_offset (1-indexed) using context ctx Return size of the data written to dest

compress([ctx = CompressionContext(T; kwargs...)], src::Array{T}, [src_range = 1:length(src)]; kwargs...)::Vector{UInt8}

Return Vector{UInt8} consisting of src_range of src in compressed form using context ctx

unsafe_decompress!([ctx = DecompressionContext(;kwargs...)], dest::Ptr{T}, dest_size, src::Ptr{UInt8}, src_size; kwargs...)

Decompress data from src buffer, into dest buffer using context ctx with no checks, return count of elements written to dest or negative value if an error. dest_size and src_size are the byte sizes of buffers dest and src respectively

Return count of the elements written to dest

The unsafe prefix on this function indicates that no validation is performed on the pointers dest and src to ensure that they are valid. Incorrect usage may corrupt or segfault your program, in the same manner as C.

decompress!([ctx = DecompressionContext(;kwargs...)], dest::Vector{T}, src::Vector{UInt8}, dest_offset = 1, src_offset = 1; kwargs...)

Decompress data from buffer src at offset src_offset (1-indexed) into vector dest at offset dest_offset (1-indexed). Return count of the elements written to dest

decompress([ctx = DecompressionContext(kwargs...)], ::Type{T}, src::Array{UInt8}, [src_offset = 1]; kwargs...)::Vector{T}

Return Vector{T} consisting of decompression data from src starting at src_offset using context ctx

decompress_items!(ctx::DecompressionContext, dest::Vector{T},  src::Vector{UInt8}, range::UnitRange{<:Integer}, dest_offset, src_offset)

Decompress items in range from buffer src at offset src_offset (1-indexed) into vector dest at offset dest_offset (1-indexed). Return count of the elements written to dest