Get Started

Zarr.jl is a Julia package for working with chunked, compressed, N-dimensional arrays, compatible with the Zarr format used across Python, Rust, and other ecosystems.

Installation

Install Julia v1.10 or above. Zarr.jl is available through the Julia package manager. You can enter it by pressing ] in the REPL and then typing add Zarr:

(@v1.x) pkg> add Zarr

alternatively, you can also do:

import Pkg
Pkg.add("Zarr")

It is recommended to check the version of Zarr.jl you have installed with the status command:

(@v1.x) pkg> status Zarr

or:

import Pkg
Pkg.status("Zarr")

Status `~/work/Zarr.jl/Zarr.jl/docs/Project.toml`
  [0a941bbe] Zarr v0.11.0 `Zarr`

Where not shown explicitly, this documentation assumes using Zarr has been evaluated in your session:

using Zarr

Creating Arrays

Use zcreate for new arrays, zzeros for zero-initialized ones, or ZArray to wrap an existing Julia array:

using Zarr

z = zcreate(Float32, 1000, 1000;
    chunks=(100, 100),
    fill_value=Float32(0),
    zarr_format=3,
    path="example_v3.zarr")

ZArray{Float32} of size 1000 x 1000

you can also wrap an existing Julia array:

using Zarr
z = ZArray(rand(Float64, 100, 100))

ZArray{Float64} of size 100 x 100

zinfo(z)

Type                : ZArray
Data type           : Float64
Shape               : (100, 100)
Chunk Shape         : (100, 100)
Order               : C
Read-Only           : false
Compressor          : ZarrBlosc.BloscCompressor(0, 5, "lz4", 1)
Filters             : nothing
Store type          : Dictionary Storage
No. bytes           : 80000
No. bytes stored    : 70102
Storage ratio       : 1.1411942597928733
Chunks initialized  : 1/1

Reading and Writing

using Zarr

z = zcreate(Float32, 1000, 1000;
    chunks=(100, 100),
    fill_value=Float32(0),
    path="example.zarr")

z[:, :] = rand(Float32, 1000, 1000)  # write entire array
z[1, :] = 1:1000                     # write a row

subset = z[1:3, 1:10]                # read a subregion

3×10 Matrix{Float32}:
 1.0       2.0       3.0        4.0       …  8.0       9.0       10.0
 0.54153   0.383932  0.0776721  0.458725     0.111857  0.635939   0.850811
 0.452845  0.948826  0.111199   0.955074     0.874294  0.246383   0.46937

Opening Existing Arrays

Zarr automatically detects v2 or v3 format on open:

z = zopen("example.zarr")

println(size(z))    # (1000, 1000)
println(eltype(z))  # Float32

(1000, 1000)
Float32

WARNING

zopen throws an ArgumentError if the path does not exist. Make sure the path points to a valid Zarr store.

Missing Values

Use fill_value and fill_as_missing together to handle missing data:

using Zarr

z = zcreate(Int64, 10, 10;
    chunks=(5, 2),
    fill_value=-1,
    fill_as_missing=true)

z[:, 1] = 1:10          # write a column
z[:, 2] .= missing      # mark a column as missing

println(eltype(z))               # Union{Int64, Missing}
println(all(ismissing, z[:, 2])) # true

Union{Missing, Int64}
true

Re-open with or without missing support:

# treat fill_value as missing
z = zopen("example.zarr", fill_as_missing=true)

ZArray{Union{Missing, Float32}} of size 1000 x 1000

or treat fill_value as a regular value

z = zopen("example.zarr")

ZArray{Float32} of size 1000 x 1000

Compression

Zarr uses Blosc (lz4, level 5) by default. Several compressors are available:

using Zarr

z = zcreate(Int32, 1000, 1000;
    chunks=(100, 100),
    compressor=Zarr.BloscCompressor(cname="zstd", clevel=3, shuffle=true),
    fill_value=Int32(0),
    path="blosc.zarr")

z[:,:] = Int32(1):Int32(1000*1000)
storageratio(z)

51.39802631578947

Controlling extension registration

By default, Zarr's compressor and storage extension packages register their codecs, compressors, and URL handlers when Julia loads them. To manage those registries yourself, add this preference to LocalPreferences.toml in the active Julia environment:

[ZarrCore]
RegisterAtInit = false

Restart Julia after changing the preference. You can then register only the packages needed by the current process, using qualified calls such as:

Zarr.ZarrBlosc.register!()
Zarr.ZarrS3.register!()

The same functions are available when importing a package directly, for example ZarrBlosc.register!(). Explicit calls work regardless of the preference and affect only the current process. Disabling automatic registration does not unload package types or methods, and it does not change Blosc's default-compressor dispatch. ZarrCore's built-in registry entries also remain available.

If a downstream package requires one of these registrations, call the qualified register! from that package's runtime __init__. Registry changes made only while precompiling are not preserved when the package is loaded.

Resizing and Appending

using Zarr

z = zzeros(Int64, 10, 10; chunks=(5, 2), fill_value=-1)

ZArray{Int64} of size 10 x 10

grow first dimension

resize!(z, 20, 10)
z

ZArray{Int64} of size 20 x 10

appends columns

append!(z, rand(Int64, 20, 5))
z

ZArray{Int64} of size 20 x 15

append a row

append!(z, rand(Int64, 15), dims=1)
z

ZArray{Int64} of size 21 x 15

Groups

Zarr allows you to create hierarchical groups, similar to directories:

using Zarr

store = Zarr.DirectoryStore("experiment.zarr")
g = zgroup(store, "", 3)  # 3 selects the Zarr v3 format

zcreate(Float64, g, "temperature", 100, 100; chunks=(50, 50), fill_value=0.0)
zcreate(Float64, g, "precipitation", 100, 100; chunks=(50, 50), fill_value=0.0)
g

ZarrGroup at DirectoryStore("experiment.zarr") and path 
Variables: temperature precipitation

Navigate into a group to access its arrays:

temp = g["temperature"]
println(size(temp))  # (100, 100)

(100, 100)

Storage Backends

Zarr supports several storage backends out of the box:

z = zopen("example.zarr")

See Storage Backends for full details on credentials and configuration.

TIP

Ready for more? Head to the User Guide for a deeper dive.