Advanced Usage

Explicit Type Remapping

Sometimes you store data using structs that you defined yourself or are shipped with some package and weeks later, when you want to load the data, the structs have changed.

using JLD2
struct A
    x::Int
end
jldsave("example.jld2"; a = A(42))

This results in warnings and sometimes even errors when trying to load the file as demonstrated here.

julia> using JLD2

julia> struct A{T}
           x::T
       end

julia> load("example.jld2")
┌ Warning: read type A is not a leaf type in workspace; reconstructing
└ @ JLD2 ~/.julia/dev/JLD2/src/data/reconstructing_datatypes.jl:273
Dict{String, Any} with 1 entry:
  "a" => var"##A#257"(42)

As of JLD2 version 0.4.21 there is a fix. The JLDFile struct contains a typemap dictionary that allows for explicit type remapping. Now you can define a struct that matches the old definition and load your data.

julia> struct A_old
           x::Int
       end

julia> f = jldopen("example.jld2","r"; typemap=Dict("Main.A" => A_old))
JLDFile /home/jonas/.julia/dev/JLD2/example.jld2 (read-only)
 └─🔢 a

julia> f["a"]
A_old(42)

Upgrading old structures on load

The section above explains how you can make JLD2 load old structs with a different DataType name as target. A different method for loading old data is described here:

# This is the old version of the struct stored in the file
struct OldStructVersion
    x::Int
    y::Float64
end
orig = OldStructVersion(1,2.0)
jldsave("test.jld2"; data=orig)

### new session

# This is the new version of your struct
struct UpdatedStruct
    x::Float64 # no longer int
    y::Float64
    z::Float64 # = x*y
end

# When upgrading a struct, JLD2 will load the fields of the old struct into a `NamedTuple`
# and call `rconvert` on it. Here we implement a conversion method that returns an `UpdatedStruct`
JLD2.rconvert(::Type{UpdatedStruct}, nt::NamedTuple) = UpdatedStruct(Float64(nt.x), nt.y, nt.x*nt.y)

# Here we provide the `typemap` keyword argument. It is a dictionary mapping the stored struct name
# to an `Upgrade` instance with the new struct.
load("test.jld2", "data"; typemap=Dict("Main.OldStructVersion" => JLD2.Upgrade(UpdatedStruct)))

Groups - Appending to files

Group objects can be constructed with two optional keyword arguments:

g = Group(file;
          est_num_entries=4
          est_link_name_len=8)

These determine how much (additional) empty space should be allocated for the group description. (list of entries) This can be useful for performance when one expects to append many additional datasets after first writing the file.

JLD2DebugTools

There is an experimental repository JLD2DebugTools.jl that may help with debugging files.

Fallback Behaviour

By default JLD2 will attempt to open files using the MmapIO backend. If that fails, it retries using IOStream.