GZip.jl

A Julia interface for gzip functions in zlib, a free, general-purpose, legally unencumbered, lossless data-compression library.

GZip.jl provides GZipStream, a drop-in IO replacement for reading and writing gzip (.gz) files. It defaults to the high-performance zlib-ng backend; standard zlib is also available via backend=GZip.ZLIB.

Installation

using Pkg
Pkg.add("GZip")

Basic Usage

Writing

using GZip

GZip.open("output.gz", "w") do io
    write(io, "hello world\n")
    println(io, "another line")
end

Reading

# Read entire file as a string
data = GZip.open(read, "output.gz")

# Read line by line
GZip.open("output.gz") do io
    for line in eachline(io)
        println(line)
    end
end

Reading and Writing Arrays

# Write an array
a = rand(Float64, 1000)
GZip.open("array.gz", "w") do io
    write(io, a)
end

# Read it back
b = zeros(Float64, 1000)
GZip.open("array.gz") do io
    read(io, b)
end

Compression Levels

The mode string accepts a compression level (0-9) and strategy flags:

# Fast compression
GZip.open("fast.gz", "wb1") do io
    write(io, data)
end

# Best compression
GZip.open("small.gz", "wb9") do io
    write(io, data)
end

Header Metadata

gzheader reads gzip header fields (RFC 1952) without decompressing:

h = gzheader("data.gz")
h.name     # original filename, or nothing
h.mtime    # modification time as Unix timestamp (0 = not set)
h.comment  # file comment, or nothing
h.os       # OS identifier (0x03 = Unix)
h.extra    # extra field data, or nothing
h.is_text  # hint that content is ASCII text

File Descriptor I/O

gzdopen wraps an existing file descriptor as a gzip stream:

# Wrap an open IOStream's file descriptor
raw = open("data.gz", "r")
gz = gzdopen(fd(raw), "r")
data = read(gz, String)
close(gz)
close(raw)

gzdopen duplicates the file descriptor internally, so closing the GZipStream does not close the original.

Error Handling

GZip.jl throws three exception types:

• GZError — gzip-level errors (corrupt data, stream errors). Contains an error code (err) and message (err_str).
• ZError — zlib-level errors (version mismatch, memory). Same fields.
• SystemError — OS-level errors (file not found, bad file descriptor).

try
    gzopen("missing.gz") do io
        read(io, String)
    end
catch e
    if e isa SystemError
        # file not found, permission denied, etc.
    elseif e isa GZError
        # corrupt gzip data, CRC mismatch, etc.
        println("gzip error $(e.err): $(e.err_str)")
    end
end

Corrupt data is typically detected on read or close (CRC check happens at stream close).

Backends: zlib-ng and zlib

GZip.jl defaults to zlib-ng, a high-performance fork of zlib with optimized implementations for modern CPUs. The standard zlib backend is also available.

using GZip

# Default (zlib-ng)
GZip.open("data.gz", "w") do io
    write(io, "compressed with zlib-ng")
end

# Explicitly use standard zlib
GZip.open("data.gz", "w"; backend=GZip.ZLIB) do io
    write(io, "compressed with zlib")
end

Files produced by either backend are standard gzip files and can be read by any gzip implementation, including the other backend:

# Write with zlib-ng, read with zlib (or vice versa)
GZip.open("data.gz", "w") do io
    write(io, "hello")
end
data = GZip.open(read, "data.gz"; backend=GZip.ZLIB)

Benchmarks

zlib 1.3.1 vs zlib-ng 2.3.3, Julia 1.12.5, Linux x86_64 (AMD EPYC 7513).

enwik9 (1GB Wikipedia XML, compression ratio 3.09x):

Silesia corpus (55MB mixed data):

Run benchmarks yourself with julia --project=. test/benchmarks.jl (see test/README.md).

Supported IO Functions

GZipStream implements the IO interface. The following functions are supported:

seekend and truncate are not available due to limitations in zlib.

In-memory compression

GZip.jl is designed for file-based gzip I/O. For one-shot in-memory compression and decompression (like Python's gzip.compress() / gzip.decompress()), use ChunkCodecLibZlib.jl which supports setting compression level and output size hints:

using ChunkCodecLibZlib
compressed = encode(ZlibCompressCodec(), data)
decompressed = decode(ZlibDecompressCodec(), compressed)

Notes

• This interface is for gzipped files only, not the streaming zlib compression interface.
• GZipStream is a subtype of IO and can be used anywhere IO is accepted.
• readline strips trailing newlines by default (keep=false), matching Base.readline.