Writing Videos

Note: Writing of audio streams is not yet implemented

Single-step Encoding

Videos can be encoded directly from image stack using VideoIO.save(filename::String, imgstack::Array) where imgstack is an array of image arrays with identical type and size.

The entire image stack can be encoded in a single step:

import VideoIO
encoder_options = (crf=23, preset="medium")
VideoIO.save("video.mp4", imgstack, framerate=30, encoder_options=encoder_options)
VideoIO.saveFunction
save(filename::String, imgstack; ...)

Create a video container filename and encode the set of frames imgstack into it. imgstack must be an iterable of matrices and each frame must have the same dimensions and element type.

Encoding options, restrictions on frame size and element type, and other details are described in open_video_out. All keyword arguments are passed to open_video_out.

See also: open_video_out, write, close_video_out!

source

Iterative Encoding

Alternatively, videos can be encoded iteratively within custom loops.

using VideoIO
framestack = map(x->rand(UInt8, 100, 100), 1:100) #vector of 2D arrays

encoder_options = (crf=23, preset="medium")
framerate=24
open_video_out("video.mp4", framestack[1], framerate=framerate, encoder_options=encoder_options) do writer
    for frame in framestack
        write(writer, frame)
    end
end

An example saving a series of png files as a video:

using VideoIO, ProgressMeter

dir = "" #path to directory holding images
imgnames = filter(x->occursin(".png",x), readdir(dir)) # Populate list of all .pngs
intstrings =  map(x->split(x,".")[1], imgnames) # Extract index from filenames
p = sortperm(parse.(Int, intstrings)) #sort files numerically
imgnames = imgnames[p]

encoder_options = (crf=23, preset="medium")

firstimg = load(joinpath(dir, imgnames[1]))
open_video_out("video.mp4", firstimg, framerate=24, encoder_options=encoder_options) do writer
    @showprogress "Encoding video frames.." for i in eachindex(imgnames)
        img = load(joinpath(dir, imgnames[i]))
        write(writer, img)
    end
end
VideoIO.open_video_outFunction
open_video_out(filename, ::Type{T}, sz::NTuple{2, Integer};
               <keyword arguments>) -> writer
open_video_out(filename, first_img::Matrix; ...)
open_video_out(f, ...; ...)

Open file filename and prepare to encode a video stream into it, returning object writer that can be used to encode frames. The size and element type of the video can either be specified by passing the first frame of the movie first_img, which will not be encoded, or instead the element type T and 2-tuple size sz. If the size is explicitly specified, the first element will be the height, and the second width, unless keyword argument scanline_major = true, in which case the order is reversed. Both height and width must be even. The element type T must be one of the supported element types, which is any key of VideoIO.VIO_DEF_ELTYPE_PIX_FMT_LU, or instead the Normed or Unsigned type for a corresponding Gray element type. The container type will be inferred from filename.

Frames are encoded with write, which must use frames with the same size, element type, and obey the same value of scanline_major. The video must be closed once all frames are encoded with close_video_out!.

If called with a function as the first argument, f, then the function will be called with the writer object writer as its only argument. This writer object will be closed once the call is complete, regardless of whether or not an error occurred.

Keyword arguments

  • codec_name::Union{AbstractString, Nothing} = nothing: Name of the codec to use. Must be a name accepted by FFmpeg, and compatible with the chosen container type, or nothing, in which case the codec will be automatically selected by FFmpeg based on the container.
  • framerate::Real = 24: Framerate of the resulting video.
  • scanline_major::Bool = false: If false, then Julia arrays are assumed to have frame height in the first dimension, and frame width on the second. If true, then pixels that adjacent to eachother in the same scanline (i.e. horizontal line of the video) are assumed to be adjacent to eachother in memory. scanline_major = true videos must be StridedArrays with unit stride in the first dimension. For normal arrays, this corresponds to a matrix where frame width is in the first dimension, and frame height is in the second.
  • container_options::OptionsT = (;): A NamedTuple or Dict{Symbol, Any} of options for the container. Must correspond to option names and values accepted by FFmpeg.
  • container_private_options::OptionsT = (;): A NamedTuple or Dict{Symbol, Any} of private options for the container. Must correspond to private options names and values accepted by FFmpeg for the chosen container type.
  • encoder_options::OptionsT = (;): A NamedTuple or Dict{Symbol, Any} of options for the encoder context. Must correspond to option names and values accepted by FFmpeg.
  • encoder_private_options::OptionsT = (;): A NamedTuple or Dict{Symbol, Any} of private options for the encoder context. Must correspond to private option names and values accepted by FFmpeg for the chosen codec specified by codec_name.
  • swscale_options::OptionsT = (;): A Namedtuple, or Dict{Symbol, Any} of options for the swscale object used to perform color space scaling. Options must correspond with options for FFmpeg's scaler filter.
  • target_pix_fmt::Union{Nothing, Cint} = nothing: The pixel format that will be used to input data into the encoder. This can either by a VideoIO.AV_PIX_FMT_* value corresponding to a FFmpeg AVPixelFormat, and must then be a format supported by the encoder, or instead nothing, in which case it will be chosen automatically by FFmpeg.
  • pix_fmt_loss_flags = 0: Loss flags to control how encoding pixel format is chosen. Only valid if target_pix_fmt = nothing. Flags must correspond to FFmpeg loss flags.
  • input_colorspace_details = nothing: Information about the color space of input Julia arrays. If nothing, then this will correspond to a best-effort interpretation of Colors.jl for the corresponding element type. To override these defaults, create a VideoIO.VioColorspaceDetails object using the appropriate AVCOL_ definitions from FFmpeg, or use VideoIO.VioColorspaceDetails() to use the FFmpeg defaults. If data in the input Julia arrays is already in the mpeg color range, then set this to VideoIO.VioColorspaceDetails() to avoid additional scaling by sws_scale.
  • allow_vio_gray_transform = true: Instead of using sws_scale for gray data, use a more accurate color space transformation implemented in VideoIO if allow_vio_gray_transform = true. Otherwise, use sws_scale.
  • sws_color_options::OptionsT = (;): Additional keyword arguments passed to sws_setColorspaceDetails.
  • thread_count::Union{Nothing, Int} = nothing: The number of threads the codec is allowed to use, or nothing for default codec behavior. Defaults to nothing.

See also: write, close_video_out!

source
Base.writeMethod
write(writer::VideoWriter, img)
write(writer::VideoWriter, img, index)

Prepare frame img for encoding, encode it, mux it, and either cache it or write it to the file described by writer. img must be the same size and element type as the size and element type that was used to create writer. If index is provided, it must start at zero and increment monotonically.

source
VideoIO.close_video_out!Function
close_video_out!(writer::VideoWriter)

Write all frames cached in writer to the video container that it describes, and then close the file. Once all frames in a video have been added to writer, then it must be closed with this function to flush any cached frames to the file, and then finally close the file and release resources associated with writer.

source

Supported Colortypes

Encoding of the following image element color types currently supported:

  • UInt8
  • Gray{N0f8}
  • RGB{N0f8}

Encoder Options

The encoder_options keyword argument allows control over FFmpeg encoding options. Optional fields can be found here.

More details about options specific to h264 can be found here.

Some example values for the encoder_options keyword argument are:

Goalencoder_options value
Perceptual compression, h264 default. Best for most cases(crf=23, preset="medium")
Lossless compression. Fastest, largest file size(crf=0, preset="ultrafast")
Lossless compression. Slowest, smallest file size(crf=0, preset="veryslow")
Direct control of bitrate and frequency of intra frames (every 10)(bit_rate = 400000, gop_size = 10, max_b_frames = 1)

If a hyphenated parameter is needed, it can be added using var"param-name" = value.

Lossless Encoding

Lossless RGB

If lossless encoding of RGB{N0f8} is required, true lossless requires passing codec_name = "libx264rgb" to the function to avoid the lossy RGB->YUV420 conversion, as well as adding crf=0 in encoder_options.

Lossless Grayscale

If lossless encoding of Gray{N0f8} or UInt8 is required, crf=0 should be set, as well as color_range=2 to ensure full 8-bit pixel color representation. i.e. (color_range=2, crf=0, preset="medium")

Encoding Performance

See util/lossless_video_encoding_testing.jl for testing of losslessness, speed, and compression as a function of h264 encoding preset, for 3 example videos.