Writing Videos
Note: Writing of audio streams is not yet implemented
Single-step Encoding
Videos can be encoded directly from image stack using VideoIO.save(filename::String, imgstack::Array) where imgstack is an array of image arrays with identical type and size.
The entire image stack can be encoded in a single step:
import VideoIO
encoder_options = (crf=23, preset="medium")
VideoIO.save("video.mp4", imgstack, framerate=30, encoder_options=encoder_options)VideoIO.save — Functionsave(filename::String, imgstack; ...)Create a video container filename and encode the set of frames imgstack into it. imgstack must be an iterable of matrices and each frame must have the same dimensions and element type.
Encoding options, restrictions on frame size and element type, and other details are described in open_video_out. All keyword arguments are passed to open_video_out.
See also: open_video_out, write, close_video_out!
Iterative Encoding
Alternatively, videos can be encoded iteratively within custom loops.
using VideoIO
framestack = map(x->rand(UInt8, 100, 100), 1:100) #vector of 2D arrays
encoder_options = (crf=23, preset="medium")
framerate=24
open_video_out("video.mp4", framestack[1], framerate=framerate, encoder_options=encoder_options) do writer
for frame in framestack
write(writer, frame)
end
endAn example saving a series of png files as a video:
using VideoIO, ProgressMeter, FileIO
dir = "" #path to directory holding images
imgnames = filter(x->occursin(".png",x), readdir(dir)) # Populate list of all .pngs
intstrings = map(x->split(x,".")[1], imgnames) # Extract index from filenames
p = sortperm(parse.(Int, intstrings)) #sort files numerically
imgnames = imgnames[p]
encoder_options = (crf=23, preset="medium")
firstimg = load(joinpath(dir, imgnames[1]))
open_video_out("video.mp4", firstimg, framerate=24, encoder_options=encoder_options) do writer
@showprogress "Encoding video frames.." for i in eachindex(imgnames)
img = load(joinpath(dir, imgnames[i]))
write(writer, img)
end
endVideoIO.open_video_out — Functionopen_video_out(filename, ::Type{T}, sz::NTuple{2, Integer};
<keyword arguments>) -> writer
open_video_out(filename, first_img::Matrix; ...)
open_video_out(f, ...; ...)Open file filename and prepare to encode a video stream into it, returning object writer that can be used to encode frames. The size and element type of the video can either be specified by passing the first frame of the movie first_img, which will not be encoded, or instead the element type T and 2-tuple size sz. If the size is explicitly specified, the first element will be the height, and the second width, unless keyword argument scanline_major = true, in which case the order is reversed. Both height and width must be even. The element type T must be one of the supported element types, which is any key of VideoIO.VIO_DEF_ELTYPE_PIX_FMT_LU, or instead the Normed or Unsigned type for a corresponding Gray element type. The container type will be inferred from filename.
Frames are encoded with write, which must use frames with the same size, element type, and obey the same value of scanline_major. The video must be closed once all frames are encoded with close_video_out!.
If called with a function as the first argument, f, then the function will be called with the writer object writer as its only argument. This writer object will be closed once the call is complete, regardless of whether or not an error occurred.
Keyword arguments
codec_name::Union{AbstractString, Nothing} = nothing: Name of the codec to use. Must be a name accepted by FFmpeg, and compatible with the chosen container type, ornothing, in which case the codec will be automatically selected by FFmpeg based on the container.framerate::Real = 24: Framerate of the resulting video.scanline_major::Bool = false: Iffalse, then Julia arrays are assumed to have frame height in the first dimension, and frame width on the second. Iftrue, then pixels that adjacent to eachother in the same scanline (i.e. horizontal line of the video) are assumed to be adjacent to eachother in memory.scanline_major = truevideos must beStridedArrays with unit stride in the first dimension. For normal arrays, this corresponds to a matrix where frame width is in the first dimension, and frame height is in the second.container_options::OptionsT = (;): ANamedTupleorDict{Symbol, Any}of options for the container. Must correspond to option names and values accepted by FFmpeg.container_private_options::OptionsT = (;): ANamedTupleorDict{Symbol, Any}of private options for the container. Must correspond to private options names and values accepted by FFmpeg for the chosen container type.encoder_options::OptionsT = (;): ANamedTupleorDict{Symbol, Any}of options for the encoder context. Must correspond to option names and values accepted by FFmpeg.encoder_private_options::OptionsT = (;): ANamedTupleorDict{Symbol, Any}of private options for the encoder context. Must correspond to private option names and values accepted by FFmpeg for the chosen codec specified bycodec_name.swscale_options::OptionsT = (;): ANamedtuple, orDict{Symbol, Any}of options for the swscale object used to perform color space scaling. Options must correspond with options for FFmpeg's scaler filter.target_pix_fmt::Union{Nothing, Cint} = nothing: The pixel format that will be used to input data into the encoder. This can either by aVideoIO.AV_PIX_FMT_*value corresponding to a FFmpegAVPixelFormat, and must then be a format supported by the encoder, or insteadnothing, in which case it will be chosen automatically by FFmpeg.pix_fmt_loss_flags = 0: Loss flags to control how encoding pixel format is chosen. Only valid iftarget_pix_fmt = nothing. Flags must correspond to FFmpeg loss flags.input_colorspace_details = nothing: Information about the color space of input Julia arrays. Ifnothing, then this will correspond to a best-effort interpretation ofColors.jlfor the corresponding element type. To override these defaults, create aVideoIO.VioColorspaceDetailsobject using the appropriateAVCOL_definitions from FFmpeg, or useVideoIO.VioColorspaceDetails()to use the FFmpeg defaults. If data in the input Julia arrays is already in the mpeg color range, then set this toVideoIO.VioColorspaceDetails()to avoid additional scaling bysws_scale.allow_vio_gray_transform = true: Instead of usingsws_scalefor gray data, use a more accurate color space transformation implemented inVideoIOifallow_vio_gray_transform = true. Otherwise, usesws_scale.sws_color_options::OptionsT = (;): Additional keyword arguments passed to sws_setColorspaceDetails.thread_count::Union{Nothing, Int} = nothing: The number of threads the codec is allowed to use, ornothingfor default codec behavior. Defaults tonothing.
See also: write, close_video_out!
Base.write — Methodwrite(writer::VideoWriter, img)
write(writer::VideoWriter, img, index)Prepare frame img for encoding, encode it, mux it, and either cache it or write it to the file described by writer. img must be the same size and element type as the size and element type that was used to create writer. If index is provided, it must start at zero and increment monotonically.
VideoIO.close_video_out! — Functionclose_video_out!(writer::VideoWriter)Write all frames cached in writer to the video container that it describes, and then close the file. Once all frames in a video have been added to writer, then it must be closed with this function to flush any cached frames to the file, and then finally close the file and release resources associated with writer.
Supported Colortypes
Encoding of the following image element color types currently supported:
UInt8Gray{N0f8}RGB{N0f8}
Encoder Options
The encoder_options keyword argument allows control over FFmpeg encoding options. Optional fields can be found here.
More details about options specific to h264 can be found here.
Some example values for the encoder_options keyword argument are:
| Goal | encoder_options value |
|---|---|
| Perceptual compression, h264 default. Best for most cases | (crf=23, preset="medium") |
| Lossless compression. Fastest, largest file size | (crf=0, preset="ultrafast") |
| Lossless compression. Slowest, smallest file size | (crf=0, preset="veryslow") |
| Direct control of bitrate and frequency of intra frames (every 10) | (bit_rate = 400000, gop_size = 10, max_b_frames = 1) |
If a hyphenated parameter is needed, it can be added using var"param-name" = value.
Lossless Encoding
Lossless RGB
If lossless encoding of RGB{N0f8} is required, true lossless requires passing codec_name = "libx264rgb" to the function to avoid the lossy RGB->YUV420 conversion, as well as adding crf=0 in encoder_options.
Lossless Grayscale
If lossless encoding of Gray{N0f8} or UInt8 is required, crf=0 should be set, as well as color_range=2 to ensure full 8-bit pixel color representation. i.e. (color_range=2, crf=0, preset="medium")
Encoding Performance
See util/lossless_video_encoding_testing.jl for testing of losslessness, speed, and compression as a function of h264 encoding preset, for 3 example videos.