Writing Videos
Note: Writing of audio streams is not yet implemented
Single-step Encoding
Videos can be encoded directly from image stack using VideoIO.save(filename::String, imgstack::Array)
where imgstack
is an array of image arrays with identical type and size.
The entire image stack can be encoded in a single step:
import VideoIO
encoder_options = (crf=23, preset="medium")
VideoIO.save("video.mp4", imgstack, framerate=30, encoder_options=encoder_options)
VideoIO.save
— Functionsave(filename::String, imgstack; ...)
Create a video container filename
and encode the set of frames imgstack
into it. imgstack
must be an iterable of matrices and each frame must have the same dimensions and element type.
Encoding options, restrictions on frame size and element type, and other details are described in open_video_out
. All keyword arguments are passed to open_video_out
.
See also: open_video_out
, write
, close_video_out!
Iterative Encoding
Alternatively, videos can be encoded iteratively within custom loops.
using VideoIO
framestack = map(x->rand(UInt8, 100, 100), 1:100) #vector of 2D arrays
encoder_options = (crf=23, preset="medium")
framerate=24
open_video_out("video.mp4", framestack[1], framerate=framerate, encoder_options=encoder_options) do writer
for frame in framestack
write(writer, frame)
end
end
An example saving a series of png files as a video:
using VideoIO, ProgressMeter, FileIO
dir = "" #path to directory holding images
imgnames = filter(x->occursin(".png",x), readdir(dir)) # Populate list of all .pngs
intstrings = map(x->split(x,".")[1], imgnames) # Extract index from filenames
p = sortperm(parse.(Int, intstrings)) #sort files numerically
imgnames = imgnames[p]
encoder_options = (crf=23, preset="medium")
firstimg = load(joinpath(dir, imgnames[1]))
open_video_out("video.mp4", firstimg, framerate=24, encoder_options=encoder_options) do writer
@showprogress "Encoding video frames.." for i in eachindex(imgnames)
img = load(joinpath(dir, imgnames[i]))
write(writer, img)
end
end
VideoIO.open_video_out
— Functionopen_video_out(filename, ::Type{T}, sz::NTuple{2, Integer};
<keyword arguments>) -> writer
open_video_out(filename, first_img::Matrix; ...)
open_video_out(f, ...; ...)
Open file filename
and prepare to encode a video stream into it, returning object writer
that can be used to encode frames. The size and element type of the video can either be specified by passing the first frame of the movie first_img
, which will not be encoded, or instead the element type T
and 2-tuple size sz
. If the size is explicitly specified, the first element will be the height, and the second width, unless keyword argument scanline_major = true
, in which case the order is reversed. Both height and width must be even. The element type T
must be one of the supported element types, which is any key of VideoIO.VIO_DEF_ELTYPE_PIX_FMT_LU
, or instead the Normed
or Unsigned
type for a corresponding Gray
element type. The container type will be inferred from filename
.
Frames are encoded with write
, which must use frames with the same size, element type, and obey the same value of scanline_major
. The video must be closed once all frames are encoded with close_video_out!
.
If called with a function as the first argument, f
, then the function will be called with the writer object writer
as its only argument. This writer
object will be closed once the call is complete, regardless of whether or not an error occurred.
Keyword arguments
codec_name::Union{AbstractString, Nothing} = nothing
: Name of the codec to use. Must be a name accepted by FFmpeg, and compatible with the chosen container type, ornothing
, in which case the codec will be automatically selected by FFmpeg based on the container.framerate::Real = 24
: Framerate of the resulting video.scanline_major::Bool = false
: Iffalse
, then Julia arrays are assumed to have frame height in the first dimension, and frame width on the second. Iftrue
, then pixels that adjacent to eachother in the same scanline (i.e. horizontal line of the video) are assumed to be adjacent to eachother in memory.scanline_major = true
videos must beStridedArray
s with unit stride in the first dimension. For normal arrays, this corresponds to a matrix where frame width is in the first dimension, and frame height is in the second.container_options::OptionsT = (;)
: ANamedTuple
orDict{Symbol, Any}
of options for the container. Must correspond to option names and values accepted by FFmpeg.container_private_options::OptionsT = (;)
: ANamedTuple
orDict{Symbol, Any}
of private options for the container. Must correspond to private options names and values accepted by FFmpeg for the chosen container type.encoder_options::OptionsT = (;)
: ANamedTuple
orDict{Symbol, Any}
of options for the encoder context. Must correspond to option names and values accepted by FFmpeg.encoder_private_options::OptionsT = (;)
: ANamedTuple
orDict{Symbol, Any}
of private options for the encoder context. Must correspond to private option names and values accepted by FFmpeg for the chosen codec specified bycodec_name
.swscale_options::OptionsT = (;)
: ANamedtuple
, orDict{Symbol, Any}
of options for the swscale object used to perform color space scaling. Options must correspond with options for FFmpeg's scaler filter.target_pix_fmt::Union{Nothing, Cint} = nothing
: The pixel format that will be used to input data into the encoder. This can either by aVideoIO.AV_PIX_FMT_*
value corresponding to a FFmpegAVPixelFormat
, and must then be a format supported by the encoder, or insteadnothing
, in which case it will be chosen automatically by FFmpeg.pix_fmt_loss_flags = 0
: Loss flags to control how encoding pixel format is chosen. Only valid iftarget_pix_fmt = nothing
. Flags must correspond to FFmpeg loss flags.input_colorspace_details = nothing
: Information about the color space of input Julia arrays. Ifnothing
, then this will correspond to a best-effort interpretation ofColors.jl
for the corresponding element type. To override these defaults, create aVideoIO.VioColorspaceDetails
object using the appropriateAVCOL_
definitions from FFmpeg, or useVideoIO.VioColorspaceDetails()
to use the FFmpeg defaults. If data in the input Julia arrays is already in the mpeg color range, then set this toVideoIO.VioColorspaceDetails()
to avoid additional scaling bysws_scale
.allow_vio_gray_transform = true
: Instead of usingsws_scale
for gray data, use a more accurate color space transformation implemented inVideoIO
ifallow_vio_gray_transform = true
. Otherwise, usesws_scale
.sws_color_options::OptionsT = (;)
: Additional keyword arguments passed to sws_setColorspaceDetails.thread_count::Union{Nothing, Int} = nothing
: The number of threads the codec is allowed to use, ornothing
for default codec behavior. Defaults tonothing
.
See also: write
, close_video_out!
Base.write
— Methodwrite(writer::VideoWriter, img)
write(writer::VideoWriter, img, index)
Prepare frame img
for encoding, encode it, mux it, and either cache it or write it to the file described by writer
. img
must be the same size and element type as the size and element type that was used to create writer
. If index
is provided, it must start at zero and increment monotonically.
VideoIO.close_video_out!
— Functionclose_video_out!(writer::VideoWriter)
Write all frames cached in writer
to the video container that it describes, and then close the file. Once all frames in a video have been added to writer
, then it must be closed with this function to flush any cached frames to the file, and then finally close the file and release resources associated with writer
.
Supported Colortypes
Encoding of the following image element color types currently supported:
UInt8
Gray{N0f8}
RGB{N0f8}
Encoder Options
The encoder_options
keyword argument allows control over FFmpeg encoding options. Optional fields can be found here.
More details about options specific to h264 can be found here.
Some example values for the encoder_options
keyword argument are:
Goal | encoder_options value |
---|---|
Perceptual compression, h264 default. Best for most cases | (crf=23, preset="medium") |
Lossless compression. Fastest, largest file size | (crf=0, preset="ultrafast") |
Lossless compression. Slowest, smallest file size | (crf=0, preset="veryslow") |
Direct control of bitrate and frequency of intra frames (every 10) | (bit_rate = 400000, gop_size = 10, max_b_frames = 1) |
If a hyphenated parameter is needed, it can be added using var"param-name" = value
.
Lossless Encoding
Lossless RGB
If lossless encoding of RGB{N0f8}
is required, true lossless requires passing codec_name = "libx264rgb"
to the function to avoid the lossy RGB->YUV420 conversion, as well as adding crf=0
in encoder_options
.
Lossless Grayscale
If lossless encoding of Gray{N0f8}
or UInt8
is required, crf=0
should be set, as well as color_range=2
to ensure full 8-bit pixel color representation. i.e. (color_range=2, crf=0, preset="medium")
Encoding Performance
See util/lossless_video_encoding_testing.jl
for testing of losslessness, speed, and compression as a function of h264 encoding preset, for 3 example videos.