The quality of streaming video is crucial for user experience. For years, engineers relied on traditional, purely mathematical metrics like PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index), but these often failed to align with actual human perception of video quality. Enter VMAF (Video Multimethod Assessment Fusion), a sophisticated metric that has become the gold standard in the streaming industry.

---

What is VMAF and How is it Used?

VMAF is a full-reference video quality metric developed by Netflix in collaboration with the University of Southern California and others. It is called "full-reference" because it requires both the original, uncompressed source video (the reference) and the compressed, distorted video (the distorted sequence) for comparison.

Unlike simpler metrics, VMAF combines human vision modeling with machine learning. It fuses multiple elementary quality features, such as:

• Visual Information Fidelity (VIF): Measures the loss of information fidelity at different spatial scales.
• Detail Loss Metric (DLM): Quantifies the loss of visual details and noticeable impairments.
• Mean Co-Located Pixel Difference (MCPD): A simple measure of temporal difference between adjacent frames, accounting for motion characteristics.

By integrating these features via a machine-learning model (specifically, a Support Vector Machine-based regression), VMAF produces a single output score that is highly correlated with how a human viewer would rate the quality. Streaming platforms like Netflix use VMAF throughout their production pipeline to:

1. Compare Codecs and Encoding Settings: Determine the most efficient and highest-quality compression settings for their vast library.
2. Guide Adaptive Bitrate (ABR) Ladders: Optimize the set of resolutions and bitrates offered to ensure the best possible quality for various network conditions.
3. Perform Quality Control: Automatically monitor for quality issues introduced during ingest or transcoding.

---

Where to Get VMAF and How to Use It

VMAF is an open-source tool and is available in several forms, making it highly accessible to engineers and researchers:

• VMAF Development Kit (VDK): The official reference implementation, including a standalone C library (libvmaf) and a Python library for training and testing custom models. The source code is available on GitHub.
• Integrated into Tools: The most common way to use VMAF is through popular media tools like FFmpeg and GStreamer, which have integrated the libvmaf filter.

To use VMAF, you fundamentally need two things:

1. The Reference Video: The original, high-quality, uncompressed source.
2. The Distorted Video: The encoded or compressed version you want to measure.

The VMAF calculation is a frame-by-frame comparison between the two videos. Importantly, both videos must have the same resolution and frame rate for an accurate comparison. If they don't, you must scale or adjust one of the videos as part of the measurement process.

---

Interpreting the VMAF Report Numbers

The VMAF score is reported on a scale from 0 to 100, where:

• 0: Indicates the lowest possible quality.
• 100: Represents perfect quality, virtually indistinguishable from the reference video.

The final report usually provides an aggregate score (the arithmetic mean of the per-frame scores), but the per-frame scores themselves are incredibly useful for identifying quality dips in specific parts of the video (e.g., high-motion scenes).

Key Interpretation Points:

• Relative Scores: VMAF scores are best used for relative comparison. For example, a score of 90 on one encode is better than a score of 80 on another encode of the same reference video. Comparing the absolute scores of two different pieces of content (e.g., a cartoon vs. a live-action movie) is generally not recommended due to differences in content complexity.
• "Good Enough" Thresholds: While subjective, a score above 90 is often considered high quality and very close to the reference. Scores in the mid-70s to mid-80s are typically considered a fair/good balance between quality and bitrate efficiency. A difference of about 6 points in VMAF is sometimes cited as a Just-Noticeable Difference (JND) to the average viewer.
• Model Dependence: The default VMAF model is trained based on the assumption of a viewer watching a 1080p video from a specific distance. Newer models exist for different scenarios, such as 4K resolution or mobile phone viewing, so ensure you are using the appropriate model for your use case.

---

VMAF in FFmpeg: A Free and Powerful Tool

One of the most accessible ways to run VMAF is through the powerful, free, and open-source command-line tool FFmpeg. The VMAF library (libvmaf) is included as a video filter in modern FFmpeg builds.

FFmpeg Code Example

The most basic command to calculate VMAF requires two input files (the distorted and the reference) and uses the libvmaf filter. The distorted video must be the first input (-i distorted.mp4) for the score to be accurate.

This command will output the final VMAF score to your console:

Bash

ffmpeg -i distorted.mp4 -i reference.mp4 -filter_complex "[0:v][1:v]libvmaf" -f null -

To get a detailed, frame-by-frame log of the score and its constituent metrics, you can add the log_path and log_fmt options:

Bash

ffmpeg -i distorted.mp4 -i reference.mp4 -filter_complex "[0:v][1:v]libvmaf=log_path=vmaf_log.json:log_fmt=json" -f null -

This will generate a JSON file (vmaf_log.json) containing the aggregate score and the score for every single frame, allowing for granular analysis of video quality over time.

---

By leveraging VMAF, video engineers can move beyond guesswork and quantify the true, perceived quality of their streaming content, leading to better optimization and a superior viewing experience for all.