Skip to content

Choose a suitable codec

When modifying videos(1), you are re-encoding your video, which often gets you greeted with an overwhelming amount of formats, codecs and a plethora configuration options¹².

  1. This means using (up)scale scripts, Smoothie, or exporting a NLE project to a video file, this does not concern LosslessCut.

Think about encoded video data like a tightly-packed suitcase , you need to unpack it into raw data in order to change anything(1) in it, and then re-encode this new data into a codec again.

  1. Not necessarily for particular scenarios: tools like LosslessCut can make cuts every I-frame which is where the compression resets and does not need to re-encode the whole video.

The choice depends on your hardware (do you own an NVIDIA GPU?), what you're uploading to (is it compatible with YouTube, Twitter, etc?), how much quality you're ready to lose (lossless means no compression artifacts) and how long you want to wait (can you let your PC render overnight?).

Each codec has it's own pros and cons:

Features H.264 / AVC H.265 / HEVC AV1 UTVideo
Fast at encoding1 Only fast when encoding with a compatible GPU
Filesize/Visual fidelity ratio2 Worst out the bunch Lossless,
makes huge files
Compatibility3 Works in some video players AFAIK Only recent Davinci Resolve versions support importing it You may need to install it
Decoding speed4 Worst out the bunch It's only got full fat I-frames, resulting in super fast decoding

Hardware-accelerated encoding

You might've also noticed NVENC, AMF and QuickSync being mentioned after H.264, H.265 and AV1, this means the encoding is done on your GPU / iGPU instead of your CPU, this comes with pros/cons:
Faster encoding

Much lower load on the CPU (especially useful when recording while you are playing a game)

Worse filesize / visual fidelity ratio than CPU counterpart

Note that AV1 Encoding is only available on last-gen cards.

Should I get an RTX 4000 card for it's AV1 encoder?

Only if you specifically have a use in mind for archiving or workflows that works best in AV1

Definitely a step up in visual fidelity for band-with limited scenarios (streaming)

Most video editors don't support importing AV1 footage yet (Voukoder does support exporting with it with software and hardware encoding)

No, you won't have better "quality" when recording videos, H264 CQP0 exists if you want lossless quality

Rate controls

Unless you are recording lossless (HUGE files), you need a way to constrain the video's bandwith

CBR: Constant bit rate

Most used for streaming nowadays

CRF / CQP: Constrained rate factor / Constant quantizer

The new prefered rate control for recording to disk, as it adapts to the "bandwith-hungryness" of the content, e.g. standing still in front a wall in a video-game will write way less data than when you're spinning and doing a bunch of 360s.

It's some complex math stuff

Couleur's routine & use of codecs

Recording w/ OBS: H.264 NVENC to encode as fast(1) as possible

  1. Even super fast encoding settings can be lossless, NVENC's P1 Preset which OBS calls "Low quality" is directed at people streaming with a constrained CBR

Pre-rendering w/ Smoothie: UTVideo Decodes as fast as possible for video editing

Exporting w/ Voukoder: H265 NVENC + Upscaling(1) Whatever video editor which supports Voukoder this is a great middleground

  1. This mentions one of my Voukoder presets that can be installed with my installation script that has an additional filter, which (up)scales to 4K and encode on the fly without needing to encode again with a batch script

Voukoder & Smoothie encoding presets

These directly translate to FFmpeg arguments, also add the 4K macro to add upscaling arguments

Tries to stay faithful to the aforementioned settings, also includes + Upscale editions

Other resources to learn more

https://trac.ffmpeg.org/wiki/Encode/H.264

  1. General rule of thumb of how long you're expected to wait while it's encoding. Note that with a touch of configuration the same codec can be lightning speed or excruciatingly slow. ↩

  2. Most important metric when streaming since you have a very limited bandwith budget unlike recording on disk/SSD. ↩

  3. Mostly concerns being able to import videos in your video editor, uploading to Twitter, and being able to embed on Discord (aka to play back directly without the need to download). YouTube supports pretty much everything since it re-encodes after. ↩

  4. Important to consider while video editing so videos play back fast in your preview. ↩