In playing with values for ffmpeg to try to get a similar output to my “standard” settings in HandBrake, this is what I came up with:
ffmpeg -i "$1" -c:v hevc_nvenc -preset slow -profile:v main10 -rc-lookahead:v 32 -spatial_aq 1 -cq 29 -tag:v hvc1 -c:a libfdk_aac -profile:a aac_he -b:a 80k "$( basename "$1" .avi ).mp4"
This works about perfectly for standard AVIs you find out on the net. It uses the “slow” profile, which says the equivalent of a 2 pass encode. I’m not sure what that means exactly, but it’s still very fast from a FPS encoding standpoint. The rc-lookahead uses the maximum number of frames (32) to look ahead that the nVidia NVENC encoder supports to to better bitrate estimations. On my GPU, termporal_aq does not work (would have been better for animation), so I turned on spatial_aq to have it look at the frame for areas that can be more or less efficiently encoded based on how “busy” the area is; i.e flat surfaces or smooth gradients vs lots of non-related shapes. To get approximately the same bitrate as the 23 CRF setting in Handbrake’s H.265 CPU encoder, I have found that the CQ setting should be 29; when watching the encode, the “q” setting seems to sit closer to 23 than 29, which is the desired output.
The tag option forces it to use hvc1 instead of hev1, because Apple only supports the hvc1 tag.
The audio is using the HE-AAC encoder. You will need to adjust the bitrate according to the number of channels in the source. For stereo, 80kbits is acceptable; you might want to up it depending on the source, but for the usual XVID/MP3 AVIs, 80k is still better than 128k MP3, so giving it more isn’t going to make much perceptible difference. YMMV.
This is giving me 900-1300 FPS with an nVidia 1080 on a laptop, so at least a full order of magnitude faster than the CPU bound encoder.
To get all the options for the nvenc hevc encoder:
ffmpeg -h encoder=nevc_nvenc