(Guide written using Voukoder 2.3 and Premiere Pro 2019)
If anybody here is new to HDR encoding in premiere, it can be particularly tricky to get it right. Basically, Premiere is not designed for doing HDR editing properly. It can recognize HDR input files, and even has HDR color correction, but it has no way to visibly monitor the HDR output, and the only way to actually export using this process is through Adobe's own HEVC codec. If you want to display your video on an HDR monitor and/or export using x265, Premiere's default settings will prevent you from doing that correctly.
If you feed Adobe Premiere an HDR file, it will read it as such and display it as SDR clipped at 100nits. Meaning all HDR highlight information is lost, and the timeline will not be in the correct picture format for exporting to HDR using x265. x265 expects to receive the raw video signal, unprocessed. HDR uses a different gamma function than SDR, called ST2084, and this results in a very washed out picture. Here is a sample of what a raw frame of HDR would look like in Premiere, if premiere had not performed a conversion for displaying it on SDR monitors:
and here is how that same image appears when Premiere decides to apply its HDR-to-SDR conversion:
Obviously overexposed, losing detail in the brightest parts and even some color volume. Here is an example of how that image may look with the exposure lowered to display all detail you would see on an HDR display (on the HDR display, it would be as bright as the above image, but with all highlight information retained)
Obviously, there was a lot lost when Premiere did its conversion thing, but we can get it back. All of that detail is still there, it's just being clipped. So if I adjust the curves in Lumetri to map 0 nits to "black" and 10,000 nits to "white", this allows for the full HDR range to be represented in the black-to-white Premiere output range, however, it still is using the rec709 transfer function and colors. This needs to be fixed. The way I do that is apply a custom 3D LUT which contains all of the mathematical conversions to revert the transfer function to ST2084, and the colors to BT2020. This will result in the image in the program monitor to look the same as the first image above. This means we can now export to x265! This also means you can display the program monitor on an HDR capable display, in order to monitor the output in real time correctly.
In order to do this, I've created a preset for Premiere that layers two Lumetri effects on top of each other and performs all of the necessary computations to revert everything to the correct color format for x265:
Download this, and import the preset into premiere, and then apply it to any video you intent to export into HDR. If you've performed multiple edits, you can either create an adjustment layer and apply the preset to that, or create a new sequence where you import your edit sequence, and then just apply the preset to that track. Alternatively, if you can trick Premiere into thinking your footage is rec709, you don't need this step. My Atomos Ninja Inferno allows me to record HDR flagged as rec709, so I use that instead.
In order to output HDR10 correctly, you will need metadata. I don't recommend simply just using x265's default settings for those as I know many people do. For one, it uses DCI-P3 primaries, so color will be wrong, and for another, it doesn't contain the correct brightness information, so TVs will not be able to tonemap this correctly to your display's capabilities.
Fortunately, I've created a solution for this:
HDR Metadata Tool.zip
The way this app works is you feed it a folder full of image files containing the same raw unprocessed video image we created above.
Even though you're dealing with HDR video, I could only get my program to work with 8bit PNG files, so make sure they aren't 16bit. The end result is only an approximation, but it's close.
EDIT: The app works now with 16bit image sources, and not just PNG either. If exporting from premiere, I recommend 16bit TIFF files (no alpha is needed, but the app will ignore alpha if it is there) TIFF, PNG, JPG, and BMP are all supported at 8/16 bit with or without Alpha. Warning: If you use 16bit TIFFs, you may need to store them on an SSD or the app may lock up at times.
What I recommend to do is watch through your entire video and export a screenshot for every unique shot in the video. Try to avoid choosing frames with quick, bright flashes, as while they may contain brighter pixels, the metadata is intending to describe your entire movie for the purpose of tonemapping, so you don't want the metadata to simply describe just those brighter frames. This I think is a major flaw in how hollywood calculates its metadata (looking at every single frame).
What you want are the MaxCLL and MaxFALL numbers. MaxCLL represents the brightest pixel the app found, and MaxFALL represents the brightest frame average luminance (APL).
Now you're ready to export to Voukoder. In the main window, Voukoder has a color space selection, leave this at the default Rec709 (HD) value. Even though this is incorrect, this is the format Premiere exports its timeline in, and modifying that will result in incorrect pixel colors, screwing up the whole process. Range should also be Limited, as this is the correct legal range for HDR10 output.
Next, open the Voukoder configuration. Make sure the HEVC (x265) encoder is selected, and press configure. Color format needs to be 4:2:0 (10bit). For encoding quality, I recommend CRF13. That may seem low to you, but I've found due to the differing picture format, the CRF values are not equivalent to what you'd use in SDR. I use CRF18 in SDR, and found that CRF13 gave equivalent level of detail in SDR. So take whatever you might normally use, and reduce CRF by 5 for optimal results. Reducing by 4 also gave pretty close results visually, but gave me a file size smaller than the SDR, so I went with -5 for something that was using more bitrate than the SDR.
Next, scroll down to the VUI section:
1. First, make sure Range is Limited again. This may not be totally necessary as we set it before, but just in case it's a good idea.
2. Color Primaries need to be BT2020, as that's what my Premiere Preset outputs the format as
3. Similarly, Transfer Characteristics should be SMPTE 2084, to match the preset output
4. Color Matrix should be BT2020nc. BT2020c may work as well, but I'm not 100% confident in that.
5. Master Display. I recommend copying these settings first:
Don't use the default setting. That defaults to DCI-P3 primaries. What I've typed above are BT2020 primaries and a peak luminance of 1000 nits. This value is representative of the HDR display you used to edit with. If you didn't use an HDR display, or didn't make any color corrections even if you did use one, then what I would recommend is to set the L value to a range at or slightly above your "MaxCLL" value. For example, my MaxCLL value for my project was 1221 nits, so I decided to round up to 1500 nits for the mastering display, giving me L(15000000,0). The 0 part of that is the minimum luminance of the display. I typically leave this at zero, but again if you know the specs of your exact mastering display, you can include them here. Again, doing that is only really useful if you've performed color correction while viewing on an HDR display. If you choose to color correct, I would recommend placing a Lumetri filter before my preset, and checking the "HDR" box. You could also instead simply modify the second Lumetri filter included with my Preset (but DON'T click the HDR checkbox as that will remove the LUT). Doing it this way will treat shadow and highlight differently, but perhaps that may be more useful in some situations as well. Leave the LUT in place but you can use the other controls.
6. Maximum Content Light Level. This is where you place the MaxCLL and MaxFALL values you obtained before. MaxCLL, followed by a comma, followed by MaxFALL.
7. HDR options. Definitely make sure this is on, as it affects other encoder settings that tune for HDR
8. UHD Blu-ray conform. This isn't 100% necessary, but I recommend it. It enforces other requirements that UHD Blu-ray has (not counting resolution and FPS) but this may improve compatibility with some devices so it's a good idea even if you don't intend to burn your video to UHD Blu-ray.
Then just make sure your container and audio settings are to your liking (which settings you choose here may affect compatibility with your devices, or they may not)
Starting with Voukoder 2.3 RC2, you can now use NVENC's HEVC codec to encode in HDR as well following this guide. Make sure your HEVC output is in 10bit mode, and then go over to the filters tab and add "setparams" to the following:
The biggest issue is you won't be able to inject the stream with proper HDR10 metadata. So if you want a fully HDR10 compliant output, make sure you use x265, but for basic HDR support in HEVC through NVENC, this is now possible. Thank you, Daniel!
The above NVENC process works correctly for x264, VP9, and ProRes as well