I'm a bit hesitant to call this a bug, maybe it is though.
With Identical settings, NVENC HEVC only uses about 15% of my GPU's video encode when 10 bit is selected, NVENC HEVC uses 90% of my GPU's video encode when 8 bit is selected. In 8 bit I get a real time export, while 10 bit takes 4-5 times longer for what seems to be no reason. Since I kind of need 10 bit for my export, this is a pretty big setback for the longer videos I render, since 30 minutes becomes 2 and a half hours.
Edit: I'm aware this could be Premiere being slow with handing frames to Voukoder.