Beiträge von Joe24

  • Multi-GPU rendering

    Well now I feel a bit silly. It wasn't a VoukoderPro problem at all.

    Removed all video nodes from a test scene, and still observed significant GPU0 copy and CUDA activity on an audio-only VoPro encode. Hmmm.

    I always have GPU acceleration turned off in Vegas because it slows renders down . . . but must have enabled it for some test, and forgot to disable it again. *facepalm*

    Once I turned GPU acceleration off in Vegas, the copy and CUDA activity is no longer present on GPU0. There is a light 3% load on GPU0 3D/CUDA engine when Vegas is onscreen, but this drops to 0% when Vegas is minimized, and seems to be just normal Windows usage of the primary graphics card.

    Tested renders on all 3 cards individually, and all GPU activity takes place on the correct intended card.

    So there is no actual problem.


    On another note . . . To my way of thinking, it would make more sense to have each node 'inherit' the GPU assignation of the previous (upstream) node in Scene Designer, rather than having to set a target GPU for every single node. If a user wanted to transfer the stream from one GPU to another at any point, couldn't they manually use Download/Upload nodes to send data from one card to another?

  • Multi-GPU rendering

    When assigning a filter/encode operation to FFmpeg GPU1, the encode uses NVENC of GPU1, but the CUDA + copy engine of GPU0. There is bus activity on both GPUs, so this is not an OS reporting anomaly. Data is being shuffled from one GPU to the other mid-task.

    When assigning filter/encode operations to both GPUs: both CUDA + copy engines of FFmpeg GPU1 remain idle.

    Testing on a 2-GPU system, VoPro version 0.7.2.8, Vegas 20 build 411. GPU was assigned by setting the target GPU in the CUDA Upload filter.

    NOTE: FFmpeg numbers the GPUs differently than Windows does. In my system:

    • Windows GPU0 = FFmpeg "GPU1"
    • Windows GPU1 = FFmpeg "GPU0"

    You can find the FFmpeg designation for each card by the following command:

    Code
    ./ffmpeg -f lavfi -i nullsrc -c:v h264_nvenc -gpu list -f null -

    Scene files to test:

    scenes.zip

    PS, to run both these Scenes simultaneously, you'll probably have to patch your drivers (because there are more than 3 total output streams):

    nvidia-patch/win at master · keylase/nvidia-patch
    This patch removes restriction on maximum number of simultaneous NVENC video encoding sessions imposed by Nvidia to consumer-grade GPUs. - keylase/nvidia-patch
    github.com

  • Dynamic file naming

    Output-file naming is a problem in VoukoderPro. To date, including the current version (0.7.2.8), file naming choices are: either completely custom, or completely set by the NLE.

    For one thing, if you manually set the file name in your VoPro Scene, this makes it super easy (barely an inconvenience) to accidentally overwrite a previous render, because all output files encoded with that Scene will have the same name.

    If you're using VoPro to encode multiple formats from the same NLE render, you cannot use the NLE-supplied name because then all your output files are simultaneously attempting to have the same name. Chaos ensues (although VoPro now has a lockout in this situation and will refuse to encode at all).

    Further, if you are using the Render Regions function in Vegas, each rendered region file will overwrite the previous one. If you rendered 3 regions, you'd be left with only a single video file which would contain the last region rendered. The first 2 regions will be lost, overwritten.

    Would be nice to see some rule-based dynamic naming functions. For instance:

    • Perhaps a function for simple sequential naming. I.E., if "00000.mp4" exists, name output "00001.mp4". Or an unconditional master counter, like the log file naming system.
    • Suffix-based naming with wildcards, based on the filename supplied by the NLE. I.E., NLE filename: "Lost in the Woods.mp4", VoPro options: "%NLEfilename - %VoProScene - %seq.mp4", VoPro output filenames would be something like: "Lost in the Woods - Monochrome 8K - 00000.mp4" and "Lost in the Woods - 192 kbps ABR - 00000.mp3".
    • Date- and time-based names.

  • Memory leak?

    It does not occur during Scene Test.

    Initial GPU memory used: 0.2 GB

    Memory during test: 0.5 GB

    Memory after test: 0.2 GB

    Memory after 30 consecutive tests: 0.2 GB


    Note regarding Vegas: No matter how much GPU memory is used by Vegas/VoPro, all of it is released on exit from Vegas.

  • Memory leak?

    Using "doesn't work.scene" from above (post #11). Single encode, single card, single instance of Vegas 20, stock drivers, VoPro 0.7.2.9.

    7 runs using normal graph speed in Task Manager:


    30 runs using slow graph speed in Task Manager:

  • Memory leak?

    Yes, I have been using YUV 420 8-bit templates exclusively for this.

    I tried installing a fresh unpatched driver (537.13) and this made no difference. The patch only relates to how many things you can do at once on a non-Quadro card, so shouldn't have anything to do with anything.

    I tried enabling GPU globally in Vegas (which is the default), and this made no difference either.

    Every time I start a render, the process allocates ~0.6 GB of VRAM. After the render, ~0.2 GB is released, and the remaining ~0.4 GB stays occupied.

    I'll run through the different CUDA filters again when I get time . . . but as I said, that was probably operator error. I was just quickly looking for something CUDA-based that wasn't an encoder. Never checked the output at all, just some of them started to encode and others wouldn't. Chroma Key was one that wouldn't work at all.

  • Memory leak?

    Are you testing using the Scene from post #5, above?

    I'm still testing, trying to pin down exactly what is causing the issue.

    What I'm seeing so far is that when the following chain is used, there is no memory leak: Video input -> NVENC encoder (either h.264 or HEVC).

    works.scene.zip


    However, when the following chain is used, the memory leak exists: Video input -> CUDA upload -> NVENC encoder (either h.264 or HEVC).

    doesn't work.scene.zip


    No apparent memory leaks if I go Upload -> [various CUDA filters] -> Download -> [software encoder]. Some of the filters just plain didn't work (FFmpeg error), but that could be operator error.

    The only other possibly-related unusual behavior I've noticed so far is with the CUDA Thumbnail filter. Maybe not a leak, but something is getting left behind in GPU memory, then cleared at the start of each render:

    Load Vegas, 1.3 GB GPU memory.

    During 1st render with Thumbnail filter: 1.8 GB

    After 1st render: 1.8 GB

    Immediately after starting 2nd render: briefly drops back to 1.3 GB

    During 2nd render: 1.8 GB

    After 2nd render: 1.8 GB

    thumbnail filter.scene.zip

  • Memory leak?

    Zitat von Vouk

    I was not aware people run parallel instance of it

    In a multi-socket server/workstation system, it's normal to have multiple jobs running, each confined to its own NUMA node. Likewise if you're running multiple hardware encoders, each with different render projects. This is part of why VoukoderPro is such a big deal.

    To be clear, the memory leak issue is occurring even with a single render in a single instance of Vegas. Simply starting and stopping the same render over and over will do this, as in Post #5 above. Leak still present in version 0.7.2.9.

  • Memory leak?

    No change in version 0.7.2.8. Tried just a single render, and the problem exists there too.

    I tried to see if there was an upper limit to the memory usage. Eventually VoukoderPro does refuse to run, displays error "Unable to start VoukoderPro: Undefined error!" Screenshot below is taken at this point, after starting and stopping the same render many times. As you can see, the total video memory usage has grown to over 35 GB.


    The last few lines of the log file at the time of the crash are:


    And here's an example scene that causes the issue:

    temp v0.1.scene.zip

  • Memory leak?

    I'm afraid it isn't fixed in 0.7.2.7. Maybe improved, but still an issue.

    Sequence of actions starting and stopping the same render job, with VRAM usage (8 GB card):

    • Windows idle: 0.4 GB
    • Loaded 2 instances of Vegas 20: 1.8 GB
    • During 1st render: 3.6 GB
    • After 1st render: 3.0 GB
    • During 2nd render: 4.8 GB
    • After 2nd render: 4.1 GB
    • During 3rd render: 5.9 GB
    • After 3rd render: 5.3 GB
    • During 4th render: 7.1 GB
    • After 4th render: 6.4 GB
    • During 5th render: 7.8 GB (paging)
    • After 5th render: 7.2 GB (possibly paging)
    • During 6th render: 7.8 GB (page 10 GB)
    • After 6th render: 7.2 GB (paging)
    • During 7th render: 7.7 GB (page 12.x GB)
    • After 7th render: 7.3 GB (page 11.3 GB)

    Screenshot taken partway through the test run:


  • Memory leak?

    Graphics card memory doesn't seem to be released after encoding with VoukoderPro. Up to and including v0.7.2.6.

    I have GPU globally disabled in Vegas 20 (because it slows rendering down - it's a Vegas thing!).

    Typically I'm running 2 instances of Vegas in parallel, and rendering in both simultaneously using NVENC through VoukoderPro. My first pair of VoPro renders usually use about 3 GB of VRAM. When the renders end, the memory is not released. If I run a second set of renders, now usage is up to 6 GB. If I run a third set of renders, now usage is 9 GB (paged, my current card is 8 GB). And so on.

    This behavior has not been observed with other render methods, including Voukoder 13.0.2. These other methods release the video memory as soon as the render is stopped/finished.

    The memory is released if I restart Vegas.