Sapphire 2019.03 Low Level Exception [Sapphire Memory issue] GPUVideoFilter::Render [Windows]

#1

Premiere CC2019 13.0.3 Build 9. Rendering Transition in timeline. Low memory sapphire bug. Nvidia GTX 660. Latest drivers. Xeon Quad core.

Exception

#2

#3

Problem is ongoing, this appears to be related to an old issue I had previously thought was fixed in Sapphire 2019, seems to occur over and over in the same locations while encoding videos, issue happens when GPU is disabled in Sapphire settings as well

#4

Thing is memory usage on GPU not quite maxed out, & on PC is not maxed out at the time of the error, according to monitoring and HWMontior & GPUZ software; Result of Media Encoder:

Tue Apr 02 16:32:32 2019: GPU render failed for S_DissolveFilm.Dissolve Film (out of memory). Falling back to CPU.
Tue Apr 02 20:48:21 2019: SBuild\sapphire-ae\final-em64t\primitives\render-context.cxx:134: CUDA error: out of memory
Tue Apr 02 20:48:21 2019: S_Dissolve failed execution plan on 1280 x 1280 clip with 32 bit depth (temps 1280x1280)
Aborted from SBuild\sapphire-ae\final-em64t\cuda\cuda-support.cxx:1241, type 7, code 0
Tue Apr 02 20:48:21 2019: ( (allocate-images :ref33.2g 4 2 gpu)
(convert-from-source 0 :ref33.2g)
(allocate-images :ref35.2g 4 2 gpu)
(convert-from-source 1 :ref35.2g)
(allocate-images :ref0.2g 4 2 gpu)
(run-primitive 1 -1 :ref0.2g dissolve :ref33.2g :ref35.2g [ 0.0328736 ] )
(free-images :ref33.2g)
(free-images :ref35.2g)
(convert-to-dest :ref0.2g)
(free-images :ref0.2g)
)
Tue Apr 02 20:48:21 2019: GPU render failed for S_Dissolve.Dissolve Simple (out of memory). Falling back to CPU.
Tue Apr 02 20:49:54 2019: SBuild\sapphire-ae\final-em64t\primitives\mem-manager.cxx:892: CUDA error: out of memory
Tue Apr 02 20:49:54 2019: S_DissolveBlur failed execution plan on 1280 x 1280 clip with 8 bit depth (temps 1280x1280)
Aborted from SBuild\sapphire-ae\final-em64t\cuda\cuda-support.cxx:1241, type 7, code 0
Tue Apr 02 20:49:54 2019: ( (allocate-images :ref42.2g 4 2 gpu)
(convert-from-source 0 :ref42.2g)
(run-primitive 1 -1 :ref42.2g postdivide :ref42.2g [ -1 ] )
(convert-to-dest :ref42.2g)
(free-images :ref42.2g)
)
Tue Apr 02 20:49:54 2019: GPU render failed for S_DissolveBlur.Dissolve Blur (out of memory). Falling back to CPU.

**[quote=“canyoncomputers, post:4, topic:5726, full:true”]
Tue Apr 02 16:32:32 2019: GPU render failed for S_DissolveFilm.Dissolve Film (out of memory). Falling back to CPU.
Tue Apr 02 20:48:21 2019: SBuild\sapphire-ae\final-em64t\primitives\render-context.cxx:134: CUDA error: out of memory
Tue Apr 02 20:48:21 2019: S_Dissolve failed execution plan on 1280 x 1280 clip with 32 bit depth (temps 1280x1280)
Aborted from SBuild\sapphire-ae\final-em64t\cuda\cuda-support.cxx:1241, type 7, code 0
Tue Apr 02 20:48:21 2019: ( (allocate-images :ref33.2g 4 2 gpu)
(convert-from-source 0 :ref33.2g)
(allocate-images :ref35.2g 4 2 gpu)
(convert-from-source 1 :ref35.2g)
(allocate-images :ref0.2g 4 2 gpu)
(run-primitive 1 -1 :ref0.2g dissolve :ref33.2g :ref35.2g [ 0.0328736 ] )
(free-images :ref33.2g)
(free-images :ref35.2g)
(convert-to-dest :ref0.2g)
(free-images :ref0.2g)
)
Tue Apr 02 20:48:21 2019: GPU render failed for S_Dissolve.Dissolve Simple (out of memory). Falling back to CPU.
Tue Apr 02 20:49:54 2019: SBuild\sapphire-ae\final-em64t\primitives\mem-manager.cxx:892: CUDA error: out of memory
Tue Apr 02 20:49:54 2019: S_DissolveBlur failed execution plan on 1280 x 1280 clip with 8 bit depth (temps 1280x1280)
Aborted from SBuild\sapphire-ae\final-em64t\cuda\cuda-support.cxx:1241, type 7, code 0
Tue Apr 02 20:49:54 2019: ( (allocate-images :ref42.2g 4 2 gpu)
(convert-from-source 0 :ref42.2g)
(run-primitive 1 -1 :ref42.2g postdivide :ref42.2g [ -1 ] )
(convert-to-dest :ref42.2g)
(free-images :ref42.2g)
)
Tue Apr 02 20:49:54 2019: GPU render failed for S_DissolveBlur.Dissolve Blur (out of memory). Falling back to CPU.

**

#5

Total available graphics memory 6143, shared system memory 4095, total PC ram 8192; Actual onboard GPU memory 2 gb.

Must be tested with GPUs like 2GB GTX 660 on machines with 8 gb of ram. Happens on pictures with Sapphire plugins as much as on 1080P videos.

#6

GTX660 with 2gb ram

#7

I disabled host_gpu_integration, and was able to finally finish encoding the one timeline segment that constantly crashed. However historically it was always hit and miss, maybe 7 out of 10 attempts would fail. I will keep you posted on my progress. CUDA was also disabled in successful test.

C:\Program Files\GenArts\SapphireAE\s_config.text

// Enable hosting of native images in gpu memory. To enable the host to
// provide native images in gpu memory, set this variable to 1.
host_gpu_integration: 0

#8

Continues to fail with CUDA enabled in Media Encoder:

ue Apr 02 22:08:00 2019: GPU render failed for S_Dissolve.Dissolve Simple (out of memory). Falling back to CPU.
Tue Apr 02 22:10:18 2019: SBuild/sapphire-ae/final-em64t/core/converter-kernels.cu:1592: CUDA error: out of memory
Tue Apr 02 22:10:18 2019: S_DissolveGlow failed execution plan on 1280 x 1280 clip with 32 bit depth (temps 1280x1280)
Aborted from SBuild\sapphire-ae\final-em64t\cuda\cuda-support.cxx:1241, type 7, code 0
Tue Apr 02 22:10:18 2019: ( (allocate-images :ref90.2g 4 2 gpu)
(convert-from-source 1 :ref90.2g)
(allocate-images :ref14.2g 3 2 gpu)
(run-primitive 1 -1 :ref14.2g glow :ref90.2g [ 0.199997 0.199997 0.199997 ] [ 1 1 ] [ 0.6 0.719998 0.839996 ] [ 3.46988 3.46988 3.46988 ] [ 1 ] [ 1 ] [ 0 ] )
(allocate-images :ref12.2g 4 2 gpu)
(run-primitive 1 -1 :ref12.2g alpha_from_rgb_max :ref14.2g [ 1 ] [ -999 -999 -999 -999 ] )
(free-images :ref14.2g)
(run-primitive 1 -1 :ref12.2g screen_super :ref12.2g :ref90.2g )
(free-images :ref90.2g)
(run-primitive 1 -1 :ref12.2g postdivide :ref12.2g [ -1 ] )
(convert-to-dest :ref12.2g)
(free-images :ref12.2g)
)
Tue Apr 02 22:10:18 2019: GPU render failed for S_DissolveGlow.Dissolve Glow (out of memory). Falling back to CPU.

#9

Seems Sapphire doesn’t effectively and successfully hand off to CPU/RAM after CUDA failure. Previous test was successful with CUDA disabled; will update.

#10

Third success in a row with both CUDA & host_gpu_integration disabled

#11

Success with CUDA disabled & Host_gpu_integreation & use_gpu enabled. (removed & updated all Visual C++ redistributables) Allow me to me explain:

Before disabling host_gpu_integration I did have a problem encoding with CUDA disabled in the same effects that would crash with CUDA on; Encoding would stay on a frame for many hours, CPU was using a steady 24% all night long with little to no progress, ucrtbase.dll!o_exp+0x20 was the thread taking up most of the CPU, I read here https://forums.adobe.com/thread/2391419 that re-installing Microsoft Visual C++ Redistributables will fix a hang related to ucrtbase.dll in Premiere. I ran Visual C++ all in one updater here https://www.majorgeeks.com/files/details/visual_c_runtime_installer.html and it seems to have sped up Premiere Pro’s loading, functionality, and so far, no longer does Adobe Media Encoder push the GPUVideoFilter::Render memory error with CUDA disabled & both host_gpu_integration & use_gpu ENABLED. My gosh what a run around.

The only other change I have made was to adobe media encoder 13.0.2 build 39 settings.ini to enable 4 cpu threads instead of the previous 3 cpu threads. I will re-run the test with 3 threads just to be sure.

#12

Set mthreadcount=3 and it encoded without error again. So far it seems so long as cuda is disabled in media encoder there are no issues. I will keep you posted

#13

Success with CUDA enabled & host_gpu + use_gpu enabled. Ok then. Previously with CUDA enabled there was 6 consecutive failures. May be redistributables, though I thought it was failing with them too, it maybe hit and miss (one out of 10). I was only encoding problematic segments of the timeline for testing. The digging continues.

(update: rendering the entire project in media encorder with cuda and the aformentioned enabled resulted in failure, disabling cuda it was fine.)

#14

This forum is not the right place to handle a support case. This forum is place for Sapphire users to discuss tips, tricks, product feature requests, and best practices. If you have a support case and/or support related question, please direct those questions to the Support page on the Boris FX website. This forum should be strictly used for Sapphire usage questions.