can't make DirectShow to use dxva2n decoder (uses avcodec instead) with HEVC video

Hi all,

I'm trying to watch a particular 180 sbs video in VR, I think it's the heaviest I've got so far. I'm adding MediaInfo for the video track at the end of the post.
I've installed K-Lite Codec Pack and MPC-HC that comes with it pretty much supports this video out of the box (I selected madVR renderer in MPC-HC Options): LavSplitter gets launched and LavVideo Properties Hardware Acceleration say:
- Hardware Decoder DXVA2 (native), Active Decoder: dxva2n, Active Hardware Accelerator: NVIDIA GeForce RTX 2070.
This configuration uses ~30% CPU and ~70% GPU video decoding capabilities and plays the video at my 3440x1440 screen just fine.

Now, when I try to play the same video through any VR video player that supports DirectShow rendering (such as DeoVR or Whirligig 4.8+), I get a different set of options in LavVideo Properties:
- Hardware Decoder DXVA2 (native), Active Decoder: avcodec, Active Hardware Accelerator: <none>.

The lack of hardware acceleration obviously makes the video almost unvieweable, immediately bringing CPU usage to 100% while GPU video decoding stays at 0.

An interesting thing to mention is that I get the same avcodec active decoder outside of VR when viewing the video in both x86 WMP but x64 WMP version plays the video in the same way MPC-HC does, via dxva2n with active 2070 RTX acceleration. So it looks like for some reason the VR players prefer... 32-bit codecs for DirectShow? I'm guessing at this point, I don't understand the codec system enough to make an educated guess.

The only things I have changed in the Codec Tweak Tool at the moment are disabling 64-bit MS codecs and selecting LAV Video as preferred directshow decoding filter for every 64-bit option in there.
Last week though when I was desperate I tried more harsh options, including things like disabling Media Foundation for all the formats, both 32-bit and 64bit & installing only 64-bit codecs from the very beginning. VR players in DirectShow mode keep falling back to avcodec for some reason. I also tried renaming the video from .mp4 to .video since I read that can lead directshow off the trail and make it use a different codec, to no avail.

Sooo TL;DR is how do I force DirectShow to use dxva2n with hw acceleration in VR?

One last thing to mention is the specifics of my hardware configuration. I've got a laptop with an external GPU, as in, plugged in via Thunderbolt. Both VR DisplayPort and the main display HDMI are plugged into the external GPU box, the USB end of the VR rig is plugged into the laptop's USB slot. I've been running SW: Squadrons lately in this configuration with no problems whatsoever so I don't believe either the configuration itself or the throughput/performance are the issue.

- Laptop is Lenovo T480, Intel Core i7-8550U
- External GPU is Aorus RTX 2070 Gaming Box
- VR rig is Oculus Rift S
- Windows 10 v2004 OS build 19041.572
- K-Lite Codec Pack 15.8.7 Mega

Thanks in advance!

MediaInfo for the video:

klcp_codec_log:

The tricks to rename to .video and to disable Media Foundation are useful for WMP only. That is the only player that dynamically chooses between DirectShow and Media Foundation.

It is weird that DXVA2 isn't working in 32-bit WMP the same way as with 64-bit. Might be due to differences in your preferred filter settings ("Use Merit" vs "LAV"). Does the file have subtitles? VSFilter will disable use of DXVA2 native when it gets used for subtitles.

When using DXVA2 Native the video decoder (LAV) needs to be connected directly to the video renderer. And that video renderer must also support DXVA2. From the 3 standard renderers that are included with Windows, only "Enhanced Video Renderer" is compatible. However, most DirectShow applications use the old "Video Mixing Renderer 7" as that is the system default renderer (for backwards compatibility reasons).

So the first solution that you can try is to use DXVA2 Copyback instead of Native. Copyback works with all renderers. Only downside is that the video data must be copied back and forth to the GPU, which costs a bit of performance. With very high resolution video, the amount of data is very large, so some systems have performance issues. CPU and memory bandwidth can be a bottleneck.

If that doesn't work, then we can use a trick to change the renderer. But test the above first.

Is DeoVR 32-bit or 64-bit?

I gave it a try and unfortunately DXVA2 copy-back decoder (active decoder dxva2cb direct) performance is worse to the point that even MPC-HC itself stutters & freezes. Same story for NVIDIA CUVID.

I get interesting results with D3D11 though. Its active decoder is "d3d11 cb direct" and it actually uses RTX 2070 for rendering, and it plays the video just fine in MPC-HC, just as good as DXVA2 native does! In VR though I see GPU usage at 100%, video decode usage at ~70% and CPU at ~50%, yet playback is too slow - I guess I hit the GPU limits, although that is a bit surprising, to say the least.

I get approximately the same load values for GPU and CPU when using either DXVA2 native or D3D11 in MPC-HC, so I assume I'd have the same performance problem in VR even if managed to force directshow to use DXVA2 native in VR.

DeoVR is 64-bit, sorry I forgot to mention it earlier.

I guess the answer to my original post is "use D3D11 decoder". I'd like to try that trick to change the directshow default renderer that you've mentioned just to make sure I've covered all bases. How do I do that?

D3D11 can automatically switch to native mode when used together with madvr/mpcvr as renderer. So that is why it is smooth in MPC-HC. In DeoVR it will use copyback.

If you enable the internal decoders in MPC-HC settings, which use exact same LAV Decoders, then you can use different settings for MPC-HC compared to other players.

You can change system default renderer this way:
GraphStudioNext > Menu > Graph > Insert Filter > Enhanced Video Renderer > Set Merit > 0x00800001 (preferred + 1)

This with the assumption that DeoVR uses the default renderer. It might actually be using its own custom renderer. You will have to ask DeoVR developers for details and ask about compatibility with DXVA2 native.

Another option is to use Media Foundation for playback if DeoVR supports that. In that case you need to install the HEVC codec from the Windows 10 store. You can also find an installer for it here:
https://free-codecs.com/hevc_video_extension_download.htm

Thank you!

I first tried changing the GraphStudioNext approach (also made sure that MERIT is chosen as the preferred way in both preferred decoders and preferred splitters in the Codec Tweak Tool) but I was still getting avcodec with no hw acceleration in DeoVR and Whirligig.

I then installed the HEVC codec via a link you provided and used Whirligig player to play the video using Media Foundation path... and it works! It actually works! No stuttering or freezes, it just works!

Thank you so much for your help, this was a month-long quest for me.