MPC Video renderer issue. D3D11 issue. HEVC_10bit doesn't render properly. D3D11, DXVA(copy-back), DXVA(native) inconsistency.

Main issue: Opening HEVC_10bit video via MPC Video Renderer or MadVR with D3D11 as HW decoder doesn't work properly. Instead of playing video - static black screen is being opened instead, no playback, no sound. Timebar hover preview works fine, you can see frames from video. But as soon as you try to jump to point on timebar - MPC-HC instantly hangs up and stops responding. On Extended EVR(custom) and Sync Renderer HEVC_10bit video renders on CPU despite ability to decode it with GPU HW acceleration.  
Additional information: AV1, AV1_10 bit, HEVC all use GPU HW acceleration properly with D3D11. Only major issue is that HEVC_10bit cannot open up on these 2 renderers and smaller issue that it is rendering on CPU instead on 2 other key renderers.

how it looks like:

Secondary issue: D3D11, DXVA(copy-back) and DXVA(native) allow different lists of HW decoding capabilities, and none support full possible list. Everything is selected as allowed though.
For D3D11:
AV1, AV1_10bit, HEVC = GPU decoded.
HEVC_10bit = CPU decoded (except MPCVR and MadVR, which don't play)

For DXVA (copy-back)
HEVC = GPU decoded
AV1, AV1_10bit, HEVC_10bit = CPU decoded

For DXVA (native)
HEVC, HEVC_10bit = GPU decoded
AV1, AV1_10bit = CPU decoded

Expected support:
HEVC, HEVC_10bit, AV1, AV1_10bit = GPU decoded


Tertiary issue: This particular issue  happens only with specific video sample, it seems. When using DXVA (both copy-back and native) and opening some HEVC_10bit video via MPC Video Renderer it can playback black screen. This time it is actually playing but "improperly", sound on. Video source is originally 8k30 AV1 video reencoded into 4k30 HEVC_10bit. Why specific video? Because videos that were reencoded to 4k30 and 8k30 HEVC_10bit play fine at the same time. This issue also doesn't appear on any other renderers besides MPC Video Renderer.
Additional information. Seems like only videos that are reencoded from specific 8k30_AV1 (probably key point is AV1 there)) to HEVC_10bit are affected. Used software for reencoding - Handbrake, version 1.6.1(latest). If needed i can apply these videos later.

this filer have issues with not rendering properly:

this file does NOT have issues:

----------------------------------------------

OS version: Windows 11, 22H2, build 22621.1778
Version of codec pack: Mega 17.6, 17.63
CPU: AMD Ryzen 5 5800X3D
GPU: AMD RX 6750XT 12GB
GPU Driver version: 23.5.1 (latest optional)

It is possible that the GPU driver only supports AV1 hardware acceleration with D3D11.

The issues with HEVC 10-bit acceleration might be due to the files itself, or a driver bug. If you can upload the problematic file that has issues with DXVA2, then I can test it on my system.

With MPC Video Renderer and MadVR the D3D11 decoder can work in "native" mode. With other renderers it is always copy-back. You can force it to work in copy-back mode by selecting the GPU in the decoder settings. Maybe then it works better on your system.

It is possible that the GPU driver only supports AV1 hardware acceleration with D3D11

Hmm... Just checked with special tool (took some googling to find this tool, actually)... Yeah, you're right, it is actually ONLY encoding that is supported only through D3D11... WHY, though?!!!. Also strange that AV1_10bit also counts as simple AV1 in terms of GPU HW decoding via D3D11... Idk, why, for other codecs it must be explicitly stated. But OK. This one is answered.

Spoiler:

In addition managed to check HEVC_10bit. It should work fine. In DXVA(copy-back) it utilizes D3D11 engine but still have HW acceleration, while with DXVA(native) it actually uses DXVA much more and i can only occasional D3D11_VideoProcessorBlt and D3D11_CreateVideoProcessor from D3D11 set.
But when i actually try to feed HEVC video to check through this tool, for some reason it only suggests D3D11 decoding even though:
1. Windows extension seemingly supports D3D11 AND DXVA2 (as it shows), but only allows HW decoding in D3D11 mode
2. I know for sure DXVA2 mode should work for HEVC and HEVC_10bit decoding. At least because it works when i set decoder into DXVA(native) mode it actually does HW decoding for HEVC_10bit... And HEVC is for sure natively supported.

Can it be issue of this extension... But it is MediaFoundation and not DirectShow. Hmm... I may look into it. Because it also shows D3D11 only even for normal HEVC files. This tool is also unable to look through LAV decoder support list.

Spoiler:

The issues with HEVC 10-bit acceleration might be due to the files itself

Maybe, potentially... Even though not as likely in case of third issue. Because i reencoded 2 files with same tool on same settings just from different sources. Both are ended up as HEVC_10bit. One runs fine, second doesn't render image despite playing back.

If you can upload the problematic file that has issues with DXVA2, then I can test it on my system

Sure.
https://drive.google.com/file/d/1TqYxt9HHgCZMJo82twXpE77EaLe65snL/view?usp=sharing
AV1 8k30 video reencoded into HEVC_10bit 1080p30 video. Size: 206 MB

AV1 only working with D3D11 is because Microsoft and the driver manufacturers no longer support Windows 7. The applications from Microsoft on Windows 10/11 use D3D11. All GPUs that support AV1 support both 8 and 10-bit. So no need to differentiate.

D3D11_VideoProcessorBlt and D3D11_CreateVideoProcessor are functions used by the video renderer. They are not HW decoding related.

I do also see an issue with DXVA2 copyback not working. I will investigate deeper later.

I would suggest to test with some files created by others. There might be some kind of issue with the files you have created yourself.

FFmpeg shows these warnings, so the file does have an issue:
[hevc @ 0000000000178fc0] VPS 0 does not exist.
[hevc @ 0000000000178fc0] SPS 0 does not exist.

So likely a bug in Handbrake or its encoder. If you used the hardware encoder, then try x265 instead (will be much slower). If there is a bug in the AMD hardware encoder, then a different driver version might help.

Due to the error, copyback mode will fall back to software decoding. That DXVA2 native works is just luck. That can not fall back so it can fail like in D3D11 case.

Admin wrote:FFmpeg shows these warnings, so the file does have an issue:

Hmm, ok, i will report it as issue to Handbrake team. It does seem to be issue related to transcoding from AV1 source.
But what about not being able to playback any HEVC_10bit file via D3D11 MPCVR and MadVR? Player just freezes. And with DXVA HW accelerated decoding works fine as well. Windows default player also plays it fine with HW acceleration working, so it shouldn't be driver issue.

Example file
https://drive.google.com/file/d/1Nak4xA3NuDsxeFq23oPkhw55umTXVIuJ/view?usp=sharing
Transcoded from HEVC 1080p60 to HEVC_10bit 1080p60. Size: 8.65 MB

This one still cannot be played, even if it doesn't meed third issue.

That file has the same warnings. So also technically broken.

I can only suggest to force copyback mode in case of D3D11 if you want to play these non-conformant files.

Admin wrote:That file has the same warnings. So also technically broken.

Yup, can see that. It can explain why it shows black screen with sound...

Also, i tried VCEEncC as full on up-to-date encoder. I definitely messed up with some parameters, as i got no sound in output (yup, just saw that i needed to write up additional argument for that), but resulting transcode from av1 to hevc_10bit was complete success. Nothing froze, neither there was black screen. So it was complete success, showing that issue isn't in HW encoder or driver at all...

BUT!!! Before i used VCEEncC i also messed up with it as being plugin to AviUtl (which i couldn't properly set up?). So i could see another sort of issue with it, kind of unrelated but similar... while inputing av1 source there was no preview and after transcode resulting video had black screen instead of video stream (bitrate also went down to 300kb/s)... But this issue probably is tied to av1 and utility being ages old, meaning it probably cannot properly decode stream. On other hand... Despite all other issues resulting file had actually proper stream header, so ffprobe (yes i found how to check these files, thanks) showed no errors, and player hadn't frozen.

Meaning that these particular issues of player freezing can be related to these two errors (aka VPS0 and SPS0 doesn't exist)... And, probably issue of no picture as well?

Also... What difference between HEVC and HEVC_10 bit in context of profiles anyway?
Because i can set up HEVC Main@L6.2@High - 8bit 10bit depth, or HEVC Main10@6.2@high - 10 bit depth. Second preset cannot go to 8 bit, but first can go to 10 bit...
Both worked fine via VCEEncC utility (and i checked, they definitely used HW encoder). [it also did beautiful transcode of 8k video of Norway... Still, original YouTube compression is on INSANE level. Original 8k video is 585 MB, while transcoded with CQP is 2.5GB. Sure, i can go lower, but tbh it is still impressive. AV1 is really powerful, for sure though]

Encoding in 10-bit should give Main10 profile. But decoder looks at video stream parameters rather than the profile "name".

Well, they fixed Main10 profile issue and VPS0/SPS0 (it was inherited from Adrenaline screen recording it seems)
But black screens still happen with their transcoded video.

We only confirmed that:
1. It is specific to source and encoder
2. Now only MPC renderer is black screening
3. With Handbrake encoded video MadVR plays nicely only in D3D11, as when in both DXVA2 modes video plays at half FPS and with horrible stutter.
4. VPS0 and SPS0 being not found was not root cause of a problem
5. It may be related to source resolution

L6.2 is also overkill. For re-encoding to 1080p and 4k it is sufficient to use profile level 5.1.

Stutter might be a AMD driver issue as well. Here is a similar issue:
https://github.com/Nevcairiel/LAVFilters/issues/496

Admin wrote:L6.2 is also overkill. For re-encoding to 1080p and 4k it is sufficient to use profile level 5.1.
Stutter might be a AMD driver issue as well. Here is a similar issue:

Well, i just cannot make that many .bat presets))) not that level 6.2 actually affects HW encoder performance much, right? Or does it?

For me that much stutter only happened for 8k videos. Probably because unlike Nvidia RTX 3000+ series which supports up to 8k60, AMD only supports up to 8k30 playback. And maybe DXVA inflicts additional performance penalty? because ffprobe test says it can be decoded with dxva with 35-50 FPS approx, which should be enough. But in MPC-HC for some reason it renders with about 14-15 FPS.
And again, with D3D11 or DXVA2 (copy-back) this 8k30 video plays nicely and smooth.

Ah, just checked my 4k60 sample, with D3D11 DXVA2 (copy-back) it plays back smoothly at 60 fps, but with DXVA2 (native) despite smoothness it plays back at about 45 fps... There is definitely something tanking performance.
On other hand EVR custom had no such issue, maybe because of it's bufferisation? Or because different support of HW acceleration?

Also there was other case of video stuttering, but that time it was because VCEEncC --fps flag despite making video with 30 fps if log and mediadata to be believed, actually created video that played back at 60FPS of source. And, AMD HW decoder cannot play back 8k60... Strange to see video which mediadata reports it to be 30FPS, but player reports it trying to render at 60 (and it was same duration, so i am inclined to think it was mediadata that was wrong)

If you use DXVA2 Native (= DirectX 9) then you should uncheck DirectX11 in MPCVR so that it also uses DirectX9. Otherwise you lose performance.

Admin wrote:If you use DXVA2 Native (= DirectX 9) then you should uncheck DirectX11 in MPCVR so that it also uses DirectX9. Otherwise you lose performance.

You can do that?... Wait... Oh, found it. This parameter is definitely placed in not easy to find place.
Yeah, you were right, if DX11 in MPC VR is disabled performance at 8k30 videos goes to normal. That explains stutter, performance actually was dampened.

Also, when i set HW decoder to D3D11 and MPC VR to DX9, these black screen videos don't play at all. While with D3D11/D3D11 they play as black screen with sound... Well, investigation still in process)

Hmm for some reason D3D11 decoder and D3D9 player for 1080p and 4k actually plays video stream correctly... Meaning black screen issue is definitely rooted into DX11 renderer / decoder and linked to specific combination of source and transcoding format. Yay, RNG bugs.
Soo... it seems like DX9 renderer is completely unable to render 8k HEVC10 videos. But is fine with 8k HEVC ones. Well, i guess DX9 must have had upper limit somewhere)