No HW Acceleration for some 4K videos

by nikkoaki Sun Nov 12, 2017 7:18 pm

Greetings,

My integrated intel gpu supports 4k decoding acceleration for h264 videos.. For example, videos shot on 4k from a iphone play fine, 4k videos downloaded from youtube also play fine.
Some days ago i bought a phone that can shoot videos in 4k too but unfortunately MPC-HC doesn't use dxva on them, so it's very slow.
Comparing it to an iphone video on MediaInfo shows that the only differences are:

Iphone:

Format profile : Main@L5.1
Format settings, CABAC : Yes

My phone:

Format profile : Baseline@L5.1
Format settings, CABAC : No

In theory, the files from my phone should be easier to decode.
Is this driver related (i'm using the latest) or something that can be fixed on your side (or even LAV's side)?
Small sample here: https://www.sendspace.com/file/4wph4n

by **Admin** Sun Nov 12, 2017 7:41 pm

The problem is that it uses baseline profile. That can include (optional) features of the H.264 format that are not compatible with hardware decoding. DXVA2 Hardware decoding is not used for such videos because there is no guarantee that it can be decoded correctly.

I may add a (hidden) option to override this restriction in the future.

by nikkoaki Sun Nov 12, 2017 8:59 pm

Thank you for your answer.
If you add it, great.. if you don't, it's no big deal really since i don't plan on shooting 4k a lot and if i do, i'll probably re-encode it to 1080p to get more detail than just shooting in 1080p natively.

Also, is this a driver/device problem? You said that the DXVA isn't usually used to decode the baseline profile but after a short google search i found (for example) this implementation that decodes it fine (at least it says so):
https://msdn.microsoft.com/library/windows/desktop/dd797815(v=vs.85).aspx

Also, i used handbrake to re-encode the video to 1080p (also baseline and CAVLC) and HW acceleration works fine.

by **Admin** Mon Nov 13, 2017 12:02 am

Baseline profile can have an additional value that indicates that it is compatible with Main profile. The original file is probably missing that, while the converted file has it. HW acceleration works fine here too for re-encoded file, even when it is 4k.

by nikkoaki Mon Nov 13, 2017 11:41 am

Admin wrote:even when it is 4k.

You are right *facepalm*. I've been using handbrake and hybrid, trying to figure out what exactly breaks compatibility but to no avail.
Oh well, it is what it is, thanks anyway.

by **Admin** Mon Nov 13, 2017 2:45 pm

Those tools most likely always encode baseline profile in compatibility mode.

by nikkoaki Mon Nov 13, 2017 5:45 pm

I just noticed that using Quicksync instead of DXVA gives me smooth playback. I kind of don't like quicksync for decoding since (at least for me) it always gave me problems in the past like the sound and image getting out of sync and small random stutters.
I'll just switch to it whenever i need then.. It's not a perfect fix but it's good enough for me

by **Admin** Tue Nov 14, 2017 4:42 pm

Quicksync is outdated and doesn't support HEVC and VP9, so it isn't something I would recommend using by default.

by nikkoaki Tue Nov 14, 2017 5:56 pm

I won't, i'll just change to it when i need to play these types of file.
Thanks again.

by notcyf Mon Nov 27, 2017 8:35 am

Admin wrote:The problem is that it uses baseline profile. That can include (optional) features of the H.264 format that are not compatible with hardware decoding. DXVA2 Hardware decoding is not used for such videos because there is no guarantee that it can be decoded correctly.

I may add a (hidden) option to override this restriction in the future.

Doesn't baseline just mean that it uses a minimal set of compression features of the H.264 spec? Main uses the normal amount and High uses the highest possible.

Most smartphones(Android) use Baseline because it requires the least amount of processing power for encoding decoding and thus ultimately saves (a lot of) power. Android 6+ requires you to have a Baseline and a Main encoder/decoder, so if you have that(Main encoder is recommended, but not required). All videos made with my camera are Baseline files that work correctly on DXVA, which means they should be main-compatible flagged(?).

You could try manually setting the flag, though not exactly sure what "Main-compatible" is supposed to mean. I guess there's a technical limitation to the spec that makes it impossible to decode certain Baseline encoded videos on a Main/High Profile HW decoder, so be careful with doing this.

by **Admin** Mon Nov 27, 2017 2:52 pm

Baseline is not a strict subset of Main. Hence the compatibility flag.

by nikkoaki Mon Nov 27, 2017 9:48 pm

notcyf wrote:
You could try manually setting the flag

I have no idea on how to do this from my side without re-encoding the video.

notcyf wrote:
so be careful with doing this

What's the worst that could happen? Player could crash, nothing more in my opinion.. Wink

BTW, i tried the file on kodi and it seems to use DXVA just fine

by **Admin** Tue Nov 28, 2017 1:08 pm

Worst would be a crash. Most likely just some video corruption.

Next version of the pack will be able to use DXVA for non-constrained baseline. With an option in the registry to disable it if necessary. I assume most videos in the wild are compatible but are just missing the compatibility flag.

by nikkoaki Tue Nov 28, 2017 1:26 pm

Hey, thanks a bunch!

by notcyf Tue Nov 28, 2017 3:22 pm

nikkoaki wrote:
notcyf wrote:
You could try manually setting the flag

I have no idea on how to do this from my side without re-encoding the video.

notcyf wrote:
so be careful with doing this

What's the worst that could happen? Player could crash, nothing more in my opinion..

BTW, i tried the file on kodi and it seems to use DXVA just fine

I have only found a tool that can change the profile level, not the actual flags :/ though an option to force decoding by DXVA in the decoder is a better solution anyway

by nikkoaki Tue Nov 28, 2017 7:55 pm

notcyf wrote:
I have only found a tool that can change the profile level, not the actual flags :/ though an option to force decoding by DXVA in the decoder is a better solution anyway

Yep, i tried the program you posted earlier, before you deleted the post and unfortunately it doesn't work. I'm able to change the level but media info always detects it as 5.1 (although the level editor detects it as level 1). But like you said, that's not what makes the video incompatible with DXVA (or LAVs implementation)

Admin, since i'm the only one complaining about the stupid implementation my phone uses and (most likely) no one is having any troubles with their files, don't you think you should take the opposite approach and leave that option disabled as default in case it breaks a perfectly good codec pack just because of one person? One (like myself) could enable it with a reg tweak or maybe a check mark in Codec Tweak Tool.
Unless of course you are sure that it won't break anything.

by **Admin** Tue Nov 28, 2017 8:32 pm

Which brand/type phone do you have? I am curious how common these files may me.

by nikkoaki Tue Nov 28, 2017 8:40 pm

Admin wrote:Which brand/type phone do you have? I am curious how common these files may me.

https://www.gsmarena.com/bq_aquaris_x_pro-8641.php

by notcyf Wed Nov 29, 2017 2:08 pm

Yep, i tried the program you posted earlier, before you deleted the post and unfortunately it doesn't work. I'm able to change the level but media info always detects it as 5.1 (although the level editor detects it as level 1). But like you said, that's not what makes the video incompatible with DXVA (or LAVs implementation)

This is because the file was ORIGINALLY encoded in L5.1, MediaInfo detects this and thus the tool is useless in that scenario.

This tool is only meant for changing an over-leveled h264 5.1 stream to 5.0 or 4.1 so that certain TV's can actually play the file due to normal BluRay's only requiring level 4.1, but it cannot magically change actual levels. Because someone encoded a 4.1 BluRay source to 5.1 means that changing the header won't cause any issues with playback. It's there because most TV's won't try and apply self-correction, and will just fail to play the file. If the source was indeed a correctly set profile level 5.1, changing it back to 5.0 or 4.1 in the header will probably cause issues in playback on those TV's and MediaInfo would still show the correct level.

The same goes for Baseline videos: If it is indeed just a wrongly set flag, then changing the header could work, but it won't magically allow DXVA to decode baseline videos that use compression features not supported by DXVA.

By the way, MKVToolNix detects a DXVA/Main compatible Baseline video as a "Constrained Baseline" video, so i guess this is the correct term for a DXVA compatible Baseline file.

by notcyf Wed Nov 29, 2017 2:31 pm

I did some testing on my Samsung Galaxy S8+(a very popular phone) and came onto something interesting:

When recording a video on my Galaxy S8+ (Exynos version) using default settings(1080p) it will use High@L4 and DXVA will be used properly when playing back the file on my Windows 10 x64 Pro using DXVA copy-back

Now here's the interesting part: When i record in 720p resolution it uses Baseline@L3.1 and DXVA will NOT be used when playing back the file which means it does not have the compatibility flag.

Edit: Indeed it has constrained_set1_flag=0 using a h264 stream analyzer. I still haven't found a way to actually edit this value though, it most likely will require hex editing.

This means most likely that this is not a "stupid implementation in my phone" but in most Android phones.
This must be a very common problem indeed.

Edit2: Played around with hex editing and found the correct bit to change to set the constrained flags. I'm not a programmer so i had to find out the hard way.

File: baseline31nonconstrained.mp4 does not work with DXVA: No HW Acceleration for some 4K videos FljAGUq

No HW Acceleration for some 4K videos FljAGUq

1) I extracted the h264 stream from MKV/MP4 using ffmpeg.exe -i baseline31nonconstrained.mp4 -c:v copy baseline31nonconstrained.h264
2) I analyzed the stream using https://sourceforge.net/projects/h264streamanalysis/ which outputs as follows:

Code:: ==================== NAL ==================== forbidden_zero_bit : 0 nal_ref_idc : 3 nal_unit_type : 7 ( Sequence parameter set ) ======= SPS ======= profile_idc : 66 constraint_set0_flag : 0 constraint_set1_flag : 0 constraint_set2_flag : 0 constraint_set3_flag : 0 constraint_set4_flag : 0 constraint_set5_flag : 0

3) Opened a hex editor and changed At the beginning of the file:00 00 00 01 67 42 00 i changed it to 00 00 00 01 67 42 C0
4)i saved file as baseline31constrained.h264 and opened it in H.264 Stream Analysis and now it outputs like follows:

Code:: ==================== NAL ==================== forbidden_zero_bit : 0 nal_ref_idc : 3 nal_unit_type : 7 ( Sequence parameter set ) ======= SPS ======= profile_idc : 66 constraint_set0_flag : 1 constraint_set1_flag : 1 constraint_set2_flag : 0 constraint_set3_flag : 0 constraint_set4_flag : 0 constraint_set5_flag : 0

6) ffmpeg.exe -i baseline31constrained.h264 -c:v copy baseline31constrained.mp4
7) Opened baseline31constrained.mp4 and now DXVA is properly used: No HW Acceleration for some 4K videos YM9JAkC

No HW Acceleration for some 4K videos YM9JAkC

I would not recommend doing this the way i did it, because it messes up all MP4 header information, but this is proof that DXVA can indeed decode the file when manually setting the constraint_set1_flag=1 flag.

Bottom line is that this should be reported to hardware manufacturers. FFMPEG's default Baseline profile sets both constraint_set0_flag=1 and constraint_set1_flag=1(Tested it by re-encoding the original nonconstrained file using v:profile Baseline)

by nikkoaki Wed Nov 29, 2017 8:29 pm

Nice work.
The fact that the video also has constraint_set0_flag=0 makes me think that they (whoever made the encoding implementation for the phones) are aware that it doesn't obey to all constraints

constraint_set0_flag equal to 1 indicates that the coded video sequence obeys all constraints specified in clause A.2.1

from PDF

I found some untouched samples at gsmarena of relatively well known phones:

-Moto Z Play (similar SOC to mine): records 4K in High@5.1 so DXVA kicks in

-Xiaomi Mi MIX 2 (Snapdragon 835): records 4K in Baseline@5.1 so no DXVA
-Xiaomi Mi 6 - Same profile as the above
-Nokia 8 - Same as the above
-OnePlus 3 - Same as the above
-OnePlus 5 - Same as the above (yep)
-Blackberry Keyone - Same

All of the above use snapdragon soc's..

Some vendors like LG, Samsung and Sony use the high profile on all their 4k capable phones, be it AVC or HEVC.. it makes me think that it may be a licensing thing like paying a royalty to get better profiles and higher bit rates.

Edit: Guess i was way wrong thinking it was an isolated case.

by notcyf Thu Nov 30, 2017 10:38 am

nikkoaki wrote:Nice work.
The fact that the video also has constraint_set0_flag=0 makes me think that they (whoever made the encoding implementation for the phones) are aware that it doesn't obey to all constraints

constraint_set0_flag equal to 1 indicates that the coded video sequence obeys all constraints specified in clause A.2.1
from PDF

I found some untouched samples at gsmarena of relatively well known phones:

-Moto Z Play (similar SOC to mine): records 4K in High@5.1 so DXVA kicks in

-Xiaomi Mi MIX 2 (Snapdragon 835): records 4K in Baseline@5.1 so no DXVA
-Xiaomi Mi 6 - Same profile as the above
-Nokia 8 - Same as the above
-OnePlus 3 - Same as the above
-OnePlus 5 - Same as the above (yep)
-Blackberry Keyone - Same

All of the above use snapdragon soc's..

Some vendors like LG, Samsung and Sony use the high profile on all their 4k capable phones, be it AVC or HEVC.. it makes me think that it may be a licensing thing like paying a royalty to get better profiles and higher bit rates.

Edit: Guess i was way wrong thinking it was an isolated case.

It depends on what resolution you are recording with.
Any video that has constraint_set1_flag=1 also has constraint_set0_flag=1 because profile_idc=66 is identical to constraint_set0_flag=1

clause A.2.1(Baseline profile):

constraint_set0_flag equal to 1 indicates that the coded video sequence obeys all constraints specified in clause A.2.1.

clause A.2.2(Constrained Baseline profile):

Conformance of a bitstream to the Constrained Baseline profile is indicated by profile_idc being equal to 66 with constraint_set1_flag being equal to 1.

https://www.itu.int/rec/T-REC-H.264-201704-I/en

Edit:

Decoders conforming to the Constrained Baseline profile(which DXVA is capable of) at a specific level shall be capable of decoding all bitstreams in
which all of the following are true:
– profile_idc is equal to 66 or constraint_set0_flag is equal to 1,
– constraint_set1_flag is equal to 1,
– level_idc and constraint_set3_flag represent a level less than or equal to the specified level

This states that constraint_set1_flag is not always identical to profile_idc=66/constraint_set0_flag=1 so it might be that not all videos will be properly decoded by a Main profile decoder if this flag is not set before encoding with a specification compliant encoder.

@Admin While it might be a good idea to introduce a setting/parameter for forcing DXVA decoding a "normal" Baseline-profile encoded stream. One test isn't enough to prove whether decoding the Android Baseline-profile encoded streams will always work correctly, and (i think) it depends on what GPU the Windows-machine has.

by **Admin** Thu Nov 30, 2017 3:31 pm

It can be enabled in 13.6.8:

Code:: Windows Registry Editor Version 5.00 [HKEY_CURRENT_USER\Software\LAV\Video\HWAccel] "AllowBaselineH264"=dword:00000001

by notcyf Thu Nov 30, 2017 8:35 pm

The key doesn't seem to work. I copy pasted it in a notepad, saved it as lav.reg and opened it, saved it in registry. The key is there but it doesn't actually force DXVA decoding since my 720p@30fps BaselineL3.1 video is being software decoded according to the EVR renderer stats display.

by **Admin** Thu Nov 30, 2017 10:46 pm

Disable the internal decoder in MPC-HC. It currently only works for external instance of LAV Video.

by Sponsored content

No HW Acceleration for some 4K videos

No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos

Re: No HW Acceleration for some 4K videos