problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

View previous topic View next topic Go down

problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by DavidB3 on Fri Jul 29, 2016 7:25 pm

Hi.

I'm a computer programmer and I made a small converter (video files to animations).
It uses DirectX and SampleGrabber to extract the frames from the video files.
I recommend K-Lite to all the users of my application.
However I have problems with some video files, for example with 480p Youtube videos.
The samplegrabber is not automatically connected to the video decoder. I also tried to manually insert
it between the video decoder and the video renderer, it doesn't "want" to.
The video decoder is "LAV decoder". I tried with "ffdshow decoder" but
it crashes my application with all the video files I tried.

Here is an example of such file: https://drive.google.com/open?id=0ByKxAD_t9uvLVXI1alI0WGxEVXc
And a test app: https://drive.google.com/open?id=0ByKxAD_t9uvLUm9NSTNlcDJYOVE
The Snapshot button and the Callback event checkbox don't do anything on this video file.

Windows version: 8.1 Pro x64
K-Lite version: 12.2.7

I tried reinstalling K-Lite, no change.

Any thoughts on what can I try next...?

Thank you in advance.

Regards,
David

DavidB3

Posts : 7
Join date : 2016-07-29

View user profile

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by Admin on Sat Jul 30, 2016 1:17 am

Sample Grabber has very limited capabilities. You can see its limitations at the bottom of this page:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd377544(v=vs.85).aspx

Most likely cause of failure in your case is due to VIDEOINFOHEADER2.

One solution would be to use for example VMR9 as renderer and grab samples directly from there. See its documentation.

Admin
Admin

Posts : 2982
Join date : 2011-06-17

View user profile http://codecs.forumotion.net

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by DavidB3 on Sat Jul 30, 2016 3:36 am

Thank you for the suggestion.

But before deciding to use the Samplegrabber I did some research on Google and found other programmers having problems with VMR9 too. Not 100% similar with my problems but enough to not choose it.

And another thing: taking snapshots with VMR9 is so slow. On my 4 x 2 GHz processor it takes 0.55 sec to get a 1080p snapshot (and you can clearly see the playback stopping during this), while with samplegrabber I can set the playback rate to 4x and still get snapshots in realtime (without affecting the playback).

A solution would be to try first with the samplegrabber and, if it doesn't work, to set the video window to VMR9 and load the video file again.
But it's a lot of work... Sad

DavidB3

Posts : 7
Join date : 2016-07-29

View user profile

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by Admin on Sat Jul 30, 2016 4:44 am

Set the sample grabber to request RGB24 if you haven't already. Then try with NullRenderer.

Admin
Admin

Posts : 2982
Join date : 2011-06-17

View user profile http://codecs.forumotion.net

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by DavidB3 on Sat Jul 30, 2016 6:25 am

Admin wrote:Set the sample grabber to request RGB24 if you haven't already.

Already did.

Admin wrote:Then try with NullRenderer.

The problem is not connecting the sample grabber output pin to null renderer, but to connect the sample grabber input pin to the decoder output pin.
Unfortunately still no change.

DavidB3

Posts : 7
Join date : 2016-07-29

View user profile

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by Admin on Sat Jul 30, 2016 7:40 am

GraphStudioNext is an excellent to for experimenting with filter graphs.

I am unable to access your sample file, so I just download a random 480p youtube video.

I am able to (manually) construct these graphs:
LAV Splitter Source > LAV Video Decoder > Sample Grabber (RGB24) > Null renderer
LAV Splitter Source > LAV Video Decoder > Sample Grabber (RGB32) > Color Space Converter > VMR
LAV Splitter Source > LAV Video Decoder > Sample Grabber (YUY2) > AVI Decompressor (YUY2) > Color Space Converter > VMR

You can click on a pin for a lot of details about the pin and the pin connection.

Admin
Admin

Posts : 2982
Join date : 2011-06-17

View user profile http://codecs.forumotion.net

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by DavidB3 on Sat Jul 30, 2016 2:36 pm

Yes, it's a good tool, I used it many times.

Found out why couldn't connect the output pin of the decoder to the input pin of the sample grabber: it wanted a "Color Space Converter" filter inserted before the sample grabber. Although the output of the decoder and the input of the sample grabber looks identical.



Now it gets the snapshots but it look like with the wrong size and/or bit depth:

Now:


It should be:



This is the video: https://www.youtube.com/watch?v=tIJQkR-ofFo

And I'm using this https://addons.mozilla.org/en-US/firefox/addon/youtube-video-and-audio-dow/?src=userprofile to get the 480p with no sound. I tried with other addons too, same problem.



Late edit:

Seems that the TSampleGrabber component from DSPack is set to detect VideoInfoHeader2:

Code:
function TSampleGrabber.GetBitmap(Bitmap: TBitmap; Buffer: Pointer; BufferLen: Integer): Boolean;
var
   hr: HRESULT;
   BIHeaderPtr: PBitmapInfoHeader;
   MediaType: TAMMediaType;
   BitmapHandle: HBitmap;
   DIBPtr: Pointer;
   DIBSize: LongInt;
begin
   Result := False;
   if not Assigned(Bitmap) then
      Exit;
   if Assigned(Buffer) and (BufferLen = 0) then
      Exit;
   hr := SampleGrabber.GetConnectedMediaType(MediaType);
   if hr <> S_OK then
      Exit;
   try
      if IsEqualGUID(MediaType.majortype, MEDIATYPE_Video) then
      begin
         BIHeaderPtr := nil;
         if IsEqualGUID(MediaType.formattype, FORMAT_VideoInfo) then
         begin
            if MediaType.cbFormat = SizeOf(TVideoInfoHeader) then // check size
               BIHeaderPtr := @(PVideoInfoHeader(MediaType.pbFormat)^.bmiHeader);
         end
         else if IsEqualGUID(MediaType.formattype, FORMAT_VideoInfo2) then
         begin
            if MediaType.cbFormat = SizeOf(TVideoInfoHeader2) then // check size
               BIHeaderPtr := @(PVideoInfoHeader2(MediaType.pbFormat)^.bmiHeader);
         end;
         // check, whether format is supported by TSampleGrabber
         if not Assigned(BIHeaderPtr) then
            Exit;
         BitmapHandle := CreateDIBSection(0, PBitmapInfo(BIHeaderPtr)^,
            DIB_RGB_COLORS, DIBPtr, 0, 0);
         if BitmapHandle <> 0 then
         begin
            try
               if DIBPtr = nil then
                  Exit;
               // get DIB size
               DIBSize := BIHeaderPtr^.biSizeImage;
               if DIBSize = 0 then
               begin
                  with BIHeaderPtr^ do
                     DIBSize := GetDIBLineSize(biBitCount, biWidth) * biHeight * biPlanes;
               end;
               // copy DIB
               if not Assigned(Buffer) then
               begin
                  // get buffer size
                  BufferLen := 0;
                  hr := SampleGrabber.GetCurrentBuffer(BufferLen, nil);
                  if (hr <> S_OK) or (BufferLen <= 0) then
                     Exit;
                  // copy buffer to DIB
                  if BufferLen > DIBSize then // copy Min(BufferLen, DIBSize)
                     BufferLen := DIBSize;
                  hr := SampleGrabber.GetCurrentBuffer(BufferLen, DIBPtr);
                  if hr <> S_OK then
                     Exit;
               end
               else
               begin
                  if BufferLen > DIBSize then // copy Min(BufferLen, DIBSize)
                     BufferLen := DIBSize;
                  Move(Buffer^, DIBPtr^, BufferLen);
               end;
               Bitmap.Handle := BitmapHandle;
               Result := True;
            finally
               if Bitmap.Handle <> BitmapHandle then // preserve for any changes in Graphics.pas
                  DeleteObject(BitmapHandle);
            end;
         end;
      end;
   finally
      FreeMediaType(@MediaType);
   end;
end;

But for that video file MediaType.formattype is set to FORMAT_VideoInfo.

DavidB3

Posts : 7
Join date : 2016-07-29

View user profile

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by DavidB3 on Sat Jul 30, 2016 11:27 pm

Found a way to solve the problem for this situation: I set RGB32 instead of RGB24.
This way it doesn't require the Color space converter, it doesn't distort the snapshot and to convert from RGB32 to RGB24 is easy.

My question is: is this solution "universal" or do I have to detect the RGB type for each video file, set it and convert later?

Another thing:  here http://stackoverflow.com/questions/6436722/regarding-the-scope-of-sample-grabber-in-directshow/6440568#6440568 is mentioned a way to force RGB24.
Is this better? Unfortunately I couldn't test it. I downloaded DirectX Sdk but couldn't find "RGBfilters" (searched Google afterwards but still nothing)... Sad

DavidB3

Posts : 7
Join date : 2016-07-29

View user profile

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by Admin on Sun Jul 31, 2016 2:36 am

LAV Video decoder supports outputting in a large number of colorspaces (see its options). It does conversion internally when needed. Outputting RGB32 should work with any input file.

Admin
Admin

Posts : 2982
Join date : 2011-06-17

View user profile http://codecs.forumotion.net

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by DavidB3 on Sun Jul 31, 2016 3:13 am

OK Smile
Thank you for helping me with this problem.

An offtopic question: from an application how to test if K-Lite is installed (maybe even if it is the latest version)? I ask because I want to implement in my application if it can't render a video file should verify if K-Lite is installed and advice the user to install it (or update it if it's possible to implement).

DavidB3

Posts : 7
Join date : 2016-07-29

View user profile

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by Admin on Mon Aug 01, 2016 2:16 am

You can check for this registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\KLiteCodecPack_is1
on 64-bit system:
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\KLiteCodecPack_is1

Or check for presence of LAV:
HKEY_CLASSES_ROOT\CLSID\{083863F1-70DE-11D0-BD40-00A0C911CE86}\Instance\{171252A0-8820-4AFE-9DF8-5C92B2D66B04}
HKEY_CLASSES_ROOT\CLSID\{083863F1-70DE-11D0-BD40-00A0C911CE86}\Instance\{EE30215D-164F-4A92-A4EB-9D4C13390F9F}

Admin
Admin

Posts : 2982
Join date : 2011-06-17

View user profile http://codecs.forumotion.net

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by DavidB3 on Mon Aug 01, 2016 2:58 am

Thank you again Smile

DavidB3

Posts : 7
Join date : 2016-07-29

View user profile

Back to top Go down

Re: problem extracting frames from 480p Youtube videos using DirectX Samplegrabber

Post by Sponsored content Today at 11:48 am


Sponsored content


Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum