Decode h264 video

I开发者_高级运维 am looking for a way to decode h264 (or indeed any video format) using c#. The ultimate goal is to be able to decode the images and very strictly control the playback in real time. The project I am working on is a non-linear video art piece where the HD footage is required to loop and edit itself on the fly, playing back certain frame ranges and then jumping to the next randomly selected frame range seamlessly.

I have created an app which reads image files (jpegs) in from the disk and plays them on screen in order, I have total control over which frame is loaded and when it is displayed but at full HD res it takes slightly longer than I want to load the images from hard drive (which are about 500k each), I am thinking that using a compressed video format would be smaller and therefore faster to read and decode into a particular frame however I cannot find any readily available way to do this.

Are there any libraries which can do this? i.e. extract an arbitrary frame from a video file and serve it to my app in less time than it takes to show the frame (running at 25fps), I have looked into the vlc libraries and wrappers for ffmpeg but I don't know which would be better or if there would be another even better option. Also I don't know which codec would be the best choice as some are key frame based making arbitrary frame extraction probably very difficult.

Any advice welcome, thanks

---

I'd say that using H.264 is not such a good idea. The reason for this is the different kind of frames that make up the stream

• I-frames or key-frames: everything needed to decode a frame is available directly, i.e. no dependencies exists to other frames.

• P-frames (predicted-frames): consists of the difference data towards frames previously decoded.

• B-frames (bidirectional): consists of the difference data towards frames previously decoded and frames that lies in the future.

The order the frames are encoded in the bistream is not the same as the order that the frames should be displayed in.

As an extreme example, H.264 content can have only 1 I-frame at the start of the clip. If you need to display the last frame, every intermediate frame needs to be decoded up to and including the last frame in order for it to be displayed.

Using JPEG as you stared to do is not a bad idea. You could play around with the compression level to find the ultimate file-size with respect to quality and decode-time.

The output from a video-decoder is almost in all cases 4:2:0 subsampled YUV uncompressed raw-data. One 1080p frame will be 1920*1080*1.5=3110400 bytes. Using this format instead of JPEG (which also decodes into YUV4:2:0) will shave off the decoding time from your "application", leaving just the view-time. Imagemagick and a bunch of other tools can convert from JPEG to YUV4:2:0. This cant be combined with the memory-mapping described in comments.

If you feel that the raw format takes to much diskspace, have a look at huffyuv which is a lossless YUV-codec.

For a viewer, I had great experience in the past using SDL which understands the YUV-format making it very simple to write a viewer. Luckily there is already one written that you can use as a template for further development. Have a look at yay

---

I just want to point out that file size doesn't equate to speed always. The smaller the file is sometimes the higher the decompression cost. You might (and I emphasize might) actually get better perf by going with an uncompressed format such as BMP or possibly WMF.