Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
GSTREAMER-VAAPI
HARDWARE-ACCELERATED
ENCODING AND DECODING
ON INTEL® HARDWARE
Victor Jaquez
GStreamer Conference 2015 / 8-...
VA-API
VIDEO ACCELERATION
—
APPLICATION PROGRAMMING INTERFACE
WHAT IS VA-API?
An API specification
A library implementation
Open Source MIT license
WHAT DOES IT DO?
1. Enables hardware accelerated video decode and encode.
Entry-points: VLD, IDCT, Motion Compensation, et...
HOW'S THE IMPLEMENTATION?
libva
http://cgit.freedesktop.org/vaapi/libva/
It is a front-end
Opens and registers a backend
WHICH BACKENDS?
i965_drv_video.so (HD Intel driver)
vdpau_drv_video.so (VDPAU —noveau/nvidia/s3g— bridge)
xvba_drv_video.s...
vainfo
libva info: VA-API version 0.38.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /opt/gnome/jh...
vainfo
libva info: VA-API version 0.38.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /home/ceyusa/...
HOW'S THE API?
VADisplay
X11, DRM, Wayland, Android, etc.
VAConfigID
VLD for requested codec.
VAContexID
"Virtual" video p...
MPEG2 DECODE
INITIALIZATION
dpy = vaGetDisplay(x11_display);
vaCreateConfig(dpy, VAProfileMPEG2Main, VAEntrypointVLD,
&att...
MPEG2 DECODE
FILL DATA
vaCreateBuffer(dpy, ctxt, VAPictureParameterBufferType,
sizeof(VAPictureParameterBufferMPEG2), 1,
&...
MPEG2 DECODE
DECODE AND DISPLAY
vaBeginPicture(dpy, ctxt, surface);
vaRenderPicture(dpy, ctxt, &pic_param_buf, 1);
vaRende...
WHAT IS GSTREAMER-VAAPI?
A helper library (libgstvaapi)
A set of GStreamer elements
Supports from GStreamer-1.2 to 1.6
LIBRARY
Wraps almost all VA-API concepts
ENCODER H264
INITIALIZATION
dpy = gst_vaapi_display_x11_new(NULL);
enc = gst_vaapi_encoder_h264_new(dpy);
st = new_gst_vid...
ENCODER H264
FILL DATA
img = gst_vaapi_image_new(dpy, GST_VIDEO_FORMAT_I420, w, h);
gst_vaapi_image_map(img);
load_raw_ima...
ENCODER H264
ENCODE
frame = g_slice_new0(GstVideoCodecFrame);
gst_video_codec_frame_set_user_data(frame,
gst_vaapi_surface...
LIBRARY
Implements video codec parsers.
OpenGL helpers (EGL and GLX).
Windowing protocol helpers (DRM, Wayland, X11).
SPLIT LIBRARIES
libgstvaapi
libgstvaapi-drm
libgstvaapi-x11
libgstvaapi-glx
libgstvaapi-egl
libgstvaapi-wayland
GStreamer elements
vaapidecode: VA-API decoder
vaapisink: VA-API sink
 
vaapipostproc: VA-API video post-processing
vaapid...
GStreamer elements
Encoders
vaapiencode_h264: VA-API H.264 encoder
vaapiencode_mpeg2: VA-API MPEG-2 encoder
vaapiencode_jp...
GStreamer elements
Parsers
vaapiparse_h264: H.264 parser
vaapiparse_h265: H.265 parser
CHALLENGES
THOU SHALL NOT PARSE TWICE
— codecparsers: add GstMetas to pass parsing
results downstream
— Add mpeg2 slice header inform...
THOU SHALL AUTO-PLUG HW VIDEO
FILTERS (DE-INTERLACERS)
— playbin: autoplugging s/w and h/w accelerated
deinterlacers
Remov...
THOU SHALL REVERSE PLAY-BACK
— videodecoder: reverse playback in non-
packetized decoders
Handle when the number of buffer...
THOU SHALL SUPPORT DMABUF
— vaapi: expose memory:dmabuf capsfeature755072
THOU SHALL SUPPORT OPENGL /
OPENGL3 / OPENGL-ES
— [metabug] GL bugs755406
THOU SHALL NOT CACHE THE
DISPLAY
— Remove display cache
— Using multiple instances of vaapisink in one
application cause p...
QUESTIONS?
 
THANK YOU!
Twitter:
Mail: vjaquez at igalia dot com
Blog:
@ceyusa
http://blogs.igalia.com/vjaquez
Nächste SlideShare
Wird geladen in …5
×

GStreamer-VAAPI: Hardware-accelerated encoding and decoding on Intel hardware (GStreamer Conference 2015)

6.859 Aufrufe

Veröffentlicht am

By Víctor M. Jáquez.

Slides at https://github.com/01org/gstreamer-vaapi/tree/master/docs/slides/gstconf2015

GStreamer-VAAPI is a set of GStreamer elements (vaapidecode, vaapipostroc, vaapisink, and several encoders) and libgstvapi, a library that wraps libva under a GObject/GStreamer semantics.

This talk will be about VAAPI and its integration with GStreamer. We will show a general overview of VAAPI architecture, the role of libgstvaapi, and finally, the design of GStreamer elements. Afterwards we will show what is ahead in the development of GStreamer-VAAPI, and the current problems and challenges.

Veröffentlicht in: Technologie
  • Dating direct: ♥♥♥ http://bit.ly/36cXjBY ♥♥♥
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • Great! Actually I use it and I'm going to get a VP8 encoding GPU. But I think I would read the autor's blog, because being just a slideshow, this presentation does not contain too much details. I did not understand whether the "challenges" presented were solved or they still exist. I think that there is another challenge - for the HW encoder quality - PSNR of HW encoded video vs PSNR of software (x264, VP8) encoded video with settings tuned for quality and low bitrate - say 0.09 .. 0.14 bits per pixel per frame
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

GStreamer-VAAPI: Hardware-accelerated encoding and decoding on Intel hardware (GStreamer Conference 2015)

  1. 1. GSTREAMER-VAAPI HARDWARE-ACCELERATED ENCODING AND DECODING ON INTEL® HARDWARE Victor Jaquez GStreamer Conference 2015 / 8-9 October (Dublin)
  2. 2. VA-API VIDEO ACCELERATION — APPLICATION PROGRAMMING INTERFACE
  3. 3. WHAT IS VA-API? An API specification A library implementation Open Source MIT license
  4. 4. WHAT DOES IT DO? 1. Enables hardware accelerated video decode and encode. Entry-points: VLD, IDCT, Motion Compensation, etc. Codecs: MPEG-2, MPEG-4 ASP/H.263, MPEG-4 AVC/H.264, VC-1/VMW3, JPEG, VP8, VP9. 2. Sub-picture blending and rendering 3. Video post processing: Color balance, skin tone enhancement, de-interlace, scaling, etc.
  5. 5. HOW'S THE IMPLEMENTATION? libva http://cgit.freedesktop.org/vaapi/libva/ It is a front-end Opens and registers a backend
  6. 6. WHICH BACKENDS? i965_drv_video.so (HD Intel driver) vdpau_drv_video.so (VDPAU —noveau/nvidia/s3g— bridge) xvba_drv_video.so (XvBA —fglrx— bridge) pvr_drv_video.so (PVR bridge) gallium_drv_video.so (Gallium bridge!) hybrid_drv_video.so (another HD Intel driver, does decoding and encoding using either CPU and GPU)
  7. 7. vainfo libva info: VA-API version 0.38.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /opt/gnome/jh/lib/dri/i965_drv_video.so libva info: Found init function __vaDriverInit_0_38 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.38 (libva 1.6.2.pre1) vainfo: Driver version: Intel i965 driver for Intel(R) Haswell Mobile - vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Simple : VAEntrypointEncSlice VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointEncSlice VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice
  8. 8. vainfo libva info: VA-API version 0.38.0 libva info: va_getDriverName() returns 0 libva info: Trying to open /home/ceyusa/jh/lib/dri//nvidia_drv_video.so libva info: Found init function __vaDriverInit_0_38 libva info: va_openDriver() returns 0 vainfo: VA-API version: 0.38 (libva 1.6.2.pre1) vainfo: Driver version: Splitted-Desktop Systems VDPAU backend for VA-API - vainfo: Supported profile and entrypoints VAProfileMPEG2Simple : VAEntrypointVLD VAProfileMPEG2Main : VAEntrypointVLD VAProfileMPEG4Simple : VAEntrypointVLD VAProfileMPEG4AdvancedSimple : VAEntrypointVLD VAProfileH264Baseline : VAEntrypointVLD VAProfileH264Main : VAEntrypointVLD VAProfileH264High : VAEntrypointVLD VAProfileVC1Simple : VAEntrypointVLD
  9. 9. HOW'S THE API? VADisplay X11, DRM, Wayland, Android, etc. VAConfigID VLD for requested codec. VAContexID "Virtual" video processing pipeline. VASurfaceID Render targets. Not accessible to the client. VABufferID data, parameters, quantization matrix, slice info, etc.
  10. 10. MPEG2 DECODE INITIALIZATION dpy = vaGetDisplay(x11_display); vaCreateConfig(dpy, VAProfileMPEG2Main, VAEntrypointVLD, &attr, 1, &cfg); vaCreateSurfaces(dpy, VA_RT_FORMAT_YUV420, w, h, &surface, 1, NULL, 0); vaCreateContext(dpy, cfg, w, h, VA_PROGRESSIVE, &surface, 1, &ctxt);
  11. 11. MPEG2 DECODE FILL DATA vaCreateBuffer(dpy, ctxt, VAPictureParameterBufferType, sizeof(VAPictureParameterBufferMPEG2), 1, &pic_param, &pic_param_buf); vaCreateBuffer(dpy, ctxt, VAIQMatrixBufferType, sizeof(VAIQMatrixBufferMPEG2), 1, &iq_matrix, &iqmatrix_buf); vaCreateBuffer(dpy, ctxt, VASliceParameterBufferType, sizeof(VASliceParameterBufferMPEG2), 1, &slice_param, &slice_param_buf); vaCreateBuffer(dpy, ctxt, VASliceDataBufferType, slice_size, 1, slice_data, &slice_data_buf);
  12. 12. MPEG2 DECODE DECODE AND DISPLAY vaBeginPicture(dpy, ctxt, surface); vaRenderPicture(dpy, ctxt, &pic_param_buf, 1); vaRenderPicture(dpy, ctxt, &iqmatrix_buf, 1); vaRenderPicture(dpy, ctxt, &slice_param_buf, 1); vaRenderPicture(dpy, ctxt, &slice_data_buf, 1); vaEndPicture(dpy, ctxt); vaSyncSurface(dpy, surface); vaPutSurface(dpy, surface, x11_window, sx, sy, sw, sh, dx, dy, dw, dh, NULL,
  13. 13. WHAT IS GSTREAMER-VAAPI? A helper library (libgstvaapi) A set of GStreamer elements Supports from GStreamer-1.2 to 1.6
  14. 14. LIBRARY Wraps almost all VA-API concepts
  15. 15. ENCODER H264 INITIALIZATION dpy = gst_vaapi_display_x11_new(NULL); enc = gst_vaapi_encoder_h264_new(dpy); st = new_gst_video_codec_state(w, h, fps_n, fps_d); gst_vaapi_encoder_set_codec_data(enc, st);
  16. 16. ENCODER H264 FILL DATA img = gst_vaapi_image_new(dpy, GST_VIDEO_FORMAT_I420, w, h); gst_vaapi_image_map(img); load_raw_image(img); gst_vaapi_image_unmap(img); gst_video_info_set_format(&vinfo, GST_VIDEO_FORMAT_ENCODED, w, h); pool = gst_vaapi_surface_pool_new_full(dpy, &vinfo, 0); proxy = gst_vaapi_surface_proxy_new_from_pool (pool); surface = gst_vaapi_surface_proxy_get_surface (proxy); gst_vaapi_surface_put_image(surface, img);
  17. 17. ENCODER H264 ENCODE frame = g_slice_new0(GstVideoCodecFrame); gst_video_codec_frame_set_user_data(frame, gst_vaapi_surface_proxy_ref(proxy), gst_vaapi_surface_proxy_unref); gst_vaapi_encoder_put_frame(encoder, frame); gst_vaapi_encoder_get_buffer_with_timeout(enc, &encbuf, 5000); buf = gst_buffer_new_and_alloc(gst_vaapi_coded_buffer_get_size(encbuf)) gst_vaapi_coded_buffer_copy_into(buf, encbuf);
  18. 18. LIBRARY Implements video codec parsers. OpenGL helpers (EGL and GLX). Windowing protocol helpers (DRM, Wayland, X11).
  19. 19. SPLIT LIBRARIES libgstvaapi libgstvaapi-drm libgstvaapi-x11 libgstvaapi-glx libgstvaapi-egl libgstvaapi-wayland
  20. 20. GStreamer elements vaapidecode: VA-API decoder vaapisink: VA-API sink   vaapipostproc: VA-API video post-processing vaapidecodebin: VA-API decode bin
  21. 21. GStreamer elements Encoders vaapiencode_h264: VA-API H.264 encoder vaapiencode_mpeg2: VA-API MPEG-2 encoder vaapiencode_jpeg: VA-API JPEG encoder vaapiencode_vp8: VA-API VP8 encoder vaapiencode_h265: VA-API H.265 encoder
  22. 22. GStreamer elements Parsers vaapiparse_h264: H.264 parser vaapiparse_h265: H.265 parser
  23. 23. CHALLENGES
  24. 24. THOU SHALL NOT PARSE TWICE — codecparsers: add GstMetas to pass parsing results downstream — Add mpeg2 slice header information to GstMpegVideoMeta 691712 704865
  25. 25. THOU SHALL AUTO-PLUG HW VIDEO FILTERS (DE-INTERLACERS) — playbin: autoplugging s/w and h/w accelerated deinterlacers Remove vaapidecodebin 687182
  26. 26. THOU SHALL REVERSE PLAY-BACK — videodecoder: reverse playback in non- packetized decoders Handle when the number of buffers in the GOP is bigger than the maximum buffers in the video buffer pool. 747574
  27. 27. THOU SHALL SUPPORT DMABUF — vaapi: expose memory:dmabuf capsfeature755072
  28. 28. THOU SHALL SUPPORT OPENGL / OPENGL3 / OPENGL-ES — [metabug] GL bugs755406
  29. 29. THOU SHALL NOT CACHE THE DISPLAY — Remove display cache — Using multiple instances of vaapisink in one application cause problems 747946 754820
  30. 30. QUESTIONS?
  31. 31.   THANK YOU! Twitter: Mail: vjaquez at igalia dot com Blog: @ceyusa http://blogs.igalia.com/vjaquez

×