We first introduced support for dmabufs and graphics offload last fall, and it is included in GTK 4.14. Since then, some improvements have happened, so it is time for an update.
Improvements down the stack
The GStreamer 1.24 release has improved support for explicit modifiers, and the GStreamer media backend in GTK has been updated to request dmabufs from GStreamer.
Another thing that happens on the GStreamer side is that dmabufs sometimes come with padding: in that case GStreamer will give us a buffer with a viewport and expect us to only show that part of the buffer. This is sometimes necessary to accommodate stride and size requirements of hardware decoders.
GTK 4.14 supports this when offloading, and only shows the part of the dmabuf indicated by the viewport.
Improvements inside GTK
We’ve merged new GSK renderers for GTK 4.14. The new renderers support dmabufs in the same way as the old gl renderer. In addition, the new Vulkan renderer produces dmabufs when rendering to a texture.
In GTK 4.16, the GtkGLArea widget will also provide dmabuf textures if it can, so you can put it in a GtkGraphicsOffload widget to send its output directly to the compositor.
You can see this in action in the shadertoy demo in gtk4-demo in git main.
Improved compositor interaction
One nice thing about graphics offload is that the compositor may be able to pass the dmabuf to the KMS apis of the kernel without any extra copies or compositing. This is known as direct scanout and it helps reduce power consumption since large parts of the GPU aren’t used.
The compositor can only do this if the dmabuf is attached to a fullscreen surface and has the right dimensions to cover it fully. If it does not cover it fully, the compositor needs some assurance that it is ok to leave the outside parts black.
One way for clients to provide that assurance is to attach a specially constructed black buffer to a surface below the one that has the dmabuf attached. GSK will do this now if it finds black color node in the rendernode tree, and the GtkGraphicsOffload widget will put that color there if you set the “black-background” property. This should greatly increase the chances that you can enjoy the benefits of direct scanout when playing fullscreen video.
In implementing this for GTK 4.16, we found some issues with mutter’s support for single-pixel buffers, but these have been fixed quickly.
To see graphics offload and direct scanout in action in a GTK4 video player, you can try the Light Video Player.
If you want to find out if graphics offload works on your system or debug why it doesn’t, this recent post by Benjamin is very helpful.
Summary
GTK 4 continues to improve for efficient video playback and drives improvements in this area up and down the stack.
A big thank you for pushing all of this forward goes to Robert Mader. ❤️