We first introduced support for dmabufs and graphics offload last fall, and it is included in GTK 4.14. Since we last talked about, more improvements have happened, so it is time for another update.
Transformations
When you rotate your monitor, it changes its aspect from landscape (say, 1920 x 1200 to portrait (1200 x 1920). But the framebuffer that the compositor deals with stays 1920 x 1200.
There are some other situations where the content may need some transformation before it can be displayed. For example when showing the video stream from a front-facing camera.
So far, GTK has treated flipped or rotated textures as not offloadable, but after some recent improvements, we will now pass such transformations along to the compositor when offloading content.
Quick detour
This is a good opportunity to explain that the GTK transformation API relies on you to provide classification into useful categories such as 2D translations, scales, rotations or arbitrary 3D transforms.
Therefore, it is much better to use specific methods such as gsk_transform_rotate or gsk_transform_translate instead of computing the transformation matrix yourself and using gsk_transform_matrix, which does not give us any information about the kind of transformation we are dealing with. And without this information, we miss out on optimizations like the transformed offloading described above.
Benjamin gave a quick overview of our approach to transformations here.
Textures
The initial goal for the offload work was to enable passthrough of dmabuf content. Often, that is not what we get though. It is still far more common that we receive content in GL textures (e.g. from gstreamer). Thankfully, mesa has GL extensions that let us export textures as dmabufs. We’ve recently used this to enable offloading for GL textures.
Note that these extensions might not be available everywhere, and individual drivers can have problems with dmabuf support.
Summary
In the coming GTK 4.16 release, graphics offloading should work much more often.
You can try these improvements today, e.g. in the nightly build of the new Showtime app:
