Recently, GTK gained not one, but two new renderers: one for GL and one for Vulkan.
Since naming is hard, we reused existing names and called them “ngl” and “vulkan”. They are built from the same sources, therefore we also call them “unified” renderers.
But what is exciting about them?
A single source
As mentioned already, the two renderers are built from the same source. It is modeled to follow Vulkan apis, with some abstractions to cover the differences between Vulkan and GL (more specifically, GL 3.3+ and GLES 3.0+). This lets us share much of the infrastructure for walking the scene graph, maintaining transforms and other state, caching textures and glyphs, and will make it easier to keep both renderers up-to-date and on-par.
Could this unified approach be extended further, to cover a Metal-based renderer on macOS or a DirectX-based one on Windows? Possibly. The advantage of the Vulkan/GL combination is that they share basically the same shader language (GLSL, with some variations). That isn’t the case for Metal or DirectX. For those platforms, we either need to duplicate the shaders or use a translation tool like SPIRV-Cross.
If that is the kind of thing that excites you, help is welcome.
The old GL renderer uses simple shaders for each rendernode type and frequently resorts to offscreen rendering for more complex content. The unified renderers have (more capable) per-node shaders too, but instead of relying on offscreens, they will also use a complex shader that interprets data from a buffer. In game programming, this approach is known as a ubershader.
The unified renderer implementation is less optimized than the old GL renderer, and has been written with a focus on correctness and maintainability. As a consequence, it can handle much more varied rendernode trees correctly.
We wouldn’t have done all this work, if there wasn’t some tangible benefit. Of course, there’s new features and capabilities. Lets look at some:
Antialiasing. A big problem with the old GL renderer is that it will just lose fine details. If something is small enough to fall between the boundaries of a single line of pixels, it will simply disappear. In particular this can affect underlines, such as mnemonics. The unified renderers handle such cases better, by doing antialiasing. This helps not just for preserving fine detail, but also prevents jagged outlines of primitives.
Fractional scaling. Antialiasing is also the basis that lets us handle fractional scales properly. If your 1200 × 800 window is set to be scaled to 125 %, with the unified renderers, we will use a framebuffer of size 1500 × 1000 for it, instead of letting the compositor downscale a 2400 × 1600 image. Much less pixels, and a sharper image.
Arbitrary gradients. The old GL renderer handles linear, radial and conic gradients with up to 6 color stops. The unified renders allow an unlimited number of color stops. The new renderers also apply antialiasing to gradients, so sharp edges will have smooth lines.
Dmabufs. As a brief detour from the new renderers, we worked on dmabuf support and graphics offloading last fall. The new renderers support this and extend it to create dmabufs when asked to produce a texture via the render_texture api (currently, just the Vulkan renderer).
Any sharp edges?
As is often the case, with new capabilities comes the potential for new gotchas. Here are some things to be aware of, as an app developer:
No more glshader nodes. Yes, they made for some fancy demos for 4.0, but they are very much tied to the old GL renderer, since they make assumptions about the GLSL api exposed by that renderer. Therefore, the new renderers don’t support them.
You have been warned in the docs:
If there is a problem, this function returns FALSE and reports an error. You should use this function before relying on the shader for rendering and use a fallback with a simpler shader or without shaders if it fails.
Thankfully, many uses of the glshader node are no longer necessary, since GTK has gained new features since 4.0, such as mask nodes and support for straight-alpha textures.
Fractional positions. The old GL renderer is rounding things, so you could get away with handing it fractional positions. The new renderers will place things where you tell it. This can sometimes have unintended consequences, so should be on the lookout and make sure that your positions are where they should be.
In particular, look out for out for cairo-style drawing where you place lines at half-pixel positions so they fill out one row of pixels precisely.
Driver problems. The new renderers are using graphics drivers in new and different ways, so there is potential for triggering problems on that side.
Please file problems you see against GTK even if they look like driver issues, since it is useful for us to get an overview how well (or badly) the new code works with the variety of drivers and hardware out there.
But is it faster?
No, the new renderers are not faster (yet).
The old GL renderer is heavily optimized for speed. It also uses much simpler shaders, and does not do the math that is needed for features such as antialiasing. We want to make the new renderers faster eventually, but the new features and correctness make them very exciting, even before we reach that goal. All of the GPU-based renderers are more than fast enough to render todays GTK apps at 60 or 144 fps.
That being said, the Vulkan renderer comes close to matching and surpassing the old GL renderer in some unscientific benchmarks. The new GL renderer is slower for some reason that we have not tracked down yet.
In the just-released 4.13.6 snapshot, we have made the ngl renderer the new default. This is a trial balloon — the renderers need wider testing with different apps too verify that they are ready for production. If significant problems appear, we can revert back to the gl renderer for 4.14.
We decided not make the Vulkan renderer the default yet, since it is behind the GL renderers in a few application integration aspects: the webkit GTK4 port works with GL, not with Vulkan, and GtkGLArea and GtkMediaStream currently both produce GL textures that the Vulkan renderer can’t directly import. All of these issues will hopefully be addressed in the not-too-distant future, and then we will revisit the default renderer decision.
If you are using GTK on very old hardware, you may be better off with the old GL renderer, since it makes fewer demands on the GPU. You can override the renderer selection using the GSK_RENDERER environment variable:
Future plans and possibilities
The new renderers are a good foundation to implement things that we’ve wanted to have for a long time, such as
Proper color handling (including HDR)
Path rendering on the GPU
Possibly including glyph rendering
Performance (on old and less powerful devices)
Some of these will be a focus of our work in the near and medium-term future.
The new renderers have some exciting features, with more to come.
Please try them out, and let us know what works and what doesn’t work for you.
Some of us in the GTK team have spent the last month or so exploring the world of linux kernel graphics apis, in particular, dmabufs. We are coming back from this adventure with some frustrations and some successes.
What is a dmabuf?
A dmabuf is a memory buffer in kernel space that is identified by a file descriptor. The idea is that you don’t have to copy lots of pixel data around, and instead just pass a file descriptor between kernel subsystems.
Reality is of course more complicated that this rosy picture: the memory may be device memory that is not accessible in the same way as ‘plain’ memory, and there may be more than one buffer (and more than one file descriptor), since graphics data is often split into planes (e.g. RGB and A may be separate, or Y and UV).
Why are dmabufs useful?
I’ve already mentioned that we hope to avoid copying the pixel data and feeding it through the GTK compositing pipeline (and with 4k video, that can be quite a bit of data for each frame).
The use cases where this kind of optimization matters are those where frequently changing content is displayed for a long time, such as
In the best case, we may be able to avoid feeding the data through the compositing pipeline of the compositor as well, if the compositor supports direct scanout and the dmabuf is suitable for it. In particular on mobile systems, this may avoid using the GPU altogether, thereby reducing power consumption.
GTK has already been using dmabufs since 4.0: When composing a frame, GTK translates all the render nodes (typically several for each widget) into GL commands, sends those to the GPU, and mesa then exports the resulting texture as a dmabuf and attaches it to our Wayland surface.
But if the only thing that is changing in your UI is the video content that is already in a dmabuf, it would be nice to avoid the detour through GL and just hand the data directly to the compositor, by giving it the file descriptor for the the dmabuf.
Wayland has the concept of subsurfaces that let applications defer some of their compositing needs to the compositor: The application attaches a buffer to each (sub)surface, and it is the job of the compositor to combine them all together.
With what is now in git main, GTK will create subsurfaces as-needed in order to pass dmabufs directly to the compositor. We can do this in two different ways: If nothing is drawn on top of the dmabuf (no rounded corners, or overlaid controls), then we can stack the subsurface above the main surface without changing any of the visuals.
This is the ideal case, since it enables the compositor to set up directscanout, which gives us a zero-copy path from the video decoder to the display.
If there is content that gets drawn on top of the video, we may not be able to get that, but we can still get the benefit of letting the compositor do the compositing, by placing the subsurface with the video below the main surface and poking a translucent hole in the main surface to let it peek through.
The round play button is what forces the subsurface to be placed below the main surface here.
GTK picks these modes automatically and transparently for each frame, without the application developer having to do anything. Once that play button appears in a frame, we place the subsurface below, and once the video is clipped by rounded corners, we stop offloading altogether. Of course, the advantages of offloading also disappear.
The graphics offload visualization in the GTK inspector shows these changes as they happen:
Initially, the camera stream is not offloaded because the rounded corners clip it. The magenta outline indicates that the stream is offloaded to a subsurface below the main surface (because the video controls are on top of it). The golden outline indicates that the subsurface is above the main surface.
How do you use this?
GTK 4.14 will introduce a GtkGraphicsOffload widget, whose only job it is to give a hint that GTK should try to offload the content of its child widget by attaching it to a subsurface instead of letting GSK process it like it usually does.
To create suitable content for offloading, the new GdkDmabufTextureBuilder wraps dmabufs in GdkTexture objects. Typical sources for dmabufs are pipewire, video4linux or gstreamer. The dmabuf support in gstreamer will be much more solid in the upcoming 1.24 release.
When testing this code, we used the GtkMediaStream implementation for pipewire by Georges Basile Stavracas Neto that can be found in pipewire-media-stream and libmks by Christian Hergert and Bilal Elmoussaoui.
What are the limitations?
At the moment, graphics offload will only work with Wayland on Linux. There is some hope that we may be able to implement similar things on MacOS, but for now, this is Wayland-only. It also depends on the content being in dmabufs.
Applications that want to take advantage of this need to play along and avoid doing things that interfere with the use of subsurfaces, such as rounding the corners of the video content. The GtkGraphicsOffload docs have more details for developers on constraints and how to debug problems with graphics offload.
The GTK 4.14 release will have some interesting new capabilities for media playback. You can try it now, with the just-released 4.13.3 snapshot.
Please try it and let us know what does and doesn’t work for you.
In the first part of this series, we introduced the concept of paths and looked at how to create a GskPath. But there’s more to paths than that.
Many interesting properties of paths can change as you move along the trajectory of the path. To query such properties, we first need a way to pin down the point on the path that we are interested in.
GTK has the GskPathPoint struct for this purpose, and provides a number of functions to obtain them, such as gsk_path_get_closest_point(), which lets you find the point on the path that is closest to a given point.
Once you have a GskPathPoint, you can query the properties of the path at that point. The most basic property is the position, but you can also get the tangent, the curvature, or the distance from the beginning of the path.
Another interesting question when using paths in a user interface is:
Is the mouse pointer hovering over the path?
You need the answer to this question if you want to highlight a path that the pointer is over, or if you want to react to the user clicking a path.
For a stroked path, it is much more complicated to provide a 100% accurate answer (in particular, if the stroke is using a dash pattern), but we can provide an approximate answer that is often good enough: a point is inside the stroke, if the distance to the closest point on the path is less than half the line width.
The next part of this series will look at rendering with paths.
It is no secret that we want to get rid of cairo as the drawing API in GTK, so we can move more of our drawing onto the GPU.
While People have found creative ways to draw things with render nodes, they don’t provide a comprehensive drawing API like Skia or, yes, cairo. Not a very satisfying state of affairs.
A few years ago, we started to investigate how to change this, by making paths available as first-class objects in GTK. This effort is finally starting to come to fruition, and you can see the first results in GTK 4.13.0.
So, what is a path? A rough definition could be:
A sequence of line segments or curves that may or may not be connected at their endpoints.
When we say curves, we specifically mean quadratic or cubic Bézier curves. On top of cairo, we also support rational quadratic Béziers (or as Skia calls them: conics), since they let us model circles and rounded rectangles precisely.
This picture shows a typical path, consisting of 4 curves and 2 lines, some of which are connected. As you can see, paths can be closed (like the 4 curves here) or open (like the 2 lines), with a start- and endpoint.
And how are paths useful for drawing? First, you can use a path to define an area (the part that’s inside the path) and fill it with a color, a gradient or some more complex content.
Alternatively, you can stroke the path with various properties such as line width, color or dash pattern.
Paths in GTK
The object that we use for paths in GTK is GskPath. It is a compact, immutable representation that is optimized for rendering. To create a GskPath, you need to use a GskPathBuilder, which has many convenience methods to create paths, either from individual curves or from predefined shapes.
This example creates a path that is a closed triangle:
that let apps set accessible attributes of their widgets.
ARIA layer improvements
Since we’ve introduced it in GTK4, the application API layer has only seen moderate changes. That has started to change over the last 9 months or so, since Lukáš is working on it.
One thing that he has started to do is adding public interfaces so that third-party widgets (such as libadwaita) can provide full accessibility support. The first such interface is GtkAccessibleRange (new in 4.10), for range widgets like GtkScale. We are looking at adding more, notably a text interface. It will be needed to make terminals accessible.
In the 4.12 cycle, we have done some work to make our implementation match the ARIA specs more closely. This involved changing the roles of some widgets: our default role is now ‘generic’, and toplevel windows use the ‘application’ role. We’ve also redone the way accessible names and descriptions (i.e. the things you hear orca read) are computed, to match the spec.
Another improvement is that most of our widgets now have the necessary labels and relations to make orca read them. If you find that there are still things missing, please let us know!
AT-SPI translation improvements
It is no secret that we would like to see some modernization of the AT-SPI D-Bus APIs. But for now, it is what we have to work with. Our translation layer works by lazily putting just the objects on the bus that ATs have asked for, to avoid creating too much bus traffic.
One of the recent improvements in our translation is that we are now using GtkAccessibleRange, so third-party range widgets can be accessible.
We have also fixed problems with the selection implementations for GtkNotebook and GtkStackSwitcher, so ATs can now change the selected tab in notebooks and stacks.
All of this is nice to hear, but if you are an app developer, you may want to know how you can find and fix accessibility problems in your app.
It is very instructive to just turn on the Screen Reader and see what it says as you navigate through your app. But we also have some tools to help you evaluate the accessibility support of your app.
The GTK inspector has a page that shows accessibility information:
It recently was improved to show not just the properties, states and relations that are set on each widgets, but also the name and description that GTK computes and passes to ATs - that is the text that orca reads.
Another tool in the inspector is brand new: the accessibility overlay shows warnings and recommendations that are based on the ARIA authoring guidelines.
It looks like this:
It is not perfect, but it should give some quick insights on where you can improve accessibility.
GTK 4.12 will have better out-of-the-box accessibility and new tools to help you make your app accessible.
Here is the first GTK snapshot of the new development cycle. A lot of things fell into place recently, so it is worth taking some time to go through the details of what is new, and what you can expect to see in 4.12.
List View Improvements
The family of GtkListView, GtkColumnView and GtkGridView widgets was one of the big additions in GTK 4. They are meant to replace GtkTreeView, but up until now, this was clearly still a bit aspirational.
In GTK 4.10, we’ve finally taken the big step to port GtkFileChooser away from tree views—a sign that list views are ready for prime time. And the next GTK 4 release will bring a number of missing features:
Finally, a fix for the longstanding scrolling bug
Better keyboard navigation, with customizable tab behavior
Some of these are already available in 4.11.1. We even managed to backport the scrolling fix to 4.10.1.
Textures are used frequently in GTKs GL renderer—for icons and images, for glyphs, and for intermediate offscreen rendering. Most of the time, we don’t have to think about them, they just work. But if the texture is the main content of your app, such as in an image viewer, you need a bit more control over it, and it is important that the corner cases work correctly.
In GTK 4.10, we introduced a GskTextureScale node, which gives applications control over the filtering that is applied when scaling a texture up or down. This lets apps request the use of mipmaps with GSK_SCALING_FILTER_TRILINEAR. GTK 4.12 will automatically use mipmaps when it is beneficial.
One corner case that we’ve recently explored is texture slicing. Whenever a texture is bigger than the GL stack supports, GSK will break it into smaller slices and use separate GL textures for each. Modern GPUs support enormous textures (on my system, the max. texture size is 16384), which means that the slicing support is rarely tested in practice and not well covered by our unit tests either.
We added support for artificially limiting the texture size (with the GSK_MAX_TEXTURE_SIZE environment variable), and promptly discovered that our texture slicing support needed some love. It will work much better in 4.12.
It landed on April 1st, but it is not a joke.
We’ve added support for the experimental wp_fractional_scale_manager_v1 protocol to the Wayland backend, and use the wp_viewporter protocol to tell the compositor about the scaling that the buffer is using. It is nice that this was easy to fit into our rendering stack, but don’t expect miracles. It works well with the cairo renderer (as you can see in the video), but we still consider it experimental with the GL and Vulkan renderers.
To try fractional scaling with the GL renderer, set
in the environment.
There’s lots of new things to explore in GTK 4.11. Please try them and let us know what you think, in gitlab or on Discourse.
Some of the core GTK developers recently got together for a few days to do some focused work and talk about current and future plans.
It is difficult to summarize three days of discussions in a blog post, but here are some of the highlights.
The GtkIconTheme code has been with us for a long time. It implements the icon theme spec, and comes from an era when we were shipping big sets of icons with the desktop and themes were expected to switch them out. That is not really how icons are made or used today.
We need a better solution for the workflow from a designer making icons a set of icons in a sheet to the developer copying individual icons into their app.
Inside GTK, this will need some form of “asset manager” to maintain the mapping from icon name to image / file / resource.
While we can’t get away from providing a C interface with gobject-introspection metadata for all our language bindings, it could be nice to use a more expressive language and a more powerful compiler than C has to offer.
Of course we can’t and won’t rewrite all of GTK in a different language. It would be nice to experiment with replacing smaller parts. Allowing new code to be written in different languages would also potentially bring in new contributors.
We discussed the scroll speed question, and decided to write up an explanatory comment in the issue about what we consider the right solution:
treat wheel and touchpad scrolling separately
inject configuration from the control-center/compositor into libinput
gtk gets it via events
The other big input question we talked about is ‘asynchronous event handling’ and its problems. The two main cases where this comes up are webkit, with its ui<>web process communication, and IBus. In both cases, we think that there is no actual interest in reinjecting unhandled events into the GTK capture/bubble propagation. Instead, such leftover event should just be handled ‘locally’ (in the IBus case, adding/removing characters to the entry, or moving the cursor).
With GTK4, we’ve made the intentional change to move away from having everything in GTK itself, and instead introduced the idea of ‘platform libraries’ like libadwaita to carry the more platform-specific widgetry.
Overall, we are happy with how this has turned out, and we would like to continue with this approach. There is maybe room for moving some things that are more plumbing than widgetry back into GTK itself.
We need to open a .90 branch to do things that would break the APIs that we have deprecated now (like the file chooser, and more general the chooser dialog/widget split). Some of us have been itching to start on that work. But there’s also still a lot of work to be done in 4.x (GtkListView fixes, for example).
With an eye towards the color management work that is planned to land in 4.12, the suggestion is to open a 4.90 development branch after 4.12. That would put it towards the end of this year, and 3 years after the 4.0 release, which seems reasonable.
On the last day, we had the pleasure of hosting both the documentation and the tracker teams at our place.
Three hackfests in a room!
We’d like to thank the GNOME foundation for supporting our meeting. ❤️
In the last post, we discussed deprecating treeviews and cell renderers, among other things. All these deprecations cause a lot of work for applications and libraries using these APIs, so why are we doing this?
One of the reasons is to enable new features. Such as a grid view for the file chooser. It only took us 18 years! You can see the original feature request in Bugzilla. This is easily possible now because GtkListView and GtkGridView can use the same data models.
Here is the file chooser, with a new view toggle:
And here is the grid view itself: Judging from the number of likes on the merge request, this is a popular feature. We hope you enjoy it. ❤️
If you want to support this work, please consider donating to the GNOME Foundation, which supports GTK development. You can do so by going here.
If you are paying attention to GTK’s git repository, you may have noticed a change in the last weeks.
We have a directory gtk/deprecations, which is destined to contain source files that implement deprecated APIs and will be dropped in the next major release. For the 4.0 release, we emptied it out, and it has been empty ever since. But recently, it started to accumulate files again.
This is a good opportunity to remind folks how we are using deprecations in GTK. But first, lets take a look at the details.
The details, part 1: cell renderers
In GTK 4, we introduced a new family of list and grid widgets that are based around list models: GtkListView, GtkColumnView, GtkGridView. There is also a new combo box implementation using list models, called GtkDropDown. Taken together, these are meant to provide replacements for everything you can do with cell renderers in GTK 3.
The ultimate goal was to remove cell renderers, since they are a whole separate rendering and layout system that tends to interfere with GTK’s CSS and layout machinery, and makes everything more complicated.
But we did not quite get to the finish line for 4.0, mainly because we still had significant uses of treeviews in GTK itself. First and foremost, the file chooser. Since the filechooser is getting ported to use a GtkColumnView in 4.10, now is the right time to deprecate the cell renderer machinery and all the widgets that use them.
This is a significant amount of code, more than 75.000 lines.
The details, part 2: dialogs
In GTK 4, we dropped gtk_main() and gtk_dialog_run(), since recursive mainloops are best avoided. Again, we did not get to the finish line and could not remove GtkDialog itself, since it is used as the base class for all our complex dialogs.
GTK 4.10 introduces replacement APIs for our ‘Chooser’ dialogs. The new APIs follow the gio async pattern. Here is an example:
With these replacements in place, we could deprecate the Chooser interfaces, their widget implementations, and their base class GtkDialog.
No need to panic
Deprecations in GTK are an early outlook at changes that will appear in the next major release that is breaking API compatibility. But the eventual GTK 5 release is still far away. We have not even made a plan for it yet.
There is absolutely no need to rush towards ‘deprecation cleanup’. You only need to remove all uses of deprecations when you want to port to GTK 5 – which does not exist yet.
There are still things you can do, though. We are introducing deprecations in 4.10 as a way to give our users time to adapt, and to provide feedback on our ideas. If you want to do so, you can file an issue in gitlab, start a discussion in discourse, or find us on matrix.
In the meantime…
Deprecation warnings can be annoying, but thankfully there are easy ways to turn them off. For the occasional call to a deprecated function, it is best to just wrap it in G_GNUC_BEGIN/END_IGNORE_DEPRECATIONS: