GTK Development Blog

We just had a GTK hackfest at FOSDEM. A good time for an update on whats new and exciting in GTK, with an eye towards 4.18.

Requirements

You can no longer call gdk_display_get_default() or gdk_display_open() before gtk_init(). This was causing problems due to incomplete initialization, so we made it fail with a (hopefully clear) error message. If you are affected by this, the usual fix is to just call gtk_init() as early as possible.

On Windows, we have a hard requirement on Windows 10 now. All older versions are long unsupported, and having to deal with a maze of ifdefs and unavailable APIs makes development harder than it should be. Dropping support for very old versions also simplifies the code down the stack, in Pango and GLib.

The same idea applies to macOS, where we now require macOS 10.15.

Spring cleaning

The old GL renderer has been removed. This may be unwelcome news for people stuck on very old drivers and hardware. But we will continue to make the new renderers work as well as possible on the hardware that they can support.

The X11 and Broadway backends have been deprecated, as a clear signal that we intend to remove them in the GTK 5. In the meantime, they continue to be available. We have also deprecated GtkShortcutsWindow, since it needs a new design. The replacement will appear in libadwaita, hopefully next cycle.

It is worth reminding everybody that there is no need to act on deprecations until you are actively porting your app to the next major version of GTK, which is not on the horizon yet.

Incremental improvements

Widget layout and size allocation has received quite a bit of attention this cycle, with the goal of improving performance (by avoiding binary search as much as possible) and correctness. Nevertheless, these changes have some potential for breakage, so if you see wrong or suboptimal layouts in applications, please let us know.

GTK has had difficulties for a while getting its pointer sizes right with fractional scaling on Wayland, but this should all be solved in GTK 4.18. No more huge pointers. Fixing this also required changes on the mutter side.

New beginnings

Accessibility in GTK 4.18 is taking a major step forward, with the new AccessKit backend, which gives us accessibility on  Windows and macOS, for the very first time. The at-spi backend is still the default on Linux, and has seen a number of improvements as well.

And, maybe the biggest news: We have an Android backend now. It is still experimental, so you should expect some rough edges and loose ends. For example, there is no GL renderer support yet. But it is exciting that you can just try gtk4-demo on your phone now, and have it mostly work.

Enjoy!

Everything is better in color. Even better if it is HDR.

In this post, we’ll provide an overview of what is happening with color in GTK, without diving too deeply into the weeds of colorimetry and color science.

The high-level goals of this effort are to enable proper handling of HDR content and color-managed workflows.

In the beginning, sRGB

Up until now, colors in GTK were always assumed to be in the sRGB color space. The omnipresent GdkRGBA struct is defined as specifying an sRGB color.

sRGB was a great thing 20 years ago, but there world is moving on to other color spaces that include a wider range of hues (such as Display-P3) or a bigger dynamic range (such as BT.2100-PQ).

After having this on our agenda for quite a while (initial investigations into this topic happened in 2021), we’re finally moving ahead with landing the foundations for a more colorful future.

Coming soon: GdkColorState

Earlier this year, we’ve merged some great work by Alice to support modern color syntax and color spaces in our CSS engine. Having expressive colors in CSS is great, but it can’t really shine if all our rendering machinery still requires colors to be specified in sRGB.

So over the past month or two, we’ve added support for doing our rendering in well-defined color spaces, and introduced an object representing these. It is called GdkColorState (don’t ask me why the name changed from space to state).

As of now, GdkColorState can represent sRGB and BT.2100-PQ, as well as their linearized variants. You cannot yet do much with these objects, they are mainly used internally by our renderers.

One thing you can already do though, is trying out how rendering in a linear color space will look, by setting the

GDK_DEBUG=linear

environment variable.

This is a change that we need to do eventually, to produce correct and understandable results, in particular when working with HDR content.

Doing all our compositing in a linear color state does look subtly different though, so want to delay this change until all of the surrounding work is done, and do it at beginning of a development cycle to give everybody time to adjust.

New protocols

The Wayland color management protocol has been a long time in the making, but it is hopefully close to leaving the experimental phase — kwin already has support for it, and there is a mutter branch as well.

We have added support for the xx-color-management-v4 protocol, so we get the preferred color state from the compositor (if it is supports that protocol), and we can tell it about the color state of the frames that we produce.

It is unlikely that your compositor will prefer an HDR color state like BT.2100-PQ today (unless you find the hidden switch to turn on experimental HDR support), but you can try how things look when GTK is rendering in that color state by setting the

GDK_DEBUG=hdr

environment variable.

Note that this doesn’t make HDR content appear on your screen — we do our rendering in HDR and translate the final frame back to  sRGB as the last step. Making HDR content appear on your screen requires a compositor that accepts such content.

In the future: More color states and linear compositing

We still have a long todo list to fully develop color support in GTK.

The highlights include

• More color states (including OKLCH for better gradients and YUV for video content)
• Color state aware rendering api (GdkColor, and new GtkSnapshot APIs)
• Passing CSS color state information down to the renderer
• Propagating color state information from gstreamer (for HDR, among other things)
• Switching to linear compositing

A few of these will hopefully make it in time for the GTK 4.16 release.

Summary

Better color support is coming soon to GTK.

We first introduced support for dmabufs and  graphics offload last fall, and it is included in GTK 4.14. Since we last talked about, more improvements have happened, so it is time for another update.

Transformations

When you rotate your monitor, it changes its aspect from landscape (say, 1920 x 1200 to portrait (1200 x 1920). But the framebuffer that the compositor deals with stays 1920 x 1200.

There are some other situations where the content may need some transformation before it can be displayed. For example when showing the video stream from a front-facing camera.

So far, GTK has treated flipped or rotated textures as not offloadable, but after some recent improvements, we will now pass such transformations along to the compositor when offloading content.

Quick detour

This is a good opportunity to explain that the GTK transformation API relies on you to provide classification into useful categories such as 2D translations, scales, rotations or arbitrary 3D transforms.

Therefore, it is much better to use specific methods such as gsk_transform_rotate or gsk_transform_translate instead of computing the transformation matrix yourself and using  gsk_transform_matrix, which does not give us any information about the kind of transformation we are dealing with. And without this information, we miss out on optimizations like the transformed offloading described above.

Benjamin gave a quick overview of our approach to transformations here.

Textures

The initial goal for the offload work was to enable passthrough of dmabuf content. Often, that is not what we get though. It is still far more common that we receive content in GL textures (e.g. from gstreamer). Thankfully, mesa has GL extensions that let us export textures as dmabufs. We’ve recently used this to enable offloading for GL textures.

Note that these extensions might not be available everywhere, and individual drivers can have problems with dmabuf support.

Summary

In the coming GTK 4.16 release, graphics offloading should work much more often.

You can try these improvements today, e.g. in the nightly build of the new Showtime app:

We first introduced support for dmabufs and graphics offload last fall, and it is included in GTK 4.14. Since then, some improvements have happened, so it is time for an update.

Improvements down the stack

The GStreamer 1.24 release has improved support for explicit modifiers, and the GStreamer media backend in GTK has been updated to request dmabufs from GStreamer.

Another thing that happens on the GStreamer side is that dmabufs sometimes come with padding: in that case GStreamer will give us a buffer with a viewport and expect us to only show that part of the buffer. This is sometimes necessary to accommodate stride and size requirements of hardware decoders.

GTK 4.14 supports this when offloading, and only shows the part of the dmabuf indicated by the viewport.

Improvements inside GTK

We’ve merged new GSK renderers for GTK 4.14. The new renderers support dmabufs in the same way as the old gl renderer. In addition, the new Vulkan renderer produces dmabufs when rendering to a texture.

In GTK 4.16, the GtkGLArea widget will also provide dmabuf textures if it can, so you can put it in a GtkGraphicsOffload widget to send its output directly to the compositor.

You can see this in action in the shadertoy demo in gtk4-demo in git main.

Improved compositor interaction

One nice thing about graphics offload is that the compositor may be able to pass the dmabuf to the KMS apis of the kernel without any extra copies or compositing. This is known as direct scanout and it helps reduce power consumption since large parts of the GPU aren’t used.

The compositor can only do this if the dmabuf is attached to a fullscreen surface and has the right dimensions to cover it fully. If it does not cover it fully, the compositor needs some assurance that it is ok to leave the outside parts black.

One way for clients to provide that assurance is to attach a specially constructed black buffer to a surface below the one that has the dmabuf attached. GSK will do this now if it finds black color node in the rendernode tree, and the GtkGraphicsOffload widget will put that color there if you set the “black-background” property. This should greatly increase the chances that you can enjoy the benefits of direct scanout when playing fullscreen video.

In implementing this for GTK 4.16, we found some issues with mutter’s support for single-pixel buffers, but these have been fixed quickly.

To see graphics offload and direct scanout in action in a GTK4 video player, you can try the Light Video Player.

If you want to find out if graphics offload works on your system or debug why it doesn’t, this recent post by Benjamin is very helpful.

Summary

GTK 4 continues to improve for efficient video playback and drives improvements in this area up and down the stack.

A big thank you for pushing all of this forward goes to Robert Mader. ❤️

GTK 4.14 brings various improvements on the accessibility front, especially for applications showing complex, formatted text; for WebKitGTK; and for notifications.

Accessible text interface

The accessibility rewrite for 4.0 provided an implementation for complex, selectable, and formatted text in widgets provided by GTK, like GtkTextView, but out of tree widgets would not be able to do the same, as the API was kept private while we discussed what ATs (assistive technologies) actually needed, and while we were looking at non-Linux implementations. For GTK 4.14 we finally have a public interface that out of tree widgets can implement to provide complex, formatted text to ATs: GtkAccessibleText.

GtkAccessibleText allows widgets to provide the text contents at given offsets; the text attributes applied to the contents; and to notify assistive technologies of changes in the text, caret position, or selection boundaries.

Text widgets implementing GtkAccessibleText should notify ATs in these cases:

• • if the text widget has a caret cursor, it needs to call gtk_accessible_text_update_caret_position() every time the caret moves
  • if the text widget has a selection, it needs to call gtk_accessible_text_update_selection_bound() every time the selection changes
  • when the text changes, the widget needs to call gtk_accessible_text_update_contents() with the description of what changed, and the boundaries of the change

Text attributes are mainly left to applications to implement—both in naming and serialization; GTK provides support for common text attributes already in use by various toolkits and assistive technologies, and they are available as constants under the GTK_ACCESSIBLE_ATTRIBUTE_* prefix in the API reference.

The GtkAccessibleText interface is a requirement for implementing the accessibility of virtual terminals; the most common GTK-based library for virtual terminals, VTE, has been ported to GTK4 thanks to the efforts of Christian Hergert and in GNOME 46 will support accessibility through the new GTK interface.

Bridging AT-SPI trees

There are cases when a library or an application implements its own accessible tree using AT-SPI, whether in the same process or out of process. One such library is WebKitGTK, which generates the accessible object tree from the web tree inside separate processes. These processes do not use GTK, so they cannot use the GtkAccessible API to describe their contents.

Thanks to the work of Georges Stavracas GTK now can bridge those accessibility object trees under the GTK widget’s own, allowing ATs to navigate into a web page using WebKit from the UI.

Currently, like the rest of the accessibility API in GTK, this is specific to the AT-SPI protocol on Linux, which means it requires libraries and applications that wish to take advantage of it to ensure that the API is available at compile time, through the use of a pkg-config file and a separate C header, similarly to how the printing API is exposed.

Notifications

Applications using in-app notifications that are decoupled by the current widget’s focus, like AdwToast in libadwaita, can now raise the notification message to ATs via the gtk_accessible_announce() method, thanks to Lukáš Tyrychtr, in a way that is respectful of the current ATs output.

Other improvements

GTK 4.12 ensured that the computed accessible labels and descriptions were up to date with the ARIA specification; GTK 4.14 iterates on those improvements, by removing special cases and duplicates.

Thanks to the work of Michael Weghorn from The Document Foundation, there are new roles for text-related accessible objects, like paragraphs and comments, as well as various fixes in the AT-SPI implementation of the accessibility API.

The accessibility support in GTK4 is incrementally improving with every cycle, thanks to the contributions of many people; ideally, these improvements should also lead to a better, more efficient protocol for toolkits and assistive technologies to share.

We are still exploring the possibility of adding backends for other accessibility platforms, like UIAutomation; and for other libraries, like AccessKit.

GTK 4.14 will be released very soon, with new renderers that were introduced earlier this year.

The new renderers have much improved support for fractional scaling—on my system, I now use 125% scaling instead of the ‘Large Text’ setting, and I find that works fine for my needs.

Magical numbers

Ever since 4.0, GTK has been advocating for linear layout.

The idea is that we just place glyphs where the coordinates tell us, and if that is a fractional position somewhere between pixels, so be it, we can render the outline at that offset just fine. This approach works—if your output device has a high-enough resolution (anything above 240 dpi should be ok). Sadly, we don’t live in a world where most laptop screens have that kind of resolution, so we can’t just ignore pixels.

Consequently, we added the gtk-hint-font-metrics setting that forces text layout to round things to integer positions. This is not a great fit for fractional scaling, since the rounding happens in application pixels, and we really need integral device pixel positions to produce crisp results.

The common fractional scales are 125%, 150%, 175%, 200% and 225%. At these scales (with the exception of 200%), most application pixel boundaries do not align with device pixel boundaries.

What now?

The new renderers gave us an opportunity to revisit the topic of font rendering and do some research on the mechanics of hinting options, and how they get passed down the stack from GTK through Pango and cairo, and then end up in freetype as a combination of render target + load flags.

The new renders recognize that there’s two basic modes of operation when it comes to glyphs:

• optimize for uniform spacing
• optimize for crisp rendering

The former leads to subpixel positioning and unhinted rendering, the latter to hinted rendering and glyphs that are placed at integral pixel positions (since that is what the autohinter expects).

We determine which case we’re in by looking at the font options. If they tell us to do hinting, we round the glyph position to an integral device pixel in the y direction. Why only y? The autohinter only applies hinting in the vertical direction and the horizontal direction is where the increased resolution of subpixel positions helps most. If we are not hinting, then we use subpixel positions for both x and y, just like the old renderer (with the notable difference that the new renderer uses subpixel positions in device pixels).

A comparison

Text rendering differences are always subtle and, to some degree, a matter a taste and preference. So these screenshots should be taken with a grain of salt—it is much better to try the new renderers for yourself.

Both of these renderings were done at a scale of 125%, with hinting enabled (but note that the old renderer handles 125% by rendering at 200% and relying on the compositor to scale things down).

Here is a look at some details: the horizontal bars of T and e are consistent across lines, even though we still allow the glyphs to shift by subpixel positions horizontally.

Summary

The new renderers in GTK 4.14 should produce more crisp font rendering, in particular with fractional scaling.

Please try it out and tell us what you think.

Update: On subpixel rendering

I should have anticipated that this question would come up, so here is a quick answer:

We are not using subpixel rendering (aka Cleartype, or rgb antialiasing) in GTK 4, since our compositing does not have component alpha. Our antialiasing for fonts is always grayscale. Note that subixel rendering is something separate from subpixel positioning.

As we often do, a few members of the GTK team and the wider GNOME community came together for a two-day hackfest before FOSDEM.

This year, we were aiming to make progress on the topics of accessibility and input. Here is a quick summary of what we’ve achieved.

Accessibility

• We agreed to merge the socket implementation for webkit accessiblity that Georges wrote
• We agreed that the accessible notification api that Lukáš suggested is fine
• We finished the GtkAccessibleText interface and moved our internal implementations over to that interface
• We discussed the possibility of an a11y backend based on AccessKit

Input

• Carlos reviewed the merge request for passing unhandled events back to the system (on macOS)
• We looked over the remnants of X11-style timestamps in our input apis and decided to provide alternatives taking an event

Wayland

• Carlos started to turn the private gtk-shell protocol into separate protocols

Thanks to the GNOME foundation for supporting this event. ❤️

Recently, GTK gained not one, but two new renderers: one for GL and one for Vulkan.

Since naming is hard, we reused existing names and called them “ngl” and “vulkan”. They are built from the same sources, therefore we also call them “unified” renderers.

But what is exciting about them?

A single source

As mentioned already, the two renderers are built from the same source. It is modeled to follow Vulkan apis, with some abstractions to cover the differences between Vulkan and GL (more specifically, GL 3.3+ and GLES 3.0+). This lets us share much of the infrastructure for walking the scene graph, maintaining transforms and other state, caching textures and glyphs, and will make it easier to keep both renderers up-to-date and on-par.

Could this unified approach be extended further, to cover a Metal-based renderer on macOS or a DirectX-based one on Windows? Possibly. The advantage of the Vulkan/GL combination is that they share basically the same shader language (GLSL, with some variations). That isn’t the case for Metal or DirectX. For those platforms, we either need to duplicate the shaders or use a translation tool like SPIRV-Cross.

If that is the kind of thing that excites you, help is welcome.

Implementation details

The old GL renderer uses simple shaders for each rendernode type and frequently resorts to offscreen rendering for more complex content. The unified renderers have (more capable) per-node shaders too, but instead of relying on offscreens, they will also use a complex shader that interprets data from a buffer. In game programming, this approach is known as a ubershader.

The unified renderer implementation is less optimized than the old GL renderer, and has been written with a focus on correctness and maintainability. As a consequence, it can handle much more varied rendernode trees correctly.

Here is an harmless-looking example:

repeat {
  bounds: 0 0 50 50;
  child: border {
    outline: 0 0 4.3 4.3;
    widths: 1.3;
  }
}

New capabilities

We wouldn’t have done all this work, if there wasn’t some tangible benefit. Of course, there’s new features and capabilities. Lets look at some:

Antialiasing. A big problem with the old GL renderer is that it will just lose fine details. If something is small enough to fall between the boundaries of a single line of pixels, it will simply disappear. In particular this can affect  underlines, such as mnemonics. The unified renderers handle such cases better, by doing antialiasing. This helps not just for preserving fine detail, but also prevents jagged outlines of primitives.

Fractional scaling. Antialiasing is also the basis that lets us handle fractional scales properly. If your  1200 × 800 window is set to be scaled to 125 %, with the unified renderers, we will use a framebuffer of size 1500 × 1000 for it, instead of letting the compositor downscale a 2400 × 1600 image. Much less pixels, and a sharper image.

Arbitrary gradients. The old GL renderer handles linear, radial and conic gradients with up to 6 color stops. The unified renders allow an unlimited number of color stops. The new renderers also apply antialiasing to gradients, so sharp edges will have smooth lines.

Dmabufs. As a brief detour from the new renderers, we worked on dmabuf support and graphics offloading last fall. The new renderers support this and extend it to create dmabufs when asked to produce a texture via the render_texture api (currently, just the Vulkan renderer).

Any sharp edges?

As is often the case, with new capabilities comes the potential for new gotchas. Here are some things to be aware of, as an app developer:

No more glshader nodes. Yes, they made for some fancy demos for 4.0, but they are very much tied to the old GL renderer, since they make assumptions about the GLSL api exposed by that renderer. Therefore, the new renderers don’t support them.

You have been warned in the docs:

If there is a problem, this function returns FALSE and reports an error. You should use this function before relying on the shader for rendering and use a fallback with a simpler shader or without shaders if it fails.

Thankfully, many uses of the glshader node are no longer necessary, since GTK has gained new features since 4.0, such as mask nodes and support for straight-alpha textures.

Fractional positions. The old GL renderer is rounding things, so you could get away with handing it fractional positions. The new renderers will place things where you tell it. This can sometimes have unintended consequences, so should be on the lookout and make sure that your positions are where they should be.

In particular, look out for out for cairo-style drawing where you place lines at half-pixel positions so they fill out one row of pixels precisely.

Driver problems. The new renderers are using graphics drivers in new and different ways, so there is potential for triggering problems on that side.

Please file problems you see against GTK even if they look like driver issues, since it is useful for us to get an overview how well (or badly) the new code works with the variety of drivers and hardware out there.

But is it faster?

No, the new renderers are not faster (yet).

The old GL renderer is heavily optimized for speed. It also uses much simpler shaders, and does not do the math that is needed for features such as antialiasing. We want to make the new renderers faster eventually, but the new features and correctness make them very exciting, even before we reach that goal. All of the GPU-based renderers are more than fast enough to render todays GTK apps at 60 or 144 fps.

That being said, the Vulkan renderer comes close to matching and surpassing the old GL renderer in some unscientific benchmarks. The new GL renderer is slower for some reason that we have not tracked down yet.

New defaults

In the just-released 4.13.6 snapshot, we have made the ngl renderer the new default. This is a trial balloon — the renderers need wider testing with different apps too verify that they are ready for production. If significant problems appear, we can revert back to the gl renderer for 4.14.

We decided not make the Vulkan renderer the default yet, since it is behind the GL renderers in a few application integration aspects: the webkit GTK4 port works with GL, not with Vulkan, and GtkGLArea and GtkMediaStream currently both produce GL textures that the Vulkan renderer can’t directly import. All of these issues will hopefully be addressed in the not-too-distant future, and then we will revisit the default renderer decision.

If you are using GTK on very old hardware, you may be better off with the old GL renderer, since it makes fewer demands on the GPU. You can override the renderer selection using the GSK_RENDERER environment variable:

GSK_RENDERER=gl

Future plans and possibilities

The new renderers are a good foundation to implement things that we’ve wanted to have for a long time, such as

• Proper color handling (including HDR)
• Path rendering on the GPU
• Possibly including glyph rendering
• Off-the-main-thread rendering
• Performance (on old and less powerful devices)

Some of these will be a focus of our work in the near and medium-term future.

Summary

The new renderers have some exciting features, with more to come.

Please try them out, and let us know what works and what doesn’t work for you.

Some of us in the GTK team have spent the last month or so exploring the world of linux kernel graphics apis, in particular, dmabufs. We are coming back from this adventure with some frustrations and some successes.

What is a dmabuf?

A dmabuf is a memory buffer in kernel space that is identified by a file descriptor. The idea is that you don’t have to copy lots of pixel data around, and instead just pass a file descriptor between kernel subsystems.

Reality is of course more complicated that this rosy picture: the memory may be device memory that is not accessible in the same way as ‘plain’ memory, and there may be more than one buffer (and more than one file descriptor), since graphics data is often split into planes (e.g. RGB and A may be separate, or Y and UV).

Why are dmabufs useful?

I’ve already mentioned that we hope to avoid copying the pixel data and feeding it through the GTK compositing pipeline (and with 4k video, that can be quite a bit of data for each frame).

The use cases where this kind of optimization matters are those where frequently changing content is displayed for a long time, such as

• Video players
• Virtual machines
• Streaming
• Screencasting
• Games

In the best case, we may be able to avoid feeding the data through the compositing pipeline of the compositor as well, if the compositor supports direct scanout and the dmabuf is suitable for it.  In particular on mobile systems, this may avoid using the GPU altogether, thereby reducing power consumption.

Details

GTK has already been using dmabufs since 4.0: When composing a frame, GTK translates all the render nodes (typically several for each widget) into GL commands, sends those to the GPU, and mesa then exports the resulting texture as a dmabuf and attaches it to our Wayland surface.

But if the only thing that is changing in your UI is the video content that is already in a dmabuf, it would be nice to avoid the detour through GL and just hand the data directly to the compositor, by giving it the file descriptor for the the dmabuf.

Wayland has the concept of subsurfaces that let applications defer some of their compositing needs to the compositor: The application attaches a buffer to each (sub)surface, and it is the job of the compositor to combine them all together.

With what is now in git main, GTK will create subsurfaces as-needed in order to pass dmabufs directly to the compositor. We can do this in two different ways: If nothing is drawn on top of the dmabuf (no rounded corners, or overlaid controls), then we can stack the subsurface above the main surface without changing any of the visuals.

This is the ideal case, since it enables the compositor to set up direct scanout, which gives us a zero-copy path from the video decoder to the display.

If there is content that gets drawn on top of the video, we may not be able to get that, but we can still get the benefit of letting the compositor do the compositing, by placing the subsurface with the video below the main surface and poking a translucent hole in the main surface to let it peek through.

The round play button is what forces the subsurface to be placed below the main surface here.

GTK picks these modes automatically and transparently for each frame, without the application developer having to do anything. Once that play button appears in a frame, we place the subsurface below, and once the video is clipped by rounded corners, we stop offloading altogether. Of course, the advantages of offloading also disappear.

The graphics offload visualization in the GTK inspector shows these changes as they happen:

Initially, the camera stream is not offloaded because the rounded corners clip it. The magenta outline indicates that the stream is offloaded to a subsurface below the main surface (because the video controls are on top of it). The golden outline indicates that the subsurface is above the main surface.

How do you use this?

GTK 4.14 will introduce a GtkGraphicsOffload widget, whose only job it is to give a hint that GTK should try to offload the content of its child widget by attaching it to a subsurface instead of letting GSK process it like it usually does.

To create suitable content for offloading, the new GdkDmabufTextureBuilder wraps dmabufs in GdkTexture objects. Typical sources for dmabufs are pipewire, video4linux or gstreamer. The dmabuf support in gstreamer will be much more solid in the upcoming 1.24 release.

When testing this code, we used the GtkMediaStream implementation for pipewire by Georges Basile Stavracas Neto that can be found in pipewire-media-stream and libmks by Christian Hergert and Bilal Elmoussaoui.

What are the limitations?

At the moment, graphics offload will only work with Wayland on Linux. There is some hope that we may be able to implement similar things on MacOS, but for now, this is Wayland-only. It also depends on the content being in dmabufs.

Applications that want to take advantage of this need to play along and avoid doing things that interfere with the use of subsurfaces, such as rounding the corners of the video content. The GtkGraphicsOffload docs have more details for developers on constraints and how to debug problems with graphics offload.

Summary

The GTK 4.14 release will have some interesting new capabilities for media playback. You can try it now, with the just-released 4.13.3 snapshot.

Please try it and let us know what does and doesn’t work for you.

In the first part of this series, we introduced the concept of paths and looked at how to create a GskPath. But there’s more to paths than that.

Path Points

Many interesting properties of paths can change as you move along the trajectory of the path. To query such properties, we first need a way to pin down the point on the path that we are interested in.

GTK has the GskPathPoint struct for this purpose, and provides a number of functions to obtain them, such as gsk_path_get_closest_point(), which lets you find the point on the path that is closest to a given point.

Once you have a GskPathPoint, you can query the properties of the path at that point. The most basic property is the position, but you can also get the tangent, the curvature, or the distance from the beginning of the path.

Input

Another interesting question when using paths in a user interface is:

Is the mouse pointer hovering over the path?

You need the answer to this question if you want to highlight a path that the pointer is over, or if you want to react to the user clicking a path.

For a filled path, GTK provides the answer with the gsk_path_in_fill() method.

For a stroked path, it is much more complicated to provide a 100% accurate answer (in particular, if the stroke is using a dash pattern), but we can provide an approximate answer that is often good enough: a point is inside the stroke, if the distance to the closest point on the path is less than half the line width.

Outlook

The next part of this series will look at rendering with paths.