Custom widgets in GTK 4 – Drawing

(This is the second part of a series about custom widgets in GTK 4. Part 1).

Drawing the old-fashioned way

Before looking at how widgets do their own drawing, it is worth pointing out that GtkDrawingArea is still a valid option if all you need is some self-contained cairo drawing.

The only difference between GTK 3 and GTK 4 is that you call gtk_drawing_area_set_draw_func() to provide your drawing function instead of connecting a signal handler to the the ::draw signal. Everything else is the same: GTK provides you with a cairo context, and you can just draw to it.

void
draw_func (GtkDrawingArea *da,
           cairo_t        *cr,
           int             width,
           int             height,
           gpointer        data)
{
  GdkRGBA red, green, yellow, blue;
  double w, h;

  w = width / 2.0;
  h = height / 2.0;

  gdk_rgba_parse (&red, "red");
  gdk_rgba_parse (&green, "green");
  gdk_rgba_parse (&yellow, "yellow");
  gdk_rgba_parse (&blue, "blue");

  gdk_cairo_set_source_rgba (cr, &red);
  cairo_rectangle (cr, 0, 0, w, h);
  cairo_fill (cr);

  gdk_cairo_set_source_rgba (cr, &green);
  cairo_rectangle (cr, w, 0, w, h);
  cairo_fill (cr);

  gdk_cairo_set_source_rgba (cr, &yellow);
  cairo_rectangle (cr, 0, h, w, h);
  cairo_fill (cr);

  gdk_cairo_set_source_rgba (cr, &blue);
  cairo_rectangle (cr, w, h, w, h);
  cairo_fill (cr);
}

...

gtk_drawing_area_set_draw_func (area, draw, NULL, NULL);

The rendering model

One of the major differences between GTK 3 and GTK 4 is that we are now targeting GL / Vulkan instead of cairo. As part of this switch, we have moved from an immediate mode rendering model to a retained mode one. In GTK 3, we were using cairo commands to render onto a surface. In GTK 4, we create a scene graph that contains render nodes, and those render nodes can be passed to renderer, or processed in some other way, or saved to a file.

In the widget API, this change is reflected in the difference between

gboolean (* draw) (GtkWidget *widget, cairo_t *cr)

and

 void (* snapshot) (GtkWidget *widget, GtkSnapshot *snapshot)

GtkSnapshot is an auxiliary object that turns your drawing commands into render nodes and adds them to the scene graph.

The CSS style information for a widget describes how to render its background, border, and so on. GTK translates this a series of function calls that add suitable render nodes to the scene graph, before and after the render nodes for the widgets’ content. So your widget automatically complies with the CSS drawing model, without any extra work.

Providing the render nodes for the content is the reponsibility of the widgets snapshot() implementation. GtkSnapshot has convenience API to make it easy.  For example, use gtk_snapshot_append_texture() to render a texture. Use gtk_snapshot_append_layout() to render text. If you want to use custom cairo drawing, gtk_snapshot_append_cairo() lets you do so.

A drawing widget

To implement a  widget that does some custom drawing, you need to implement the snapshot() function that creates the render nodes for your drawing:

void
demo_snapshot (GtkWidget *widget, GtkSnapshot *snapshot)
{
  GdkRGBA red, green, yellow, blue;
  float w, h;

  gdk_rgba_parse (&red, "red");
  gdk_rgba_parse (&green, "green");
  gdk_rgba_parse (&yellow, "yellow");
  gdk_rgba_parse (&blue, "blue");

  w = gtk_widget_get_width (widget) / 2.0;
  h = gtk_widget_get_height (widget) / 2.0;

  gtk_snapshot_append_color (snapshot, &red,
                             &GRAPHENE_RECT_INIT(0, 0, w, h));
  gtk_snapshot_append_color (snapshot, &green,
                             &GRAPHENE_RECT_INIT(w, 0, w, h));
  gtk_snapshot_append_color (snapshot, &yellow,
                             &GRAPHENE_RECT_INIT(0, h, w, h));
  gtk_snapshot_append_color (snapshot, &blue,
                             &GRAPHENE_RECT_INIT(w, h, w, h));
}

...

widget_class->snapshot = demo_snapshot;

This example produces four color nodes:

If your drawing needs a certain size, you should implement the measure() function too:

void
demo_measure (GtkWidget      *widget,
              GtkOrientation  orientation,
              int             for_size,
              int            *minimum_size,
              int            *natural_size,
              int            *minimum_baseline,
              int            *natural_baseline)
{
  *minimum_size = 100;
  *natural_size = 200;
}

...

widget_class->measure = demo_measure;

GTK keeps the render nodes produced by your snapshot() function and reuses them until you tell it that your widget needs to be drawn again by calling gdk_widget_queue_draw().

Going deeper

The GTK documentation has an overview of the GTK drawing model, if you are interested in reading more about this topic.

Outlook

In the next post, we’ll look how widgets in GTK 4 handle child widgets.

Custom widgets in GTK 4 – Introduction

With GTK 4 getting closer to completion, now is a good time to provide an overview of how custom widgets will look in GTK 4.

This series of posts will look at the major aspects of writing a widget, and how they changed compared to GTK 3. The articles will provide a high-level overview; for a detailed checklist for porting an application to GTK 4, look at the  migration guide.

Introduction

The general direction of our API changes has been to emphasize delegation over subclassing. One of the motivations for this is to make writing your own widgets easier and less error-prone. As a consequence, you will see a lot more auxiliary objects that take over aspects of functionality from core widget classes. And many widgets are now final classes – deriving directly from GtkWidget is expected.

Another general trend in our API is that “everything is a widget.” In GTK 3, we slowly broke up complex widgets into their constituent parts, first with CSS nodes and then with gadgets. In GTK 4, for example the trough and the slider of a GtkScale are fully formed sub-widgets which can maintain their own state and receive input like any other widget.

A big loser in the GTK 4 transition is the GtkContainer base class. It has become much less important. Any widget can have child widgets now. Child properties have been replaced by layout children and their properties. And all focus handling has been moved from GtkContainer to GtkWidget.

Another big loser is GtkWindow. In GTK 3, all the “popups”  (entry completion, menus, tooltips, etc) were using a GtkWindow underneath. In GTK 4, most of them have been converted to popovers, and the GtkPopover implementation has been untangled from GtkWindow. In addition, many pieces of toplevel-specific functionality have been broken out in separate interfaces called GtkRoot and GtkNative.

Outlook

In the next post, we’ll look how widgets in GTK 4 do their own drawing with render nodes.

GTK 3.98.2

When we released 3.98.0, we promised more frequent snapshots, as the remaining GTK 4 features are landing. Here we are a few weeks later, and 3.98.1 and 3.98.2 snapshots have quietly made it out.

So, what is new ?

Features

There is still work left to do, but a few more big features have landed.

The first is that we have completed the reimplementation of GtkPopovers as xdg-popup surfaces, and split up the GdkSurface API into separate GdkToplevel and GdkPopup interfaces (there’s a GdkDragSurface interface too), which reflect the different roles of surfaces:

  • Toplevels are sovereign windows that are placed by the user and can be maximized, fullscreened, etc.
  • Popups are positioned relative to a parent surface and often grab input, e.g. when used for menus.

In GTK, popovers have lost their :relative-to property, since they are now part of the regular hierarchy like any other widget, and GtkWindow has lost its :window-type property, since all instances of GTK_WINDOW_POPUP have been converted to popovers, and windows are just used for proper toplevels.

Another major feature is the new infrastructure for keyboard shortcuts. In the past, GTK has had a plethora of APIs to implement key bindings, mnemonics and accelerators. In GTK 4, all of this is handled by event controllers. GtkShortcutController is a bit more complex than typical event controllers, since it handles all the different kinds of shortcuts with a unified API.

Thankfully, most of the complexity is hidden. For widget implementors, the important APIs are the variants of gtk_widget_class_add_shortcut(), which are used to add key bindings. For applications, mnemonics and global accels (with gtk_application_set_accels_for_action()) work the same as before. Additionally, it is possible to create shortcut controllers and shortcuts in ui files.

A set of smaller features has landed in the form of a few GtkTextTag properties that expose new pango features such as overlines, visible rendering of spaces and control over hyphenation. These can now be controlled in a GtkTextView via tags. In entries, they can already be controlled by directly adding pango attributes.

Completions

When I wrote about 3.98, I said that the Drag-and-Drop refactoring was complete. That turned out to be not quite correct, and another round of DND work has landed since. These changes were informed by developer feedback on the Drag-and-Drop API. Yay for user testing!

We introduced separate GtkDropTarget and GtkDropTargetAsync event controllers, with the former being simplified to avoid all async API, which makes it very easy to handle local cases.

We also cleaned up internals of the DND implementation to group DND events into event sequences, handle them in just the same way as normal motion events,  and introduced GtkDropControllerMotion, which is an event controller that is designed to handle things like tab switching during a DND operation.

Finally, we could remove the remnants of X11-style property and selection APIs; GtkSelectionData and GdkAtom are gone.

Cleanups and fixes

As always, there’s a large number of smaller cleanups and fixes that have happened.

The biggest group of cleanups happened in the file chooser, where a number of marginally useful APIs (extra widgets, overwrite confirmation, :local-only, GTK_FILE_CHOOSER_ACTION_CREATE_FOLDER, etc) have been dropped. To make up for it, the portal implementation of the native file chooser supports selecting folders now.

Another big cleanup was that GdkEvent is now an immutable boxed type. This was mainly an internal cleanup; the effect on application-level APIs is small, since event controllers have replaced direct event handling for the most part.

One new such event controller is GdkEventControllerFocus, which was split of from the key event controller to provide just focus handling.

GtkMenuButton lost its ability to have mnemonics when it was turned from a GtkButton subclass into a plain widget. This functionality has been reinstated, with a :use-underline property.

The HighContrast and HighContrastInverse themes that are included in GTK are now derived from Adwaita, for a much reduced maintainance burden and improved quality. Trying these themes out in gtk4-widget-factory is now easier, since we added a style menu.

The new HighContrast theme has also been backported to GTK 3.

Whats ahead

We will continue our snapshots and hope to get more developer feedback on the new APIs and features described above.

Here are things that we still want to integrate before GTK 4:

  • Row-recycling list and grid views
  • Revamped accessibility infrastructure
  • Animation API

If you want to follow the GTK 4 work, go here.

Building and testing GTK

… or: how GTK developers check their work on the toolkit.

Since GNOME’s collective move to GitLab, GTK has taken advantage of the features provided by that platform—especially when it comes to its continuous integration pipeline.

In days of old, the only way to check that our changes to the toolkit were correct was to wait until the Continuous build bot would notify us of any breakage on the main development branches. While this was better than nothing, it didn’t allow us to prevent breakage during the development phase of anything—from features to bug fixes, from documentation improvements to adding new tests.

These days, the CI pipeline available in GitLab is run on every branch and merge request, long before the changes reach the public development branches used by everybody else.

Topic branches and merge requests

When developing a topic branch against the GTK 4 main development one, we run a CI pipeline that starts with a simple coding style check for the changes applied in the branch. The style check uses clang-format, which is often good enough for the GTK coding style; the coding style has a few “special” caveats, and clang-format can raise false negatives and false positives. For that reason, the style check is allowed to fail, but contributors and reviewers are strongly encouraged to check the logs in case of failure.

Once the style check is passed, we run the build phase, which currently contains three separate jobs:

  • a Linux debug build, using a Fedora container
  • an MSYS2 build on Windows
  • a Linux release build

The Linux debug build is pretty standard fare.

The MSYS2 build catches any issue with a GNU toolchain on Windows.

The release build is necessary to ensure that we don’t rely on side effects of the debugging code we have in place during development.

All of these jobs run the GTK test suite.

We publish the tests reports both as a JUnit file, taking advantage of GitLab’s support; and as an HTML report, stored as a pipeline artifact. This makes it easier for us to check what failed and what succeeded.

Ideally, we want to add more environments:

  • Linux builds based on other mainstream distributions
  • a Windows build using the MSVC toolchain
  • a macOS build, once the GDK backend is fixed

After the build and testing jobs pass, we step into an analysis phase. We run the Clang static analysis tool on GTK’s code base and generate a report. In the near future we could also run sanitizer tools like UBSan and ASan; fuzzying tools for our parsers, like GtkBuilder and CSS; or tools that verify that our UI definitions contain the appropriate accessibility descriptors.

Just like the tests, we publish the analysis reports as GitLab artifacts for review.

Once the analysis phase is passed, we build the API references, and check the result so that newly added symbols are properly documented.

Finally, we have manual CI jobs to build Flatpak bundles for the GTK demo application; the widget factory; and the icon browser. This allows designers to immediately test changes in Adwaita, or newly added widgets, without necessarily building GTK from a scratch on their systems.

Mainline development branches

Once the CI pipeline for a topic branch/merge request passes, we can merge the changes into the main development branch with a certain level of confidence that the code is correct and does what we want.

The main development branch runs the same pipeline as previously described, except that the Flatpak jobs are not manual any more—thus is always possible to test locally the current bleeding edge of GTK. Additionally, the documentation is published online, so it’s always up to date.

The GTK CI pipeline

What about GTK 3?

In the GTK 3 branch we have a simpler pipeline that runs the following jobs:

  • a full Meson debug build on Linux and Windows/MSYS2, for both static and shared libraries artifacts, on the current stable versions of Fedora and Debian
  • a full Meson release build on Linux, which also generates the API reference
  • an Autotools build on Linux and Windows/MSYS2
  • an optional Autotools distcheck build on Linux

The Autotools jobs will be in place for as long as GTK 3 supports Autotools. Ideally, we want to add other jobs for macOS and Windows/MSVC, taking advantage of the Meson build.

The GTK3 CI pipeline

Once the GTK 4 CI pipeline reaches a certain level of features and stability, we’re going to backport it to GTK 3, so we can be even more confident that the current stable branch does not regress.


For more information, you can check the GTK repository:

GTK Hackfest 2020 — Roadmap and accessibility

Between January 28th and January 31st, the GTK team held what’s now the third hackfest in Brussels.

The main topics of the hackfest were:

  • the schedule for the next development snapshot of GTK4
  • the missing features blocking the release of GTK 4.0
  • the current state of the accessibility support in the toolkit

The first two items occupied the most of the first two days of the hackfest; you can read the GTK 3.98 release announcement for what we’ve been working on for the past 300 days since the 3.96 release. The missing features are:

  • Event controllers for keyboard shortcuts
  • Movable popovers on Wayland
  • Row-recycling list and grid views
  • Animation API

and all of them are being worked on in topic branches. The keyboard shortcuts branch has recently been rebased, and it’s in the process of being documented and cleaned up; the movable popovers is also being reviewed after a few iterations. The last two remaining branches are fairly sizeable, and will require some more iterations to get them right—with the animations API currently being mostly a prototype.

The final topic of the hackfest was the largest, and was a discussion long overdue.

GTK’s accessibility support was added as part of the GTK 2.0 release by the Sun Accessibility Team; it depends on the abstract data types provided by ATK (the Accessibility Tool Kit), which are then implemented concretely in GTK classes like GtkWidgetAccessible, or GtkEntryAccessible. Each widget has an “accessible” object associated to it, which is either automatically created by GTK, or can be provided by application code when subclassing a GTK widget. Non-widget types can also have accessible objects associated to them—the most notable case is the set of cell renderers for tree views and combo boxes. Underneath it all, sits AT-SPI, a protocol that is used by AT—Accessible Technologies, like a screen reader—to consume the data provided by applications. Typically, ATs will use a library like libatspi to deal with the protocol itself.

The main issues with the existing stack are:

  • there’s a lot of indirection caused by the existence of ATK; any new feature or bug fix needs to be defined inside ATK and then implemented into GTK and libatspi
  • ATK was written in a very different environment, and while it has seen a few deprecations, it shows its age in the assumptions it makes—like global coordinate spaces—and in its design
  • there’s a certain overlap between AT requirements and requirements for GUI testing that end up creating friction in the API design
  • the stack has fell in disrepair since the Sun accessibility team was disbanded; most of the ongoing work is still pretty much happening in the AT space (like Orca) and in web browsers
  • the entire stack was written when CORBA was a thing, and then ported to DBus in time for GNOME3; the protocol, though, is not really efficient and requires lots of roundtrips to move around small amounts of data, instead of having bulk operations and notifications

The last point is also the reason why we need a separate accessibility bus in order to avoid spamming the session bus, and making everything slower as soon as the accessibility support is enabled. A separate bus means that we need to poke an additional hole in any sandbox, and still lets everything that connects to the accessibility bus potentially snoop into what happens in every application.

Finally, GTK only supports accessibility on Linux; there is no support for macOS or Windows, which means applications written in GTK and ported to other platforms are not accessible to ATs there. As we expose ATK in our API, adding support for accessibility features on other platforms would require bridging ATK, creating further complexity.

As we want to redesign and update the accessibility features in GTK4, we need to understand what are the requirements for existing consumers of the accessibility stack, and what kind of use cases we need to target. For that, we asked Hypra, a company dedicated to the development of accessible solutions based on free and open source software, to help us.

Hypra developers are familiar with GNOME, and have been working on the Linux accessibility stack. Their clients cover a wide gamut of accessibility users, so they are in the best position to describe what kind of ATs are in actual use on a day to day basis.

There are a wide range of tools and functionality that have to be provided by different layers of the stack, from the toolkit to the compositor; application developers must also have access to the tools necessary to provide proper support to ATs, as they have a much better idea of what their applications should look and behave than the toolkit.

Over the course of two days we have identified a plan for moving forward:

  • drop ATK from the stack, and have GTK talk the AT-SPI protocol directly; this is similar to what Qt does from the toolkit side, and it makes it easier to both expand and verify eventual protocol changes
  • clean up the AT-SPI protocol itself, updating it where needed when it comes to using DBus more efficiently
  • drop the global accessibility bus, and have ATs negotiate a peer-to-peer connection to each application
  • make ATs ask the compositor to gather global state, like key shortcuts, instead of talking to applications that would then have to ask the windowing system—if that’s possible—or return invalid data when it isn’t
  • decouple GUI testing from accessibility
  • write widget and application authoring guides for application developers, and provide validation tools that can be used as part of the build and CI process to check if UI elements have the correct accessible description and links

There are more information available on the wiki for the notes and the roadmap, and we have already scheduled an additional check point meeting for this summer.

There’s a lot of work to be done, but we have now a much clearer idea of the scope and deliverables for such a redesign. If you want to help making things happen faster, feel free to join the effort; you can also make a donation to the GNOME Foundation.

The GTK team would like to thank the GNOME Foundation for the sponsorship for the venue and the attendees, and the fine folks at Hypra for joining the hackfest and explaining use cases and the current state of the accessibility stack, as well as helping out on the development side.

GTK 3.98

A few days ago, I’ve released a GTK 3.98 tarball. This is another step towards GTK 4. It is a little bit behind schedule, and does not quite include all the things we wanted to get into it, but it gets a lot closer to what we want to ship in GTK 4.

Almost 9 months have passed since the 3.96 snapshot, so there are quite a few new things to look at. Too many to cover them all, but here are some highlights:

Performance

The GL renderer has seen a steady flow of optimizations and performance improvements.

After the Westcoast hackfest last year, the GtkTextView scrolling performance has been greatly improved by making it cache render nodes for the visible range. At the same hackfest, the text caret blinking was changed to be smoothly animated, which is not relevant for performance at all, but looks cool.

Since the new year, a big focus has been on improving the performance of the CSS machinery. The CSS value implementation has been optimized to avoid computing values whenever possible. CSS lookups are using a Bloom filter now. And the IO for icon loading has been moved to a thread.

Most of the recent work was made possible by the sysprof profiling support that was added after the performance hackfest, and has recently been enhanced to report more information. To use it, simply start a GTK application with GTK_TRACE=1 in the environment, and load the resulting syscap file with sysprof.

DND

The DND refactoring has been completed. The GTK API for DND has been turned into event controllers: GtkDragSource and GtkDropTarget. Support for file transfers via file transfer portal has been added for both DND and the clipboard.  The underlying new infrastructure for data transfers has been covered in detail before.

GDK

The move of GDK towards Wayland concepts is continuing. This cleanup is not 100% complete yet.

Child surfaces have been removed. GDK only supports toplevel and popup surfaces now. The client-side window implementation has been removed too. Global positions and related APIs such as gdk_surface_move() are no longer available.

Grabs are no longer exposed as API. As a replacement, popup surfaces can be configured to hide on outside clicks.

XI2 is now mandatory when building the X11 backend, and support for the xim input method has been removed in favor of IBus.

The Wayland backend no longer relies on libwayland-cursor to load cursor themes, and loads individual cursors on demand.

GTK removals

Many classes have been made explicitly non-subclassable, and the widget hierarchy has been simplified, by making widgets derive directly from GtkWidget where possible.

GtkMenu, GtkMenuBar, GtkToolbar and related classes have been removed. They are being replaced by GMenu and popover-based variants. Popover menus can now do traditional, nested menus, and also show accelerators.

Context menus are no longer created with ::populate-popup signals, but also use menu models and actions. Creating those actions has been made easier with APIs like gtk_widget_class_install_action(), to create them in class_init.

GtkGestureMultiPress has been renamed to GtkGestureClick, to make it more obvious what this event controller is for.

GTK additions

We did not just remove things. Some new things have been added too.

The GtkNative interface has been introduced for widgets that have their own surface. This has been split off from the GtkRoot interface, which is exclusively for toplevel widgets without a parent.

A constraint-based layout manager has been added. It would be great to see people try this out. Please give us feedback if you do.

GtkTextView and other text widgets have gained a simple undo stack that can be used with Ctrl-Z.

The Emoji chooser widget has been made public.

Whats ahead

After 3.98, I’m planning to do more frequent snapshots, as the remaining outstanding items are landing. What are those items, you are asking ?

Here are the things that we still want to integrate before GTK 4:

– Event controllers for keyboard shortcuts
– Movable popovers
– Row-recycling list and grid views
– Revamped accessibility infrastructure
– Animation API

 

 

Data transfer in GTK4

The traditional methods for user-initiated data transfers between desktop apps are the clipboard or Drag-and-Drop. GTK+ has supported these methods since the beginning of time, but up until GTK3, the APIs we had for this kind of data transfer were thinly disguised copies of the corresponding X11 apis: selections, and properties and atoms. This is not too surprising, since the entire GDK api was modeled on X11. Unfortunately, the implementation includes horrors such as incremental transfers and string format conversions.

For GTK4, we’re leaving these things behind as we are moving things in GDK around to be closer to the Wayland API. Data transfer is one the areas in most urgent need of this modernization. Thankfully, it is almost complete at this point, so it is worth taking a look at what has changed, and how things will work in the future.

Concepts

If the data your application wants to send is not a string, it is probably some object, such as GFile, a GdkTexture, or a GdkRGBA. The app on the receiving side may be not use GTK or GLib, and thus won’t know these types. And even if it does, there’s no way to get the objects from one process to the other in one piece.

At the core, the data transfer works be sending a file descriptor from the source app, and the target app reading a stream of bytes from it. The protocols for clipboard and DND use mime types such as text/uri-list, image/png or application/x-color to identify the format of the byte stream.

Sending an object involves negotiating a data format that both sides support, serializing the object on the source side into a byte stream of that format, transferring the data, and deserializing the object on the target side.

The local case

Before moving on to concrete APIs, it is worth pausing a moment to consider another common case of data transfer: inside a single application. It is quite common to use the same clipboard and DND mechanisms to send data from the one side of the application to the other. Since we are not crossing process boundaries in this case, we can avoid the byte stream and the associated serialization and deserialization overhead, and transfer the data by just passing a reference to the object.

APIs

The objects we’ve alluded to in the previous section are described by GTypes such as G_TYPE_FILE or GDK_TYPE_TEXTURE, while, as mentioned, the data exchange formats in Wayland and X11 protocols are described by mime types.

The first API we have introduced to deal with these types is the GdkContentFormats object. It can contain a list of formats, which may be GTypes or mime types. We use GdkContentFormats objects to describe the formats that an application can offer data in, as well as the formats that an application can receive data it.

You may wonder why we are mixing GTypes and mime types in the same object. The answer is that we want to handle both the cross-process and the local case with the same APIs. And while we need to match mime types for the cross-process case, we need the GTypes for the local case.

Conversion

We still need a way to associate GTypes and mime types that we can convert into each other. This is handled by the GdkContentSerializer and GdkContentDeserializer APIs. These are essentially registries of conversion functions: GdkContentSerializer knows how to convert GTypes into mime types, and GdkContentDeserializer handles the other direction.

GDK has predefined conversions for common types, but the system is extensible using gdk_content_register_serializer and gdk_content_register_deserializer.

Content

Now we know how to describe formats and convert between them, but to  put this all together, we still need a convenient api that takes an object on the one side, and provides a bytes stream on the other.  For this, we added the GdkContentProvider API.

A GdkContentProvider lets you combine an object with a format description on the input side, and provides an async writer API on the output side that can be connected to the file descriptor that we want to send the data over.

Typical content providers are created like this:

gdk_content_provider_new_for_value (gvalue)
gdk_content_provider_new_for_bytes (gbytes, mimetype)

The GValue that it takes contains both an object and information about its type, so we don’t need extra type information, if we provide the object as  a GBytes (essentially a just a bit of memory), we need to provide the type information separately.

The Clipboard

GTK3 has a GtkClipboard object, which provides the implemention for copy/paste operations. Having this object in GTK is not ideal, since it needs different implementations on the platforms that are supported by GTK. Therefore, GTK4 moves the object to GDK, and consequently renames it to GdkClipboard. It has also been ported to the new data transfer apis that are described above. To put data on the clipboard in GTK4, you use one of the ‘set’ apis:

gdk_clipboard_set_content()
gdk_clipboard_set_value()
gdk_clipboard_set_text()

Ultimately, all of these functions end up associating a GdkContentProvider with the clipboard.

To read data from the clipboard in GTK4, you use one of the async ‘read’ apis:

gdk_clipboard_read_async()
gdk_clipboard_read_value_async()
gdk_clipboard_read_text_async()

Drag-and-Drop

The GTK3 Drag-and-Drop api involves listening to a number of signals on GtkWidget, and calling a few special setup functions for drag sources and destinations. It is flexible, but generally considered confusing, and we won’t describe it in detail here.

In GTK4,  the Drag-and-Drop api has been reorganized around the concepts of content providers and event controllers. To initiate a Drag-and-Drop operation, you create a GtkDragSource event controller that reacts to drag gestures (you can also start ‘one-off’ Drag-and-Drop operations by just calling gdk_drag_begin yourself), and you give it a GdkContentProvider for the data that you want to transfer. To receive Drag-and-Drop operations, you create a GtkDropTarget event controller and call an async read method when it emits the ::drop-done signal:

gdk_drop_read_value_async()
gdk_drop_read_text_async()

GTK BoF at Guadec

As every year, we had a GTK BoF at Guadec in Thessaloniki. This year, we had a pretty good turnout — everybody was interested in GTK4 plans.

But since Emmanuele was busy in the morning, we started the discussion with some other topics.

GLib

We collected a few suggestions for useful GLib additions from the room.

  • API for OS information (basically, the data that is in /etc/os-release). This seemed uncontrolversial; Robert is going to implement it
  • An ordered map. This is implemented ad-hoc in many places by combining a hash table with a list or array.  There seemed to be agreement that it would be worthwhile to provide this in GLib, if somebody does the work to propose an API

The discussion of ordered maps also touched on generic container interfaces; Philipp described how that could be done, see details here.

Still on the topic of containers, Alex described a problem with transfer annotations. We discussed various ideas, but there may not be a perfect solution.

Matthias pointed out that there is still some Unicode data in Pango. We briefly discussed how nice it would be to have an agreed-on, mmappable binary format for Unicode data, so that everybody could share it. Short of that, moving the last bits of data to GLib was uncontroversial.

Dark mode

Since the “dark mode” BoF joined us, we switched to discussing dark mode next. There was a more discussion of this topic in the vendor theme BoF the next day; the GTK discussion focused on technical details of how to implement dark mode.

There are various options:

  • Add extra metadata to theme index files to mark themes as dark
  • Add a “dark-theme-name” setting and treat dark and light themes as independent
  • Keep the existing convention of appending “-dark” to theme names to find the dark variant of a theme

The pragmatic solution of keeping the existing convention seemed to have support in the room. Matthias started exploring some application support APIs here.

GTK

Eventually we swiched to talking about the state of and progress towards GTK4. The high-level summary is that there is still a list of  features that need to be completed for GTK4:

  • A scalable list view that recycles row widgets. This includes a broader switch to using list models in more places. To make it complete, it should also include a grid view using the technologies. Benjamin is working on this
  • Infrastructure and APIs for animations. This will be similar to the way animations work in CSS, and part of the work is to port not just our existing CSS animation support, but also stack switching animations, the revealer, progress bars and spinners to the new framework. Emmanuele is working on this.
  • Complete the menu/popover rework. Some people tried the new popover menubar. The feedback was that we should probably go back to nesting submenus, at least for menubars, and push forward with dropping menus, since some of the ways in which menus are special (such as keep-up triangles, scrolling) are hard to keep working (or keep working well). Matthias is getting back to working on this after Guadec.
  • Shortcuts – replace mnemonics, accelerators, and key bindings with event controllers. There is a fairly complete branch with code and APIs that several people have worked on; help with reviewing and testing it would be appreciated.
  • The new Drag-and-Drop API needs to be completed.

The good news is that this list is fairly short and has names next to most items. The bad news is that each item is a considerable amount of work. Therefore, it is not a good idea to promise a tight timeline towards the 4.0 release before we have all of them merged. Thus, the following is tentative, but (we hope) somewhat realistic:

  • another GTK 3.9x snapshot before the end of this year
  • a feature-complete 3.99 release around the same time as GNOME 3.36 in spring 2020
  • a 4.0 release around the same time as GNOME 3.38 in fall  2020

Inevitably, we also discussed other things that would be nice to have. None of these are on the GTK4 roadmap; but if somebody shows up to do the work, they can happen:

  • A “widget repository” or “hig” library in order to not overload GTK with too specific or experimental widgets
  • A “UI designer” widget. This could live in a separate library as well
  • Better support for split headerbars and state transitions

We also discussed things outside GTK proper that will keep applications from porting to GTK4. This includes commonly used libraries such as GtkSourceView, vte and webkitgtk, which all will need GTK4 ports before applications that depend on them can be ported. Some of this work is already underway; but any help in this area is appreciated!

Another potential blocker for GTK4 porting is platform support. The GL renderer works well on Linux; the Vulkan renderer needs some fixups. On Windows we currently use cairo fallback, which may be good enough for 4.0. Alternatively, we could merge existing work for using the GL renderer with ANGLE. The situation is less pleasant on OS X, where we don’t have a working backend; if you want to help us here, the first still would be to adapt the GDK backend to changes in GDK.

Hacking time

In the afternoon, the room drifted from discussion to hacking, and various GTK-related works-in-progress could be spotted on peoples laptops: work to speed up GtkBuilder template loading, nested popover menus, a half-finished GtkSourceView port.

You will hopefully see these (and others) in GTK master soon.

Constraint layouts

What are constraints

At its most basic, a constraint is a relation between two values. The relation
can be described as a linear equation:

target.attribute = source.attribute × multiplier + constant

For instance, this:

Can be described as:

blue.start = red.end × 1.0 + 8.0

Or:

  • the attribute, “start”, of the target, “blue”, which is going to be set by the constraint; this is the left hand side of the equation
  • the relation between the left and right hand sides of the equation, in this case equality; relations can also be greater than or equal to,
    and less than or equal to
  • the attribute, “end”, of the source, “red”, which is going to be read by the constraint; this is the right hand side of the equation
  • the multiplier, “1.0”, applied to the attribute of the source
  • the constant, “8.0”, an offset added to the attribute

A constraint layout is a series of equations like the one above, describing all the relationships between the various parts of your UI.

It’s important to note that the relation is not an assignment, but an equality (or an inequality): both sides of the equation will be solved in a way that satisfies the constraint; this means that the list of constraints can be rearranged; for instance, the example above can be rewritten as:

red.end = blue.start × 1.0 - 8.0

In general, for the sake of convenience and readability, you should arrange your constraints in reading order, from leading to trailing edge, from top to bottom. You should also favour whole numbers for multipliers, and positive numbers for constants.

Solving the layout

Systems of linear equations can have one solution, multiple solutions, or even no solution at all. Additionally, for performance reasons, you don’t really want to recompute all the solutions every time.

Back in 1998, the Cassowary algorithm for solving linear arithmetic constraints was published by Greg J. Badros and Alan Borning, alongside its implementation in C++, Smalltalk, and Java. The Cassowary algorithm tries to solve a system of linear equations by finding its optimal solution; additionally, it does so incrementally, which makes it very useful for user interfaces.

Over the past decade various platforms and toolkits started providing layout managers based on constraints, and most of them used the Cassowary algorithm. The first one was Apple’s AutoLayout, in 2011; in 2016, Google added a ConstraintLayout to the Android SDK.

In 2016, Endless implemented a constraint layout for GTK 3 in a library called Emeus. Starting from that work, GTK 4 now has a GtkConstraintLayout layout manager available for application and widget developers.

The machinery that implements the constraint solver is private to GTK, but the public API provides a layout manager that you can assign to your GtkWidget class, and an immutable GtkConstraint object that describes each constraint you wish to add to the layout, binding two widgets together.

Guiding the constraints

Constraints use widgets as sources and targets, but there are cases when you want to bind a widget attribute to a rectangular region that does not really draw anything on screen. You could add a dummy widget to the layout, and then set its opacity to 0 to avoid it being rendered, but that would add unnecessary overhead to the scene. Instead, GTK provides GtkConstraintGuide, and object whose only job is to contribute to the layout:

An example of the guide UI element

In the example above, only the widgets marked as “Child 1” and “Child 2” are going to be visible, while the guide is going to be an empty space.

Guides have a minimum, natural (or preferred), and maximum size. All of them are constraints, which means you can use guides not just as helpers for alignment, but also as flexible spaces in a layout that can grow and shrink.

Describing constraints in a layout

Constraints can be added programmatically, but like many things in GTK, they can also be described inside GtkBuilder UI files, for convenience. If you add a GtkConstraintLayout to your UI file, you can list the constraints and guides inside the special “<constraints>” element:

  <object class="GtkConstraintLayout">
    <constraints>
      <constraint target="button1" target-attribute="width"
                     relation="eq"
                     source="button2" source-attribute="width" />
      <constraint target="button2" target-attribute="start"
                     relation="eq"
                     source="button1" source-attribute="end"
                     constant="12" />
      <constraint target="button1" target-attribute="start"
                     relation="eq"
                     source="super" source-attribute="start"
                     constant="12" />
      <constraint target="button2" target-attribute="end"
                     relation="eq"
                     source="super" source-attribute="end"
                     constant="-12"/>
    </constraints>
  </object>

You can also describe a guide, using the “<guide>” custom element:

  <constraints>
    <guide min-width="100" max-width="500" />
  </constraints>

Visual Format Language

Aside from XML, constraints can also be described using a compact syntax called “Visual Format Language”. VFL descriptions are row and column oriented: you describe each row and column in the layout using a line that visually resembles the layout you’re implementing, for instance:

|-[findButton]-[findEntry(<=250)]-[findNext][findPrev]-|

Describes an horizontal layout where the findButton widget is separated from the leading edge of the layout manager by some default space, and followed by the same default amount of space; then by the findEntry widget, which is meant to be at most 250 pixels wide. After the findEntry widget we have some default space again, followed by two widgets, findNext and findPrev, flush one against the other; finally, these two widgets are separated from the trailing edge of the layout manager by the default amount of space.

Using the VFL notation, GtkConstraintLayout will create all the required constraints without necessarily having to describe them all manually.

It’s important to note that VFL cannot describe all possible constraints; in some cases you will need to create them using GtkConstraint’s API.

Limits of a constraint layout

Constraint layouts are immensely flexible because they can implement any layout policy. This flexibility comes at a cost:

  • your layout may have too many solutions, which makes it ambiguous and unstable; this can be problematic, especially if your layout is very complex
  • your layout may not have any solution. This is usually the case when you’re not using enough constraints; a rule of thumb is to use at least two constraints per target per dimension, since all widgets should have a defined position and size
  • the same layout can be described by different series of constraints; in some cases it’s virtually impossible to say which approach is better, which means you will have to experiment, especially when it comes to layouts that dynamically add or remove UI elements, or that allow user interactions like dragging UI elements around

Additionally, at larger scales, a local, ad hoc layout manager may very well be more performant than a constraint based one; if you have a list box that can grow to an unknown amount of rows you should not replace it with a constraint layout unless you measure the performance impact upfront.

Demos

Of course, since we added this new API, we also added a few demos to the GTK Demo application:

A constraints demo
The constraints demo window, as part of the GTK demo application.

As well as a full constraints editor demo:

The GTK constraints editor demo
A screenshot of the GTK constraints editor demo application, showing the list of UI elements, guides, and constraints in a side bar on the left, and the result on the right side of the window

More information

GTK 3.96.0

This week, we released GTK 3.96.0. Again, it has been a while since the last release, so it is worth summarizing whats new in this release. There is really too much here to cover it all, so this post will only highlight the most important changes.

This release is another milestone on our way towards GTK 4. And while there are still some unfinished things, this release is much closer to we hope to achieve with GTK 4.

GSK

GSK has seen a number of bug fixes and new tests that are made much easier using a new debug tool, gtk4-node-editor. It can load and display serialized render node trees, such as this one that was saved from the GTK inspector, and compare the output of different renderers.

The 3D transformation support has been brought up to the level where we can do animated transitions like the cube spin below.

GDK

The trend to move toward Wayland inspired APIs has continued, with more X11-only apis being moved to the X11 backend or just removed. Use of child surfaces and global coordinates has been greatly reduced, but this work remains incomplete.

The refactoring of Drag-and-Drop has also continued, with the introduction of GdkDrag and GdkDrop objects. The GTK part of this refactoring is still incomplete.

Events have been simplified and are now used just for input. Other event have been replaced by signals and properties on GdkSurface. In detail, expose events have been replaced by the ::render signal, configure events have been replaced by the ::size-changed signal. Map events have been replaced by the :mapped property, and gdk_event_handler_set() has been replaced by the ::event signal.

The Wayland backend has gained support for the Settings portal for GtkSettings, and uses the text-input-unstable-v3 protocol for its input method support.

GTK

Widgets

One big change for custom widgets is the introduction of GtkLayoutManager, which is a new delegate object that takes over size allocation. Layout managers can optionally use layout children for holding layout properties. This replaces the layout-related child properties in GTK containers such as GtkBox or GtkGrid.

A number of layout managers are available:

  • GtkBinLayout, for simple single-child containers
  • GtkBoxLayout, for children that are arranged linearly
  • GtkGridLayout, for children that are arranged in a grid
  • GtkFixedLayout, for freely positioned and transformed children
  • GtkCustomLayout, as a quick way to turn traditional measure and size_allocate vfuncs into a layout manager

More layout manager implementations will appear in the future. Most prominently, work is underway on a constraints-based layout manager.

GtkAssistant, GtkStack and GtkNotebook have publicly
accessible page objects for their children. The page objects
are also exposed via a list model. They non-layout related child properties of these containers have been converted into regular properties on these page objects.

Since all existing child properties have been converted to regular properties, moved to layout properties or moved to such page objects, support for child properties has been dropped from GtkContainer.

The core GtkEntry functionality has been moved into a new GtkText widget, which also implements an expanded GtkEditable interface. All existing entry subclasses in GTK have been turned into GtkEditable implementations wrapping a GtkText widget. This also includes a new GtkPasswordEntry.

Other Changes

GTK widgets can transform their children using projective linear
transformations. This functionality is available in CSS and
as a GskTransform argument to gtk_widget_allocate. GtkFixed is
the first container that exposes this functionality. For further examples,
see the swing transition of GtkRevealer, the rotate transitions
of GtkStack or the Fixed Layout example in gtk4-demo.

A number of list models have been introduced, for internal use
and as public API: GtkMapListModel, GtkSliceListModel, GtkSortListModel, GtkSelectionModel, GtkSingleSelection. These will become more widely used when we introduce a list model-based GtkListView.

GtkBuilder can specify object-valued properties inline, instead of referring to them by ID, and the simplify command of gtk4-builder-tool has gained an option to automatically convert GTK 3 UI definition files to GTK 4.

Coming soon

For more information on the things that are still still coming for GTK 4, find us on Discourse, IRC, or look here.