Constraint layouts

What are constraints

At its most basic, a constraint is a relation between two values. The relation
can be described as a linear equation:

target.attribute = source.attribute × multiplier + constant

For instance, this:

Can be described as:

blue.start = red.end × 1.0 + 8.0

Or:

  • the attribute, “start”, of the target, “blue”, which is going to be set by the constraint; this is the left hand side of the equation
  • the relation between the left and right hand sides of the equation, in this case equality; relations can also be greater than or equal to,
    and less than or equal to
  • the attribute, “end”, of the source, “red”, which is going to be read by the constraint; this is the right hand side of the equation
  • the multiplier, “1.0”, applied to the attribute of the source
  • the constant, “8.0”, an offset added to the attribute

A constraint layout is a series of equations like the one above, describing all the relationships between the various parts of your UI.

It’s important to note that the relation is not an assignment, but an equality (or an inequality): both sides of the equation will be solved in a way that satisfies the constraint; this means that the list of constraints can be rearranged; for instance, the example above can be rewritten as:

red.end = blue.start × 1.0 - 8.0

In general, for the sake of convenience and readability, you should arrange your constraints in reading order, from leading to trailing edge, from top to bottom. You should also favour whole numbers for multipliers, and positive numbers for constants.

Solving the layout

Systems of linear equations can have one solution, multiple solutions, or even no solution at all. Additionally, for performance reasons, you don’t really want to recompute all the solutions every time.

Back in 1998, the Cassowary algorithm for solving linear arithmetic constraints was published by Greg J. Badros and Alan Borning, alongside its implementation in C++, Smalltalk, and Java. The Cassowary algorithm tries to solve a system of linear equations by finding its optimal solution; additionally, it does so incrementally, which makes it very useful for user interfaces.

Over the past decade various platforms and toolkits started providing layout managers based on constraints, and most of them used the Cassowary algorithm. The first one was Apple’s AutoLayout, in 2011; in 2016, Google added a ConstraintLayout to the Android SDK.

In 2016, Endless implemented a constraint layout for GTK 3 in a library called Emeus. Starting from that work, GTK 4 now has a GtkConstraintLayout layout manager available for application and widget developers.

The machinery that implements the constraint solver is private to GTK, but the public API provides a layout manager that you can assign to your GtkWidget class, and an immutable GtkConstraint object that describes each constraint you wish to add to the layout, binding two widgets together.

Guiding the constraints

Constraints use widgets as sources and targets, but there are cases when you want to bind a widget attribute to a rectangular region that does not really draw anything on screen. You could add a dummy widget to the layout, and then set its opacity to 0 to avoid it being rendered, but that would add unnecessary overhead to the scene. Instead, GTK provides GtkConstraintGuide, and object whose only job is to contribute to the layout:

An example of the guide UI element

In the example above, only the widgets marked as “Child 1” and “Child 2” are going to be visible, while the guide is going to be an empty space.

Guides have a minimum, natural (or preferred), and maximum size. All of them are constraints, which means you can use guides not just as helpers for alignment, but also as flexible spaces in a layout that can grow and shrink.

Describing constraints in a layout

Constraints can be added programmatically, but like many things in GTK, they can also be described inside GtkBuilder UI files, for convenience. If you add a GtkConstraintLayout to your UI file, you can list the constraints and guides inside the special “<constraints>” element:

  <object class="GtkConstraintLayout">
    <constraints>
      <constraint target="button1" target-attribute="width"
                     relation="eq"
                     source="button2" source-attribute="width" />
      <constraint target="button2" target-attribute="start"
                     relation="eq"
                     source="button1" source-attribute="end"
                     constant="12" />
      <constraint target="button1" target-attribute="start"
                     relation="eq"
                     source="super" source-attribute="start"
                     constant="12" />
      <constraint target="button2" target-attribute="end"
                     relation="eq"
                     source="super" source-attribute="end"
                     constant="-12"/>
    </constraints>
  </object>

You can also describe a guide, using the “<guide>” custom element:

  <constraints>
    <guide min-width="100" max-width="500" />
  </constraints>

Visual Format Language

Aside from XML, constraints can also be described using a compact syntax called “Visual Format Language”. VFL descriptions are row and column oriented: you describe each row and column in the layout using a line that visually resembles the layout you’re implementing, for instance:

|-[findButton]-[findEntry(<=250)]-[findNext][findPrev]-|

Describes an horizontal layout where the findButton widget is separated from the leading edge of the layout manager by some default space, and followed by the same default amount of space; then by the findEntry widget, which is meant to be at most 250 pixels wide. After the findEntry widget we have some default space again, followed by two widgets, findNext and findPrev, flush one against the other; finally, these two widgets are separated from the trailing edge of the layout manager by the default amount of space.

Using the VFL notation, GtkConstraintLayout will create all the required constraints without necessarily having to describe them all manually.

It’s important to note that VFL cannot describe all possible constraints; in some cases you will need to create them using GtkConstraint’s API.

Limits of a constraint layout

Constraint layouts are immensely flexible because they can implement any layout policy. This flexibility comes at a cost:

  • your layout may have too many solutions, which makes it ambiguous and unstable; this can be problematic, especially if your layout is very complex
  • your layout may not have any solution. This is usually the case when you’re not using enough constraints; a rule of thumb is to use at least two constraints per target per dimension, since all widgets should have a defined position and size
  • the same layout can be described by different series of constraints; in some cases it’s virtually impossible to say which approach is better, which means you will have to experiment, especially when it comes to layouts that dynamically add or remove UI elements, or that allow user interactions like dragging UI elements around

Additionally, at larger scales, a local, ad hoc layout manager may very well be more performant than a constraint based one; if you have a list box that can grow to an unknown amount of rows you should not replace it with a constraint layout unless you measure the performance impact upfront.

Demos

Of course, since we added this new API, we also added a few demos to the GTK Demo application:

A constraints demo
The constraints demo window, as part of the GTK demo application.

As well as a full constraints editor demo:

The GTK constraints editor demo
A screenshot of the GTK constraints editor demo application, showing the list of UI elements, guides, and constraints in a side bar on the left, and the result on the right side of the window

More information

GTK 3.96.0

This week, we released GTK 3.96.0. Again, it has been a while since the last release, so it is worth summarizing whats new in this release. There is really too much here to cover it all, so this post will only highlight the most important changes.

This release is another milestone on our way towards GTK 4. And while there are still some unfinished things, this release is much closer to we hope to achieve with GTK 4.

GSK

GSK has seen a number of bug fixes and new tests that are made much easier using a new debug tool, gtk4-node-editor. It can load and display serialized render node trees, such as this one that was saved from the GTK inspector, and compare the output of different renderers.

The 3D transformation support has been brought up to the level where we can do animated transitions like the cube spin below.

GDK

The trend to move toward Wayland inspired APIs has continued, with more X11-only apis being moved to the X11 backend or just removed. Use of child surfaces and global coordinates has been greatly reduced, but this work remains incomplete.

The refactoring of Drag-and-Drop has also continued, with the introduction of GdkDrag and GdkDrop objects. The GTK part of this refactoring is still incomplete.

Events have been simplified and are now used just for input. Other event have been replaced by signals and properties on GdkSurface. In detail, expose events have been replaced by the ::render signal, configure events have been replaced by the ::size-changed signal. Map events have been replaced by the :mapped property, and gdk_event_handler_set() has been replaced by the ::event signal.

The Wayland backend has gained support for the Settings portal for GtkSettings, and uses the text-input-unstable-v3 protocol for its input method support.

GTK

Widgets

One big change for custom widgets is the introduction of GtkLayoutManager, which is a new delegate object that takes over size allocation. Layout managers can optionally use layout children for holding layout properties. This replaces the layout-related child properties in GTK containers such as GtkBox or GtkGrid.

A number of layout managers are available:

  • GtkBinLayout, for simple single-child containers
  • GtkBoxLayout, for children that are arranged linearly
  • GtkGridLayout, for children that are arranged in a grid
  • GtkFixedLayout, for freely positioned and transformed children
  • GtkCustomLayout, as a quick way to turn traditional measure and size_allocate vfuncs into a layout manager

More layout manager implementations will appear in the future. Most prominently, work is underway on a constraints-based layout manager.

GtkAssistant, GtkStack and GtkNotebook have publicly
accessible page objects for their children. The page objects
are also exposed via a list model. They non-layout related child properties of these containers have been converted into regular properties on these page objects.

Since all existing child properties have been converted to regular properties, moved to layout properties or moved to such page objects, support for child properties has been dropped from GtkContainer.

The core GtkEntry functionality has been moved into a new GtkText widget, which also implements an expanded GtkEditable interface. All existing entry subclasses in GTK have been turned into GtkEditable implementations wrapping a GtkText widget. This also includes a new GtkPasswordEntry.

Other Changes

GTK widgets can transform their children using projective linear
transformations. This functionality is available in CSS and
as a GskTransform argument to gtk_widget_allocate. GtkFixed is
the first container that exposes this functionality. For further examples,
see the swing transition of GtkRevealer, the rotate transitions
of GtkStack or the Fixed Layout example in gtk4-demo.

A number of list models have been introduced, for internal use
and as public API: GtkMapListModel, GtkSliceListModel, GtkSortListModel, GtkSelectionModel, GtkSingleSelection. These will become more widely used when we introduce a list model-based GtkListView.

GtkBuilder can specify object-valued properties inline, instead of referring to them by ID, and the simplify command of gtk4-builder-tool has gained an option to automatically convert GTK 3 UI definition files to GTK 4.

Coming soon

For more information on the things that are still still coming for GTK 4, find us on Discourse, IRC, or look here.

Future of relative window positioning

With emerging display server technologies, toolkits sometimes need to adapt how they implement the features they provide. One set of features that needs adaptation is how GTK+ positions popup windows such as menus, popovers and tooltips, so that they will be placed within the work area of the monitor.

In the old days, when GTK+ wanted to position a menu, it would first look up the global position of the parent window of the menu. It would then look up the work areas of all the monitors connected. With the given work areas, the global position of the parent window, and the intended menu position relative to the parent window it wanted to place the menu, GTK+ would use a clever algorithm calculating a reasonable position for the menu to be placed so that it would be visible to the user. For example, if the File menu doesn’t have enough space to popup below the parent menu item, then GTK+ would re-position it above the parent menu item instead.

popup-flip

For various reasons the concept of “global window positions” has been removed from clients in these new display server technologies, meaning we cannot use our clever algorithm in GTK+ any more.

But we still want to make our menus, tooltips, popovers, etc. fully visible to the user expecting to interact with them, so how can we ensure this without knowing anything about where our windows are positioned?

To tackle this in GTK+, we had to solve a number of problems.

  • The positioning logic needs to be moved to GDK, while still allowing GTK+ to affect how the menu positioning behaves should the initially intended position end up outside of the work area.
  • Different GDK backends might do things differently.
  • Some types of windows need to know the position it ended up at, so they can adapt how they draw them self.
  • Some windows simply want to take up as much space as they may (for example a menu with far too many choices should not be taller than the screen).

Last year, William Hua and I started working on bringing GTK+ into the bright future of global position-less menu windows. After having come up with a set of patches doing just that, the discussion of how such an API would actually look like started. After 200-300 comments, we decided we should probably discuss this in person.

Enter GTK2016 in Toronto!

At the hackfest, we got the chance to sit down with a whiteboard in front of us and go through the different use cases, the problems that needed to be solved, how backends would work, and eventually we came up with an API.

william-draws-whiteboard(photo credit: Allison Lortie)

The API we came up with looks as this:

From the GDK side we introduce a new function (so far without any API stability promises; it’s intended to be used only by GTK+ so far) gdk_window_move_to_rect () which takes a set of arguments describing how the application wants its window to be placed in relation to some parent surface. It takes

  • a transient-for window

The parent window it is to be placed relative to an anchor rectangle on the parent window a popup or a menu will often want to be placed in relation to a rectangle on the parent, for example a right-click context menu should expand in some direction from the pixel the pointer was located at when clicking, or a file menu should be placed either below or above the file menu item rectangle on the parent window.

  • a rectangle anchor gravity

Different popup menus might want to open in a certain direction. For example a vertical menu might want to open to the right, while a horizontal menu might want to open downwards.

  • a window anchor gravity

Different popup menus might want to be aligned to the anchor rectangle of the parent anchor rectangle differently. For example, while a combo box might want to expand in a certain direction, it will want to cover the rectangle it expanded from.

  • an anchor hint

Different popup menus want to adjust their positions differently; some will want to expand in different directions from the parent anchor rectangle, some will want to just be slid into visibility, some will want to be resized, while some will want some combination of all the three.

  • an rectangle anchor offset

The offset is simply a nudge factor for a common use case where a popup menu will offset its position relative to an anchor.

By having GTK+ come up with a declarative description of how it wants its menu to be positioned, we allow GDK to implement the actual positioning differently depending on how the display server system is designed. On Mir, a MirSurfaceSpec is created, while on Wayland an xdg_positioner object is created. On X11, Windows and Mac OS X, the backends can use the available global positions as well as monitor work areas and calculate an optimal position just as before.

Application developers are, however, not expected to use this API directly yet. Normally what is wanted is to create a menu, a popover, a combo box, and for this we have introduced a set of parameters and helper functions to make this very convenient. The API consists of a few new properties:

  • GtkMenu:anchor-hints – the positioning strategy.
  • GtkMenu:rect-anchor-dx – horizontal offset to shift window.
  • GtkMenu:rect-anchor-dy – vertical offset to shift window.
  • GtkMenu:menu-type-hint – a window type – this is still needed so that the X11 backend can let the window manager know what type of popup window is being mapped.

and a few more functions:

  • gtk_menu_popup_at_rect () – given the parameters set, popup a menu relative to a given rectangle on a parent window.
  • gtk_menu_popup_at_widget () – given the parameters set, popup a menu relative to a given widget on a parent window.
  • gtk_menu_popup_at_pointer () – given the parameters set, popup a menu relative to where the user just clicked.

With these functions, developers of custom widgets can now position popup menus in a portable manner. So far, GTK+’s own popup menus have already been ported to use these new functions. There is already a basic proof-of-concept in the Mir backend, and a Wayland implementation is in progress.

Head over to the bug to see all the details of how to place your menus in the future.

gnome-sponsored-badge-shadow

GTK+ 2.15.3 unstable release

This is the third development release leading up to GTK+ 2.16.
8 bugs fixed in this release!

  • General:
    • Keyboard shortcut handling has been changed, to help with a longstanding complaint about the way GTK+ handles multiple layouts. GTK+ now only uses keys from groups other than the current group if they are not present in the current group. Feedback on this change is appreciated.

Read the original announcement for more info and downloads.

GTK+ 2.15.2 unstable release

This is the second development release leading up to GTK+ 2.16.
4 bugs fixed in this release!

  • GtkAction:
    • Make toolitems pick up icon names from actions
    • Draw proxies of radio actions properly
    • Make menu proxies of recent actions work
    • Avoid accidental activations when changing actions on proxies
    • Make derived button classes work as proxies
  • Input methods
    • Avoid an assertion due to early use of input methods
  • GtkScale
    • Avoid a segfault in the marker drawing code
  • GtkImageMenuItem
    • Add a property to override the show-menu-images setting

Read the original announcement for more info and downloads.

GTK+ 2.15.1 unstable release

This is the first development release leading up to GTK+ 2.16.
35 bugs fixed in this release!

  • GtkFileChooser:
    • Remember the file chooser’s size across invocations
    • Handle uris that are entered in the entry
    • Improve autocompletion, in particular for uris
  • GtkEntry:
    • New property “im-module” for selecting input methods per-widget
    • New icon-related API got renamed for consistency
    • Added properties and setters for icon tooltips
  • GtkTextView:
    • New property “im-module” for selecting input methods per-widget
    • New signal “paste-done” to allow better handling of async pasting
  • GtkScale:
    • New api to add annotated marks: gtk_scale_add_mark.
  • GtkAction:
    • Rework the way actions and proxies interact, to make the interaction less ad hoc, more extensible, and better suited for support in GUI builders like glade. To be used as a proxy, a widget must now implement the GtkActivatable interface, and GtkActivatable implementations are responsible for syncing their appearance with the action and for activating the action. All the widgets that are commonly used as proxies implement GtkActivatable now. This is a big change, and it is not unlikely to break some current users of GtkAction, so feedback about problems caused by this is appreciated.
    • Add a “gicon” property to specify the icon with a GIcon
  • GDK:
    • On X11, GDK now caches cursors to avoid cursor theming overhead
    • New cursor type for blank cursors: GDK_BLANK_CURSOR
  • New deprecations:
    • gtk_scale_Button_get_orientation()
    • gtk_scale_button_set_orientation()
    • gtk_action_connect_proxy()
    • gtk_action_disconnect_proxy()
    • gtk_widget_get_action()
    • gtk_action_block_activate_from()
    • gtk_action_unblock_activate_from()
    • direct access to “gtk-action” object data
  • Changes that are relevant for translators:
    • Navigation and Media stock labels have separate message contexts now
    • The caps lock warning string has been changed

Read the original announcement for more info and downloads.

GTK+ 2.15.0 unstable release

This is the first development release leading up to GTK+ 2.16.

Overview of changes between 2.14.x and 2.15.0:

  • GtkFileChooser
    • Optionally shows file sizes
    • Mounts volumes when necessary
    • Picks better mime icons
  • GtkEntry
    • Can show icons at either side of the entry, which can be made clickable, drag sources, etc
    • Can show progress information
    • Picks the best available placeholder character for invisible entries unless it is explicitly set. See the invisible-char-set property
    • Input methods work again in invisible entries
    • Invisible entries can optionally display a caps-lock warning. This can be turned off with the caps-lock-warning property
  • GtkStatusIcon
  • GtkLinkButton
    • Respects user-defined tooltips
    • Has a default url hook
  • GtkBuilder
    • Can construct menus
    • Can associate accel groups with windows
    • Child properties can now be translatable, e.g. GtkAssistant::page-title
  • GtkOrientable
    • A new interface implemented by all widgets that have horizontal and vertical variants
    • Printing support
    • Print-to-file can save to non-local files
    • Page rendering can be deferred to a thread to avoid blocking the mainloop
  • GDK
    • GdkKeymap emits a state-changed signal when the Caps Lock state changes
  • Newly deprecated functions:
    • gdk_window_get_toplevels(),
    • gtk_font_selection_dialog_get_apply_button(),
    • gtk_status_icon_set_tooltip(),
    • gtk_toolbar_set_orientation()
  • Changes that are relevant for theme authors
    • The GtkMenu::arrow-placement style property allow more space efficient layout of scrolling menus
    • Submenu arrows can be scaled relative to the font size, with the
    • GtkMenuItem::arrow-scaling style property
    • Themes can set the GtkDialog::content-area-spacing style property to change the spacing between elements of the content area
    • The GtkEntry::state-hint style property can be used to make GTK+ pass the correct state when drawing the background of entries
    • The GtkEntry::prelight style property can be used to suppress prelighting of icons in entries on mouse-over
  • Changes that are relevant for translators
    • GTK+ has been switched to use the two-argument C_() macro instead of Q_() for messages with context

Read the original announcement for more information including bug fixes and download locations.