Tag: gdk

Future of relative window positioning

With emerging display server technologies, toolkits sometimes need to adapt how they implement the features they provide. One set of features that needs adaptation is how GTK+ positions popup windows such as menus, popovers and tooltips, so that they will be placed within the work area of the monitor.

In the old days, when GTK+ wanted to position a menu, it would first look up the global position of the parent window of the menu. It would then look up the work areas of all the monitors connected. With the given work areas, the global position of the parent window, and the intended menu position relative to the parent window it wanted to place the menu, GTK+ would use a clever algorithm calculating a reasonable position for the menu to be placed so that it would be visible to the user. For example, if the File menu doesn’t have enough space to popup below the parent menu item, then GTK+ would re-position it above the parent menu item instead.

popup-flip

For various reasons the concept of “global window positions” has been removed from clients in these new display server technologies, meaning we cannot use our clever algorithm in GTK+ any more.

But we still want to make our menus, tooltips, popovers, etc. fully visible to the user expecting to interact with them, so how can we ensure this without knowing anything about where our windows are positioned?

To tackle this in GTK+, we had to solve a number of problems.

  • The positioning logic needs to be moved to GDK, while still allowing GTK+ to affect how the menu positioning behaves should the initially intended position end up outside of the work area.
  • Different GDK backends might do things differently.
  • Some types of windows need to know the position it ended up at, so they can adapt how they draw them self.
  • Some windows simply want to take up as much space as they may (for example a menu with far too many choices should not be taller than the screen).

Last year, William Hua and I started working on bringing GTK+ into the bright future of global position-less menu windows. After having come up with a set of patches doing just that, the discussion of how such an API would actually look like started. After 200-300 comments, we decided we should probably discuss this in person.

Enter GTK2016 in Toronto!

At the hackfest, we got the chance to sit down with a whiteboard in front of us and go through the different use cases, the problems that needed to be solved, how backends would work, and eventually we came up with an API.

william-draws-whiteboard(photo credit: Allison Lortie)

The API we came up with looks as this:

From the GDK side we introduce a new function (so far without any API stability promises; it’s intended to be used only by GTK+ so far) gdk_window_move_to_rect () which takes a set of arguments describing how the application wants its window to be placed in relation to some parent surface. It takes

  • a transient-for window

The parent window it is to be placed relative to an anchor rectangle on the parent window a popup or a menu will often want to be placed in relation to a rectangle on the parent, for example a right-click context menu should expand in some direction from the pixel the pointer was located at when clicking, or a file menu should be placed either below or above the file menu item rectangle on the parent window.

  • a rectangle anchor gravity

Different popup menus might want to open in a certain direction. For example a vertical menu might want to open to the right, while a horizontal menu might want to open downwards.

  • a window anchor gravity

Different popup menus might want to be aligned to the anchor rectangle of the parent anchor rectangle differently. For example, while a combo box might want to expand in a certain direction, it will want to cover the rectangle it expanded from.

  • an anchor hint

Different popup menus want to adjust their positions differently; some will want to expand in different directions from the parent anchor rectangle, some will want to just be slid into visibility, some will want to be resized, while some will want some combination of all the three.

  • an rectangle anchor offset

The offset is simply a nudge factor for a common use case where a popup menu will offset its position relative to an anchor.

By having GTK+ come up with a declarative description of how it wants its menu to be positioned, we allow GDK to implement the actual positioning differently depending on how the display server system is designed. On Mir, a MirSurfaceSpec is created, while on Wayland an xdg_positioner object is created. On X11, Windows and Mac OS X, the backends can use the available global positions as well as monitor work areas and calculate an optimal position just as before.

Application developers are, however, not expected to use this API directly yet. Normally what is wanted is to create a menu, a popover, a combo box, and for this we have introduced a set of parameters and helper functions to make this very convenient. The API consists of a few new properties:

  • GtkMenu:anchor-hints – the positioning strategy.
  • GtkMenu:rect-anchor-dx – horizontal offset to shift window.
  • GtkMenu:rect-anchor-dy – vertical offset to shift window.
  • GtkMenu:menu-type-hint – a window type – this is still needed so that the X11 backend can let the window manager know what type of popup window is being mapped.

and a few more functions:

  • gtk_menu_popup_at_rect () – given the parameters set, popup a menu relative to a given rectangle on a parent window.
  • gtk_menu_popup_at_widget () – given the parameters set, popup a menu relative to a given widget on a parent window.
  • gtk_menu_popup_at_pointer () – given the parameters set, popup a menu relative to where the user just clicked.

With these functions, developers of custom widgets can now position popup menus in a portable manner. So far, GTK+’s own popup menus have already been ported to use these new functions. There is already a basic proof-of-concept in the Mir backend, and a Wayland implementation is in progress.

Head over to the bug to see all the details of how to place your menus in the future.

gnome-sponsored-badge-shadow

Drawing in GTK+

The topic of how GTK+ draws the content of a window is a fairly complex one; it involves drilling down from GtkWidget, to GdkWindow, to Cairo, to the windowing system currently in use. This task can seem somewhat daunting, even for people that are familiar with the GTK+ API from an application development standpoint, so I decided to write down a quick introduction of how GTK+ draws, going from widgets, to windows, to surfaces, to native windowing resources.

How it starts

GTK+ always draws because something asked it to. This request may come from the windowing system — for instance, because the window manager presented your application window to the user, or because the user resized it — but more often it’ll come from a widget updating its contents. Let’s say, a progress bar going from 50% to 60%; or a label, changing its text; or a spinner, doing a new iteration. This request invalidates the backing GdkWindow of the widget — which usually it’s the GdkWindow of the top-level GtkWindow that contains the widget. Each invalidation carries with itself the region of the window to be invalidated (the “damage”), so that when we get to actually drawing, we know which parts of the window need to be updated, and we can avoid drawing outside of the damaged areas.

Race the clock

The first invalidation will start the “frame clock”; this clock is an object that keeps track of each phase inside a frame, like painting windows, laying out widgets, or processing the event queue. This allows GTK+ to be synchronized to things like the windowing system compositor, and to avoid performing unnecessary work that won’t be seen by the user — for instance, drawing something at 1000 frames per second when your display can only run at 60 Hz.

Once the clock reaches the “paint” phase, we process all the scheduled updates on a window; this will cause a GDK_EXPOSE event to be emitted. The GDK_EXPOSE event contains the GdkWindow that needs to be updated, and the union of all the invalidated areas. It’s important to note that, by and large, only top level windows will receive a GDK_EXPOSE event; for historical reasons, though, some widgets may apply a particular event mask that will cause GDK_EXPOSE events to be delivered to them as well. You should not write code that depends on that, and if you have legacy code ported from older versions of GTK+ 2.x you should really consider dropping the GDK_EXPOSURE_MASK from the event mask.

Rendering

GTK+ takes the window and invalidated region out of the GDK_EXPOSE event and figures out which top level widget they belong to. Once that’s found, GTK+ will begin the actual rendering process. First of all, GTK+ will ask the GdkWindow to create a buffer where to draw the contents of the window; the buffer is going to be clipped to the region that needs to be drawn, and will be cleared with the background color of the window. GDK will create a “drawing context” — a transient object that keeps track of things like OpenGL and Cairo drawing. Then, GTK+ will ask the widget to draw itself using a Cairo context. For leaf widgets this means drawing themeselves on that context; for container widgets, this additionally means recursing through all their children. At the end of this process, GTK+ will end the frame by telling GDK to take the buffer that contains all the rendered widgets and use it to replace the current contents of the window. GDK will then ask the windowing system to present the window to the user, whenever it’s more appropriate.

Changing History

The process outlined above has various caveats, and the code that deals with invalidation and validation of windows inside GDK is fairly complex; it also has a long history, which means that its API is littered by the headstones of ages past.

Before GTK+ 3.0, for instance, you were supposed to handle the “expose” events yourself, and create a Cairo context to draw on a widget by using gdk_cairo_create(); this has long since been unnecessary, because the GtkWidget::draw virtual function already provides us with a Cairo context with which to draw. The gdk_cairo_create() function, though, has been deprecated in GTK+ 3.22, and should not be used in newly written code; if you need a Cairo context you should create a similar Cairo surface, call cairo_create() on it, and then use the surface as the source for the Cairo context that GTK+ provides to you when drawing a widget. On the other hand, if you were using gdk_cairo_create() to draw on a top-level, native GdkWindow in response to a GDK_EXPOSE event then you should use the newly added gdk_window_begin_draw_frame(), gdk_window_end_draw_frame(), and GdkDrawingContext API instead.

Shaping Future

The internals of the drawing code in GTK+ have been progressively updated over the years, to cope with things like new windowing systems, as well as other rendering API. It’s fairly certain that they will change again, especially when it comes to improving the rendering performance. Many of the changes that may seem arbitrary are, in reality, stepping stones towards reducing the time spent inside the toolkit in each frame, and leave more time to the application logic.