Tag: hackfest

GTK Hackfest 2020 — Roadmap and accessibility

Between January 28th and January 31st, the GTK team held what’s now the third hackfest in Brussels.

The main topics of the hackfest were:

  • the schedule for the next development snapshot of GTK4
  • the missing features blocking the release of GTK 4.0
  • the current state of the accessibility support in the toolkit

The first two items occupied the most of the first two days of the hackfest; you can read the GTK 3.98 release announcement for what we’ve been working on for the past 300 days since the 3.96 release. The missing features are:

  • Event controllers for keyboard shortcuts
  • Movable popovers on Wayland
  • Row-recycling list and grid views
  • Animation API

and all of them are being worked on in topic branches. The keyboard shortcuts branch has recently been rebased, and it’s in the process of being documented and cleaned up; the movable popovers is also being reviewed after a few iterations. The last two remaining branches are fairly sizeable, and will require some more iterations to get them right—with the animations API currently being mostly a prototype.

The final topic of the hackfest was the largest, and was a discussion long overdue.

GTK’s accessibility support was added as part of the GTK 2.0 release by the Sun Accessibility Team; it depends on the abstract data types provided by ATK (the Accessibility Tool Kit), which are then implemented concretely in GTK classes like GtkWidgetAccessible, or GtkEntryAccessible. Each widget has an “accessible” object associated to it, which is either automatically created by GTK, or can be provided by application code when subclassing a GTK widget. Non-widget types can also have accessible objects associated to them—the most notable case is the set of cell renderers for tree views and combo boxes. Underneath it all, sits AT-SPI, a protocol that is used by AT—Accessible Technologies, like a screen reader—to consume the data provided by applications. Typically, ATs will use a library like libatspi to deal with the protocol itself.

The main issues with the existing stack are:

  • there’s a lot of indirection caused by the existence of ATK; any new feature or bug fix needs to be defined inside ATK and then implemented into GTK and libatspi
  • ATK was written in a very different environment, and while it has seen a few deprecations, it shows its age in the assumptions it makes—like global coordinate spaces—and in its design
  • there’s a certain overlap between AT requirements and requirements for GUI testing that end up creating friction in the API design
  • the stack has fell in disrepair since the Sun accessibility team was disbanded; most of the ongoing work is still pretty much happening in the AT space (like Orca) and in web browsers
  • the entire stack was written when CORBA was a thing, and then ported to DBus in time for GNOME3; the protocol, though, is not really efficient and requires lots of roundtrips to move around small amounts of data, instead of having bulk operations and notifications

The last point is also the reason why we need a separate accessibility bus in order to avoid spamming the session bus, and making everything slower as soon as the accessibility support is enabled. A separate bus means that we need to poke an additional hole in any sandbox, and still lets everything that connects to the accessibility bus potentially snoop into what happens in every application.

Finally, GTK only supports accessibility on Linux; there is no support for macOS or Windows, which means applications written in GTK and ported to other platforms are not accessible to ATs there. As we expose ATK in our API, adding support for accessibility features on other platforms would require bridging ATK, creating further complexity.

As we want to redesign and update the accessibility features in GTK4, we need to understand what are the requirements for existing consumers of the accessibility stack, and what kind of use cases we need to target. For that, we asked Hypra, a company dedicated to the development of accessible solutions based on free and open source software, to help us.

Hypra developers are familiar with GNOME, and have been working on the Linux accessibility stack. Their clients cover a wide gamut of accessibility users, so they are in the best position to describe what kind of ATs are in actual use on a day to day basis.

There are a wide range of tools and functionality that have to be provided by different layers of the stack, from the toolkit to the compositor; application developers must also have access to the tools necessary to provide proper support to ATs, as they have a much better idea of what their applications should look and behave than the toolkit.

Over the course of two days we have identified a plan for moving forward:

  • drop ATK from the stack, and have GTK talk the AT-SPI protocol directly; this is similar to what Qt does from the toolkit side, and it makes it easier to both expand and verify eventual protocol changes
  • clean up the AT-SPI protocol itself, updating it where needed when it comes to using DBus more efficiently
  • drop the global accessibility bus, and have ATs negotiate a peer-to-peer connection to each application
  • make ATs ask the compositor to gather global state, like key shortcuts, instead of talking to applications that would then have to ask the windowing system—if that’s possible—or return invalid data when it isn’t
  • decouple GUI testing from accessibility
  • write widget and application authoring guides for application developers, and provide validation tools that can be used as part of the build and CI process to check if UI elements have the correct accessible description and links

There are more information available on the wiki for the notes and the roadmap, and we have already scheduled an additional check point meeting for this summer.

There’s a lot of work to be done, but we have now a much clearer idea of the scope and deliverables for such a redesign. If you want to help making things happen faster, feel free to join the effort; you can also make a donation to the GNOME Foundation.

The GTK team would like to thank the GNOME Foundation for the sponsorship for the venue and the attendees, and the fine folks at Hypra for joining the hackfest and explaining use cases and the current state of the accessibility stack, as well as helping out on the development side.

GTK BoF at Guadec

As every year, we had a GTK BoF at Guadec in Thessaloniki. This year, we had a pretty good turnout — everybody was interested in GTK4 plans.

But since Emmanuele was busy in the morning, we started the discussion with some other topics.

GLib

We collected a few suggestions for useful GLib additions from the room.

  • API for OS information (basically, the data that is in /etc/os-release). This seemed uncontrolversial; Robert is going to implement it
  • An ordered map. This is implemented ad-hoc in many places by combining a hash table with a list or array.  There seemed to be agreement that it would be worthwhile to provide this in GLib, if somebody does the work to propose an API

The discussion of ordered maps also touched on generic container interfaces; Philipp described how that could be done, see details here.

Still on the topic of containers, Alex described a problem with transfer annotations. We discussed various ideas, but there may not be a perfect solution.

Matthias pointed out that there is still some Unicode data in Pango. We briefly discussed how nice it would be to have an agreed-on, mmappable binary format for Unicode data, so that everybody could share it. Short of that, moving the last bits of data to GLib was uncontroversial.

Dark mode

Since the “dark mode” BoF joined us, we switched to discussing dark mode next. There was a more discussion of this topic in the vendor theme BoF the next day; the GTK discussion focused on technical details of how to implement dark mode.

There are various options:

  • Add extra metadata to theme index files to mark themes as dark
  • Add a “dark-theme-name” setting and treat dark and light themes as independent
  • Keep the existing convention of appending “-dark” to theme names to find the dark variant of a theme

The pragmatic solution of keeping the existing convention seemed to have support in the room. Matthias started exploring some application support APIs here.

GTK

Eventually we swiched to talking about the state of and progress towards GTK4. The high-level summary is that there is still a list of  features that need to be completed for GTK4:

  • A scalable list view that recycles row widgets. This includes a broader switch to using list models in more places. To make it complete, it should also include a grid view using the technologies. Benjamin is working on this
  • Infrastructure and APIs for animations. This will be similar to the way animations work in CSS, and part of the work is to port not just our existing CSS animation support, but also stack switching animations, the revealer, progress bars and spinners to the new framework. Emmanuele is working on this.
  • Complete the menu/popover rework. Some people tried the new popover menubar. The feedback was that we should probably go back to nesting submenus, at least for menubars, and push forward with dropping menus, since some of the ways in which menus are special (such as keep-up triangles, scrolling) are hard to keep working (or keep working well). Matthias is getting back to working on this after Guadec.
  • Shortcuts – replace mnemonics, accelerators, and key bindings with event controllers. There is a fairly complete branch with code and APIs that several people have worked on; help with reviewing and testing it would be appreciated.
  • The new Drag-and-Drop API needs to be completed.

The good news is that this list is fairly short and has names next to most items. The bad news is that each item is a considerable amount of work. Therefore, it is not a good idea to promise a tight timeline towards the 4.0 release before we have all of them merged. Thus, the following is tentative, but (we hope) somewhat realistic:

  • another GTK 3.9x snapshot before the end of this year
  • a feature-complete 3.99 release around the same time as GNOME 3.36 in spring 2020
  • a 4.0 release around the same time as GNOME 3.38 in fall  2020

Inevitably, we also discussed other things that would be nice to have. None of these are on the GTK4 roadmap; but if somebody shows up to do the work, they can happen:

  • A “widget repository” or “hig” library in order to not overload GTK with too specific or experimental widgets
  • A “UI designer” widget. This could live in a separate library as well
  • Better support for split headerbars and state transitions

We also discussed things outside GTK proper that will keep applications from porting to GTK4. This includes commonly used libraries such as GtkSourceView, vte and webkitgtk, which all will need GTK4 ports before applications that depend on them can be ported. Some of this work is already underway; but any help in this area is appreciated!

Another potential blocker for GTK4 porting is platform support. The GL renderer works well on Linux; the Vulkan renderer needs some fixups. On Windows we currently use cairo fallback, which may be good enough for 4.0. Alternatively, we could merge existing work for using the GL renderer with ANGLE. The situation is less pleasant on OS X, where we don’t have a working backend; if you want to help us here, the first still would be to adapt the GDK backend to changes in GDK.

Hacking time

In the afternoon, the room drifted from discussion to hacking, and various GTK-related works-in-progress could be spotted on peoples laptops: work to speed up GtkBuilder template loading, nested popover menus, a half-finished GtkSourceView port.

You will hopefully see these (and others) in GTK master soon.

Report from the GTK hackfest in Brussels

Thanks to the GNOME Foundation, various GTK developers were able to meet in Brussels right after FOSDEM, for one of our yearly hackfests.

The main topics of the hackfest were:

  • recap the work that landed into the master branch in the past 6-12 months, in order to have everyone on the same page
  • discuss the features still in flight in separate branches, assess their state of completion, and identify blockers
  • figure out what are the blockers for the first release of GTK 4.0

Hackfests allow us to have this kind of discussions with a large bandwidth at our disposal, compared to online communication channels, so they are very important for the project.

You can see the full agenda on the wiki, and we’ll make sure to write articles on the biggest items on it.

The largest items of the discussion were the introduction of new list models and list/grid view widgets; a unified key handling API; the decoupling of layout management policies from containers, and the introduction of constraint layout management; the possibility of merging widgets from libhandy, to allow for writing applications responsive to form factor changes; the switch to a purely declarative menu description API, and the removal of public menu widgets; adding 2D and 3D transformations to GtkWidget; implementing an animation API that applications can consume.

  • list models and list/grid widgets — we’d really like to retire GtkTreeView and GtkIconView, but the existing replacements, GtkListBox and GtkFlowBox, are not performant enough when scaling to very large and dynamic data sets. We need better data storage types, that can be composed to perform operations such as mapping, filtering, and sorting, but can also avoid iterating over all the elements when sizing and drawing widgets. Benjamin Otte already added various models to GTK, and is working on a list and a grid view widgets that can efficiently display their contents. Benjamin and other GNOME application developers are in the process of identifying various stakeholders for  a separate hackfest specifically for gathering more requirements and getting feedback on the new API.
  • unified key handling API — now that we moved all our pointer and touch input handling away from events and towards gestures, we want to do the same for key handling, like key bindings, mnemonics, and accelerators. The overall design is based on triggering actions, and allow introspection of all the “shortcuts” currently available to the GTK inspector, for ease of debugging. There is a development branch already available.
  • layout managers — in GTK 3, layout is imposed by containers on their children; we want to be able to decouple that from widgets and move it into a separate delegate objects hierarchy. Layout managers allow us to reduce the complexity of writing new widgets; they keep the layout code in a separate, non-derivable type; and they allow us to simplify the toolkit internals to the point that we might even make GtkWidget and instantiable type in the future. Layout managers are the first step towards adding constraint-based layout management to GTK, which do away with nesting boxes to create complex UIs. There is a development branch already available. For more information on constraint layouts, you can see the Emeus experimental library for GTK 3.
  • merging widgets from libhandy — Adrien Plazas gave an overview of what’s currently provided by libhandy, and what would be useful to have straight from GTK4 in the future. We discussed reactive layouts, and the ability express sizing with percentages, as well as possibly using constraints to get similar results.
  • declarative menus — GTK has iterated over different menus API over the years; from building menus out of widgets, to GtkUIManager, to GtkBuilder, to GMenu; we also moved to declaring the behaviour of pop up menus, in order to have the windowing system display them more accurately without exposing global coordinates. There’s a lot of overlap, but no clear winner, mostly because we still allow using widgets to build application menus and context menus. Fully switching to declarative style menus, adding new API to make them more expressive, and making GtkMenu and friends private implementations for the toolkit, would allow us to get things like being able to inspect all menus, even out of process; menus manipulable by plugin systems without necessarily creating widgets and keeping track of them; avoiding positioning bugs. There is a full strawman proposal available on the wiki, and Matthias Clasen is working on switching context menus to GMenu in a development branch.
  • widget transformations — Sadly, Timm Bädert couldn’t make it to the hackfest, but we’ve been reviewing his development branch that adds 2D and 3D transformations to GTK widgets, and we’re very excited about it.
  • animations — one last thing we’d like to land for GTK4 is an animation framework for GTK widgets to replace the current generic “frame tick callback”. The model for it is the Clutter explicit animation API, which in turn was based on Core Animation and CSS3 transitions. This work is still in the design phase, but you can expect development branches for it to land soon.

Aside from the big topics, we also discussed various smaller ones:

  • improving performance and memory use; we want to expose the SysProf counters during the frame clock phases, so we can easily identify problems.
  • improving the test suite, especially when it comes to reporting failures; right now, we have to go through the CI failure log, but we’d like to publish proper reports using the GitLab CI infrastructure
  • replacing child properties with real GObject properties on ancillary objects, especially for layout managers; would make documentation, introspection, and usage clearer.
  • finishing the drag and drop rework, to get a more modern API.
  • adding a top-level interface for “window-like” objects—such as windows, dialogs, popovers, menus/popups—useful for establishing common behaviour, and removing hacks and complexity in GtkWindow.

And, finally, yes: we did remove the “plus” from GTK. ;-)

A report from the Guadec GTK+ BoF

The GTK+ team had a full day planning session during the BoF days at Guadec, and we had a full room, including representatives from several downstreams, not just GNOME.

We had a pretty packed agenda, too.

GTK+ 3

We started out by reviewing the GTK+ 3 plans that we’ve outlined earlier.

In addition to what was mentioned there, we also plan to backport the new event controllers, to make porting to GTK+ 4 easier. We will also add meson build support to help with Windows builds.

The 3.24 releases will effectively be a continuation of the 3.22 branch and should be entirely safe to put out as stable updates in distributions.

We plan to release GTK+ 3.24.0 in time for GNOME 3.30.

GTK+ 4 leftovers

The bulk of the day was taken up by GTK+ 4 discussion. We’ve reviewed the list of leftover tasks on the roadmap:

  • Finish DND: Gestures on the GTK+ level, local shortcuts
  • Introduce GtkToplevel and cleanly support popovers
  • Add transformations
  • Create a shortcuts event controller to replace key bindings
  • Port GtkTextView to render nodes
  • Profile the cairo backend, make sure its performance is on par with GTK+ 3
  • Port various dependent libraries:
    • vte
    • webkit
    • libchamplain
    • gtk-vnc
    • gtk-spice

Most of these tasks have names next to them, but if you want to help with any of these tasks, by all means, contact us!

Noticeably absent from this list are a few things that were on the roadmap before:

  • Constraint-based layout (emeus)
  • Shader compiler and application provided shaders
  • Designer support

All of these can still happen if merge requests appear, but we don’t think that we should block on them. They can be developed externally to GTK+ 4, and become GTK+ 5 material.

GTK+ backends

We spent some time evaluating the state of GDK backends in GTK+ master.

The Windows backend is in OK shape. We have several people who help with maintenance and feature development for it, meson makes building it a lot easier, and we have ci for it.

The Quartz backend is in a much worse state. It has not been kept in buildable shape, nobody is providing fixes or feature development for it, and we don’t have ci. We had a macbook offered that could be used for ci, and it was suggested that we could use travis ci for the OS X.

GTK+ timeline

We spent a long time on this, and did not reach a 100% consensus, but it seems realistic to aim for a GTK+ 4 release in spring of 2019, if we keep making good progress on the outstanding leftovers.

When we release GTK+ 3.96, we will also announce a date for GTK+ 4.0. We hope to be able commit to release before GNOME 3.32, so GNOME application developers can switch their master branches to GTK+ 4 without worrying about whether that will disrupt other development for 3.32.

Application porting

We really want feedback from application ports at this point. But we are in a bit of a difficult position, since we can’t plausibly claim to be done with major API work until the GtkToplevel and shortcuts controller work is done.

Our recommendation to app authors at this point is:

  • If you are a bit adventurous, do a port to 3.94 on a branch. It should be possible to keep it working without too much work during the remainder of GTK+ 4 development.
  • If you are not quite as adventurous, wait until 3.24 is released, use it to prepare your port, and port to GTK+ 3.96.
  • Either way, please make your port available to users for testing, either as a regular release, or as a Flatpak with a bundled GTK+.

GLib diversion

In the afternoon, we spent a while talking about GLib. We went over a laundry list of larger and smaller items. Notable highlights: GProperty may happen for 2.60 and we may be able to use g_autoptr soon.

Other ideas

We discussed a great number of other things that we could and should do.

For example, it was suggested (and generally agreed to) that we should merge gsk into gdk, since it is small and the internals are somewhat intertwined. It was also suggested to create subdirectories in gtk/, for example for the css machinery.