Media in GTK 4

Showing moving pictures is ever more important. GTK 4 will make it easier for GTK apps to show animations; be that a programmatic animation, a webm file or a live stream.

Everything is paintable

Before looking at animations, it is worth spending a little bit of time on the underlying abstractions that GTK uses for content that can be drawn. In GTK 2 and 3, that was mainly GdkPixbuf: you load a file, and you get a block of pixel data (more or less in a single format).  If you wanted to animate it, there is GdkPixbufAnimation, but it is fair to say that it was not a very successful API.

GTK 4 brings a new API called GdkPaintable that was inspired by the CSS Houdini effort. It is very flexible—anything that you can plausibly draw can be a GdkPaintable. The content can be resizable (like svg), or change over time (like webm).

Widgets that typically show image content, like GtkImage or GtkPicture know how to use paintables. And many things that in the past would have produced pixel data in some form can now be represented as paintables: textures, icons, and even widgets.

If you have more specialized needs, anything that can be captured in a GtkSnapshot can be turned into a paintable with gtk_snapshot_to_paintable(). If you make a custom widget that wants to draw a paintable, that is very straightforward. Just call gdk_paintable_snapshot().

Getting animated

As I’ve said earlier, paintables can change their content over time. All it takes is for them to emit the ::contents-changed signal, and widgets like GtkPicture will do the right thing and update their display.

So, where do we get a GdkPaintable that changes its content? We can load it from a file, using GTK 4’s builtin GtkMediaFile api. This is a high-level api, comparable to GstPlayer: you stuff in a uri, and you get an object that has a play() function and a pause() function, and works as a paintable.

GTK ships with two implementations of GtkMediaFile, one using gstreamer and another one using ffmpeg. Since we don’t want to make either of these a hard dependency of GTK,  they are loadable modules.

You can open the GTK inspector to find out which one is in use:

Keeping control

The GtkMediaFile API is what gtk4-widget-factory demos with the animated GTK logo on its frontpage:

As you can see, it is not just a moving picture, there are media controls there too – you get these for free by using the GtkVideo widget.

Beyond the basics

Loading animations from files is maybe not that exciting, so here is another example that goes a little further. It is a little weekend project that combines GtkVideo, libportal and pipewire to demonstrate how to show a video stream in a GTK application.

The bad news is that we haven’t found a permanent home for the supporting glue code yet (a GstSink, a GdkPaintable and a GtkMediaStream). It doesn’t fit into GTK since, as mentioned, we don’t want to depend on gstreamer, and it doesn’t fit into gstreamer since GTK 4 isn’t released yet. We will certainly work that out before too long, since it is very convenient to turn a gstreamer pipeline into a paintable with a few lines of code.

The good news is that the core of the code is just a few lines:

fd = xdp_portal_open_pipewire_remote_for_camera (portal);
stream = gtk_gst_media_stream_new_for_pipewire_fd (fd, NULL);
gtk_video_set_media_stream (video, stream);

 

Custom widgets in GTK 4 – Actions

(This is the fifth part of a series about custom widgets in GTK 4. Part 1, part 2, part 3, part 4).

Activate all the things

Many things in GTK can be activated: buttons, check boxes, switches, menu items, and so on. Often, the same task can be achieved in multiple ways, for example copying the selection to the clipboard is available both via the Control-C shortcut and an item in the context menu.

Inside  GTK, there are many ways things can proceed: a signal may be emitted (::activate, or ::mnemonic-activate, or a keybinding signal), a callback may be called, or a GAction may be activated. None of this is entirely new in GTK 4, but we are moving towards using GActions as the primary mechanism for connecting actions.

Actions

Actions can appear in various forms in a GTK application.

First, there are global application actions, added to GtkApplication or GtkApplicationWindow (both of these implement the GActionGroup interface). This is where actions first appeared in GTK 3, mainly for the purpose of exporting them on the session bus for use with the app menu.

We also allow to associate actions with widgets by calling gtk_widget_insert_action_group(). Actions that are added in this way are only considered for activation when it originates in below the widget in the hierarchy.

A new way to create actions in GTK 4 is to declare actions in the class_init function, via gtk_widget_class_install_action(), similar to how properties are declared with g_object_class_install_property(). Actions created in this way are available for every instance of the widget.

Here is an example from GtkColorSwatch:

gtk_widget_class_install_action (widget_class,
                                 "color.customize", "(dddd)",
                                 customize_color);

The customize_color function is called when the color.customize action is activated. As you can see, actions can declare that they expect parameters. This is using GVariant syntax; you need to provide four double values.

A convenient shorthand allows you to create a stateful action to  set a property of your widget class:

gtk_widget_class_install_property_action (widget_class,
                                         "misc.toggle-visibility",
                                         "visibility");

This declares an action with the name misc.toggle-visibility, which toggles the value of the boolean visibility property.

Actionables and Menus

Declaring actions only goes so far, you also need to connect your actions to the UI in some form. For widgets like buttons or switches that implement the actionable interface, this is as easy as setting the action-name property:

gtk_actionable_set_action_name (GTK_ACTIONABLE (button),
                                "misc.toggle-visibility");

Of course, you can also do this in a ui file.

If you want to activate your actions from a menu, you will likely use a menu model that is constructed from XML, such as this:

<menu id="menu">
  <section>
    <item>
      <attribute name="label">Show text</attribute>
      <attribute name="action">misc.toggle-visibility</attribute>
    </item>
  </section>
</menu>

In GTK 3, you would connect to the ::populate-popup signal to add items to the context menus of labels or entries. In GTK 4, this is done by adding a menu model to the widget:

gtk_entry_set_extra_menu (entry, menu_model);

Going deeper

To learn more about actions in GTK 4, you can read the action overview in the GTK documentation.