Media in GTK 4

Showing moving pictures is ever more important. GTK 4 will make it easier for GTK apps to show animations; be that a programmatic animation, a webm file or a live stream.

Everything is paintable

Before looking at animations, it is worth spending a little bit of time on the underlying abstractions that GTK uses for content that can be drawn. In GTK 2 and 3, that was mainly GdkPixbuf: you load a file, and you get a block of pixel data (more or less in a single format).  If you wanted to animate it, there is GdkPixbufAnimation, but it is fair to say that it was not a very successful API.

GTK 4 brings a new API called GdkPaintable that was inspired by the CSS Houdini effort. It is very flexible—anything that you can plausibly draw can be a GdkPaintable. The content can be resizable (like svg), or change over time (like webm).

Widgets that typically show image content, like GtkImage or GtkPicture know how to use paintables. And many things that in the past would have produced pixel data in some form can now be represented as paintables: textures, icons, and even widgets.

If you have more specialized needs, anything that can be captured in a GtkSnapshot can be turned into a paintable with gtk_snapshot_to_paintable(). If you make a custom widget that wants to draw a paintable, that is very straightforward. Just call gdk_paintable_snapshot().

Getting animated

As I’ve said earlier, paintables can change their content over time. All it takes is for them to emit the ::contents-changed signal, and widgets like GtkPicture will do the right thing and update their display.

So, where do we get a GdkPaintable that changes its content? We can load it from a file, using GTK 4’s builtin GtkMediaFile api. This is a high-level api, comparable to GstPlayer: you stuff in a uri, and you get an object that has a play() function and a pause() function, and works as a paintable.

GTK ships with two implementations of GtkMediaFile, one using gstreamer and another one using ffmpeg. Since we don’t want to make either of these a hard dependency of GTK,  they are loadable modules.

You can open the GTK inspector to find out which one is in use:

Keeping control

The GtkMediaFile API is what gtk4-widget-factory demos with the animated GTK logo on its frontpage:

As you can see, it is not just a moving picture, there are media controls there too – you get these for free by using the GtkVideo widget.

Beyond the basics

Loading animations from files is maybe not that exciting, so here is another example that goes a little further. It is a little weekend project that combines GtkVideo, libportal and pipewire to demonstrate how to show a video stream in a GTK application.

The bad news is that we haven’t found a permanent home for the supporting glue code yet (a GstSink, a GdkPaintable and a GtkMediaStream). It doesn’t fit into GTK since, as mentioned, we don’t want to depend on gstreamer, and it doesn’t fit into gstreamer since GTK 4 isn’t released yet. We will certainly work that out before too long, since it is very convenient to turn a gstreamer pipeline into a paintable with a few lines of code.

The good news is that the core of the code is just a few lines:

fd = xdp_portal_open_pipewire_remote_for_camera (portal);
stream = gtk_gst_media_stream_new_for_pipewire_fd (fd, NULL);
gtk_video_set_media_stream (video, stream);