GRI-174: Phase 0 renderer spike - image, video and web page in one GTK window #2

Merged
vivy merged 1 commit from feature/GRI-174-renderer-spike into dev 2026-10-18 09:18:03 +00:00
Owner

GRI-174: Phase 0 renderer spike — image, video and web page in one GTK window

Validates plan/decisions/0007-gtk-gstreamer-and-wry-for-the-renderer.md
before Phase 1 is built on top of it. The roadmap puts this first because a red
spike would change Phase 1.

This pull request was first opened on GitHub as #2, where no CI run ever
started. It is re-opened here because the forge is now the only system of
record (GRI-365).

What is in it

An ItemRenderer trait in crates/client/src/renderer.rs (widget() /
show() / hide()), with one backend per item kind behind it, as
plan/client/renderer.md describes:

File Backend
renderer/image.rs GtkImage, loaded through GdkPixbuf, so a bad file is a reported RenderError and not a blank widget
renderer/video.rs GStreamer playbin into gtksink, which returns a plain GtkWidget
renderer/web.rs wry (WebKitGTK on Linux) built into a GtkBox with WebViewBuilderExtUnix::build_gtk

crates/client/src/bin/renderer_spike.rs is the demonstration: one GTK window
with a GtkStack, swapping between all three on a timer. It generates its own
throwaway image, video and HTML assets, so no binary fixtures are committed,
and prints a timing and VmRSS checkpoint at each step.

No unsafe. ADR 0007 expected the renderer would need it for the C
libraries. gtksink hands back a GtkWidget directly, so no raw window handle
crosses the GStreamer/GTK boundary and none was needed yet.

Also anchors bin/ to /bin/ in .gitignore. It came unanchored from a
leftover C#/.NET template section and was silently swallowing
crates/client/src/bin/, which is Cargo's own convention for extra binaries.

Why the renderer is behind a Cargo feature

The whole stack sits behind a gtk-renderer feature, with required-features
on the spike binary. No CI runner has the GTK3, GStreamer or WebKitGTK -dev
packages, and nothing else in the codebase needs them yet, so
cargo build/clippy/test --workspace — what CI and the default gate run do —
never reaches any of this.

To build and run it:

cargo run -p client --features gtk-renderer --bin renderer_spike

with libgtk-3-dev, libgstreamer1.0-dev plus -plugins-{base,good},
libwebkit2gtk-4.1-dev and gstreamer1.0-gtk3. If Phase 1 needs the stack at
build time in general, that is a CI-configuration change and a separate issue.

What was observed

Run under Xvfb with no GPU or DRI access, so WebKit and any GL path fell back
to software rendering. Screenshots confirmed real pixels rather than only
"reached PLAYING":

  • Image — a generated gradient PNG, through the real GdkPixbuf::from_file
    path.
  • Video — videotestsrc through playbin into gtksink, SMPTE colour
    bars visibly on screen.
  • Web — a generated local HTML page with CSS applied and a <script>
    confirmed executing, then https://example.com, including Arabic
    right-to-left, CJK and Cyrillic text rendering correctly.

What was measured

On an x86_64 developer machine, software rendering, no GPU. One full cycle:
~2.31 s user plus 0.48 s system CPU, peak RSS ~431 MB, steady state ~290 MB.

Per-step VmRSS on a warm run:

Step Delta
GTK and GStreamer init ~94 MB baseline of shared libraries
Image widget +0 MB
Video widget (gtksink) +1 MB
Web widget (WebKitGTK context) +42 MB
First page navigation +125 MB — WebKit helper processes and JS warm-up
Second navigation +8 MB

Widget construction is cheap once the libraries are warm, tens of milliseconds
each. The cost is WebKitGTK's one-time context and first navigation, and it is
memory rather than CPU.

This is not a Pi number. ~290 MB steady state with a ~430 MB transient peak
is a figure to carry into the Pi issue's budget, not a pass or a fail on its
own. The test board has 905 MiB of RAM.

The recommendation — ADR 0007's decision rule

Keep WebKitGTK behind the ItemRenderer trait. It rendered a real
external site correctly, including multi-language text, with no unsafe and no
external browser process, which is what the ADR expected of it. Nothing here
argues for moving URL items to CEF.

Provisional on one point: only one real external site was tried. A JS-heavy or
WebGL-heavy site that the target users actually need is worth a second check
before Phase 1 locks the choice in.

What this does not answer

  • The Raspberry Pi half is a separate issue, and the board now exists. Note
    that its firmware has H.265, VP8, VP9, MPEG-2 and VC-1 decode disabled, so
    video must be H.264 or the measurement is a software-decode measurement.
  • The three display environments. plan/client/renderer.md requires X11,
    Wayland and cage on real hardware. This spike covers X11 under Xvfb only.
  • The off-screen self-test (Q1). Still open. It needs the update-handover
    design, not only the renderer.

Gates

./scripts/local-gates.sh passed with exit 0 and no skips on this tree: git
hooks, cargo fmt, cargo clippy --workspace --all-targets --locked -D warnings, cargo test (37 passed), cargo test --doc, cargo deny check bans licenses sources, and the front-end check and build. cargo deny advisories is CI's own job and does not block a merge, by ADR 0008.

The gates do not compile the renderer itself, because it is behind the
gtk-renderer feature. What they prove is that the branch breaks nothing CI
builds today. What proves the renderer works is the Xvfb run above.

# GRI-174: Phase 0 renderer spike — image, video and web page in one GTK window Validates `plan/decisions/0007-gtk-gstreamer-and-wry-for-the-renderer.md` before Phase 1 is built on top of it. The roadmap puts this first because a red spike would change Phase 1. This pull request was first opened on GitHub as #2, where no CI run ever started. It is re-opened here because the forge is now the only system of record (GRI-365). ## What is in it An `ItemRenderer` trait in `crates/client/src/renderer.rs` (`widget()` / `show()` / `hide()`), with one backend per item kind behind it, as `plan/client/renderer.md` describes: | File | Backend | | --- | --- | | `renderer/image.rs` | `GtkImage`, loaded through `GdkPixbuf`, so a bad file is a reported `RenderError` and not a blank widget | | `renderer/video.rs` | `GStreamer` `playbin` into `gtksink`, which returns a plain `GtkWidget` | | `renderer/web.rs` | `wry` (`WebKitGTK` on Linux) built into a `GtkBox` with `WebViewBuilderExtUnix::build_gtk` | `crates/client/src/bin/renderer_spike.rs` is the demonstration: one GTK window with a `GtkStack`, swapping between all three on a timer. It generates its own throwaway image, video and HTML assets, so no binary fixtures are committed, and prints a timing and `VmRSS` checkpoint at each step. **No `unsafe`.** ADR 0007 expected the renderer would need it for the C libraries. `gtksink` hands back a `GtkWidget` directly, so no raw window handle crosses the GStreamer/GTK boundary and none was needed yet. Also anchors `bin/` to `/bin/` in `.gitignore`. It came unanchored from a leftover C#/.NET template section and was silently swallowing `crates/client/src/bin/`, which is Cargo's own convention for extra binaries. ## Why the renderer is behind a Cargo feature The whole stack sits behind a `gtk-renderer` feature, with `required-features` on the spike binary. No CI runner has the GTK3, GStreamer or WebKitGTK `-dev` packages, and nothing else in the codebase needs them yet, so `cargo build/clippy/test --workspace` — what CI and the default gate run do — never reaches any of this. To build and run it: ``` cargo run -p client --features gtk-renderer --bin renderer_spike ``` with `libgtk-3-dev`, `libgstreamer1.0-dev` plus `-plugins-{base,good}`, `libwebkit2gtk-4.1-dev` and `gstreamer1.0-gtk3`. If Phase 1 needs the stack at build time in general, that is a CI-configuration change and a separate issue. ## What was observed Run under Xvfb with no GPU or DRI access, so WebKit and any GL path fell back to software rendering. Screenshots confirmed real pixels rather than only "reached PLAYING": - **Image** — a generated gradient PNG, through the real `GdkPixbuf::from_file` path. - **Video** — `videotestsrc` through `playbin` into `gtksink`, SMPTE colour bars visibly on screen. - **Web** — a generated local HTML page with CSS applied and a `<script>` confirmed executing, then `https://example.com`, including Arabic right-to-left, CJK and Cyrillic text rendering correctly. ## What was measured On an x86_64 developer machine, software rendering, no GPU. One full cycle: ~2.31 s user plus 0.48 s system CPU, peak RSS ~431 MB, steady state ~290 MB. Per-step `VmRSS` on a warm run: | Step | Delta | | --- | --- | | GTK and GStreamer init | ~94 MB baseline of shared libraries | | Image widget | +0 MB | | Video widget (`gtksink`) | +1 MB | | Web widget (`WebKitGTK` context) | **+42 MB** | | First page navigation | **+125 MB** — WebKit helper processes and JS warm-up | | Second navigation | +8 MB | Widget construction is cheap once the libraries are warm, tens of milliseconds each. The cost is WebKitGTK's one-time context and first navigation, and it is memory rather than CPU. **This is not a Pi number.** ~290 MB steady state with a ~430 MB transient peak is a figure to carry into the Pi issue's budget, not a pass or a fail on its own. The test board has 905 MiB of RAM. ## The recommendation — ADR 0007's decision rule **Keep `WebKitGTK` behind the `ItemRenderer` trait.** It rendered a real external site correctly, including multi-language text, with no `unsafe` and no external browser process, which is what the ADR expected of it. Nothing here argues for moving URL items to CEF. Provisional on one point: only one real external site was tried. A JS-heavy or WebGL-heavy site that the target users actually need is worth a second check before Phase 1 locks the choice in. ## What this does not answer - **The Raspberry Pi half** is a separate issue, and the board now exists. Note that its firmware has H.265, VP8, VP9, MPEG-2 and VC-1 decode disabled, so video must be H.264 or the measurement is a software-decode measurement. - **The three display environments.** `plan/client/renderer.md` requires X11, Wayland and `cage` on real hardware. This spike covers X11 under Xvfb only. - **The off-screen self-test (Q1).** Still open. It needs the update-handover design, not only the renderer. ## Gates `./scripts/local-gates.sh` passed with exit 0 and no skips on this tree: git hooks, `cargo fmt`, `cargo clippy --workspace --all-targets --locked -D warnings`, `cargo test` (37 passed), `cargo test --doc`, `cargo deny check bans licenses sources`, and the front-end check and build. `cargo deny advisories` is CI's own job and does not block a merge, by ADR 0008. The gates do not compile the renderer itself, because it is behind the `gtk-renderer` feature. What they prove is that the branch breaks nothing CI builds today. What proves the renderer works is the Xvfb run above.
GRI-174: renderer spike - image, video and web page in one GTK window
Some checks failed
CI / cargo fmt (pull_request) Successful in 23s
CI / cargo clippy (pull_request) Successful in 1m7s
CI / cargo test (pull_request) Successful in 1m11s
CI / frontend-web (pull_request) Successful in 4m29s
CI / cargo deny advisories (pull_request) Failing after 4m41s
CI / cargo deny (pull_request) Successful in 4m43s
CI / CI (pull_request) Successful in 5s
fc606dcd2d
Validates the ADR 0007 stack (GTK3 + GStreamer/gtksink + WebKitGTK via
wry) behind an ItemRenderer trait with one backend per item kind. The
gtk-renderer feature keeps the GTK/GStreamer/WebKit dependencies out of
the default workspace build, since the CI runner has none of their
system libraries installed.
vivy merged commit 5df4e3f4d0 into dev 2026-10-18 09:18:03 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Grimoirix/grimoire-of-digital-signage!2
No description provided.