linebender / xilem Goto Github PK

Masonry introduces a lot of complex concepts compared to other GUI frameworks. Many of these concepts are intended to reduce future complexity, but crate users don't get those benefits if they can't use the API in the first place.

Masonry should have a tutorial integrated directly into the documentation with rustdoc.

The tutorial should have the following chapters:

• Getting started. The steps to creating a "TODO" app.
• Creating a Widget. Present the Widget trait, the different methods, the create_widget macro, how to do tests.
• Creating a Container Widget. Present WidgetPod, container invariants, etc.
• Widget statuses. Hot, active, focused, stashed.

Other parts of the documentation should refer to these tutorials.

See here:

Running on M1 Mac in release mode.

What's the performance bottleneck here? How can this be fixed?

I was experimenting with xilem (see #14 (comment)) and noticed that the memory and CPU consumption of xilem was exceptionnally high:

Now, I'm totally aware that xilem is in the experimental state, but surely it shouldn't ever consume that much, right?

I didn't properly investigate the CPU consumption yet, but as for the memory consumption, I used heaptrack to do some heap profiling. It's the first time I do something like this so I was quite amazed at how quickly I arrived to a conclusion:


There are 2 huge culprits of memory consumption, one taking ~874 MB and the other taking ~894 MB. It turns out, both come from parley::FontCache::new().

I looked at the code inside parley, and it seems that it's loading every single font from my system into memory, which is less than ideal...
I'm of course going to forward this issue to parley, but in the mean time, is there any simple workaround for xilem so that it doesn't take 2 gigs of ram just for experimentation?

Here's the .zst file that you can open with heaptrack_gui (I think it only works on Linux).
heaptrack.xilem.44968.zip

Looking at the rustdoc documentation my thoughts are the following:

• The example on the main page is too long ... either shorten it, or move it into a submodule or just reference the examples in the examples/ directory.

• The structs and enums in the top-level namespace are all rather randomly jumbled together ... making it very hard to grasp the API from looking at its documentation. I think it would be better to make modules public rather than pub useing all these structs / enums in lib.rs.

• Seeing things like InternalEvent and InternalLifeCycle makes me question why these are exposed in the public API in the first place ... in any case they should probably not be in the top-level scope.

• In general I think the code could use some more structure ... e.g. the command and event modules are currently on the same level, begging the question what the difference between commands and events is. Looking into it Command appears to be part of an Event variant, so I think it would make sense to move the command module under the event module (same with the mouse module which currently only contains the MouseEvent enum).

One thing that came up in the review of #26 is that the Widget trait now depends on Message, which is conceptually at the Xilem level. That's arguably a layering violation, and may make life more difficult for other possible reactive layers on top of the widget set.

The most flexible thing to do would be to make the Widget trait generic on the message type, but that would add a lot of type noise. The Message type is perhaps an acceptable type to bring into the widget layer, as it's basically just a Vec<Id> and a Box<dyn Any>.

Part of the reason I'm writing this issue is to get feedback about other potential uses of the widget layer. Is just having widgets produce Message good enough, or do we really need the type to be more flexible? Accommodating other reactive layers than Xilem feels a bit YAGNI right now, and perhaps we want to do a more careful review of layering if and when we get to that point.

Hi! I wanna start contribute to xilem and wanted to know if there is a roadmap / board for what needs to be done and what would be easy and generally how to work with this code structure and what the goals of the project are. Also: is there a discord server? I read that there's a zulip thing but i never worked with it and it seems more like another issue tool instead of active communication

Trying to build an example you get this error

error[E0609]: no field `inner` on type `&AppRootInner`
   --> /Users/weethet/.cargo/registry/src/index.crates.io-6f17d22bba15001f/masonry-0.1.2/src/app_root.rs:846:14
    |
846 |         self.inner.borrow().app.hide()
    |              ^^^^^ unknown field
    |
    = note: available fields are: `app_handle`, `debug_logger`, `app_delegate`, `command_queue`, `action_queue` ... and 8 others

I suppose this is due to maintainers not having macOS laptop, so it's hard to check, but maybe github actions could bes set up to prevent this any further

The event model of Masonry still needs a lot of improvement.

(I'm including lifecycle here. Lifecycle events are kept separate from Events, but it's unclear if they're actually different, and the code handling them is mostly the same)

The event workflow

Container widgets are expected to call the event method on each of their children. This often doesn't make sense because an event will be targeted directly to a specific widget, and most of those children don't need to be called. And this creates fragility if containers forget to recurse in some situations.

The WidgetPod has a lot of special-case machinery to handle specific events, which makes it pretty hard to read and understand.

Command-type events

Commands, Notifications and Promises are integrated into events, and have their own problems:

• Notifications and Promises are single-target, and the target may want to take ownership of their payload.
• Some Commands are single-target, some commands are broadcast. There's no type-level difference between the two.

Actions

Widgets can also send Actions. Actions are a centralized queue of messages intended for the root application, mostly for a Panoramix-like framework to handle. They are inspired by Facebook's Flux architecture. Each Action represents a semantically meaningful event from the UI (eg "button clicked").

Actions are currently under-designed. Right now, they only store the emitter's id, plus a payload among a small set of types with Box<dyn Any> as a wildcard. Ideally, I'd want a way to connect the type of an action from the widget that sent it, so that each widget would have an associated action type. This should probably be co-designed in parallel with Panoramix.

Changes I'd like to make

Container widgets should implement a recurse() method that would call an FnMut param on each of their children. They could also have specialized recurse_at_pos and recurse_at_id methods for better targeting.

Instead of the workflow being "WidgetPod::on_event calls Widget::on_event calls WidgetPod::on_event calls Widget::on_event etc...", the workflow would be more like "framework calls Widget::recurse which calls Widget::recurse etc... all of which call Widget::on_event. Widgets would never be expected to call WidgetPod::on_event.

Some events might still follow a browser-like model where parent widgets can intercept events from their children; if we implemented that, it would have to be explicit intercepting, not just the parent choosing not to call .on_event on a child.

Lifecycle events might be merged with regular events, unless they prove to have a distinct niche.

Users should be able to send commands from any Widget and from Delegates, to any other Widgets, to windows, and to delegates (?). Delegates should also be able to intercept notifications (?).

Target::Auto should be removed.

Other thoughts

• https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-command_system-md
• https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-event_methods-md

As discussed on Zulip.

Masonry should have infrastructure similar to rustc's to measure the evolution of build times.

While at the moment I'm writing this, short build times are far from a priority, on the long term they will be an essential part of the development experience. Short build times mean short edit-build-get-feedback cycles, something we're definitely aiming for.

Some indicators to consider:

• Time for clean build (dependencies and all).
• Time incremental build.
• Time for incremental build for someone building a "hello world" app on top of masonry.
• Time for incremental build for someone building a reasonably complicated app on top of masonry.
• Time for incremental build for someone building an artificially large app on top of masonry.

The proposals here came from me looking into what it would take to integrate Taffy layout into Xilem. But nothing proposed here is really specific to the CSS style layout modes (Flexbox and CSS Grid) that Taffy implements. Nor would they commit Xilem to CSS style layout. Rather, I believe they would enable Taffy layout modes to implemented in Xilem as widgets (which could live in an external crate), in much the same way that the existing Flex widget is implemented in Druid.

I suspect that we could make a much more streamlined system if support for associating arbitrary data (e.g. "styles") with elements such that a parent widget could access them on a child widget the chidl widget having to add support for them was implemented (ala linebender/druid#2207). But that's a much more significant change, which I think can wait.

I have also written a prototype integration of Taffy with Iced (Iced also uses a similar layout mechanism to Druid and Xilem). And despite having to work around some limitation of Iced's system (like no measure method, and layout taking &self rather than &mut self), the integration actually ended up being relatively straightforward (you can see the implementation of Iced's layout method here (calling into Taffy from Iced), and the implementation of Taffy's perform_child_layout method here (calling back into Iced from Taffy)).

Review of Existing Systems

Here I lay out the state of things as they are in Xilem, Druid, and Taffy.

Prerequisite Type Defintions

A Size<T> in Taffy is defined as:

struct Size<T> {
    width: T,
    height: T,
}

A Size in Druid/Xilem (kurbo) is a Size<f64> using the above definition. For the remainder of this post I will translate this to Size<f64> in the function signatures below for clarity.

A BoxConstraints in Druid is defined as:

struct BoxConstraints {
    min: Size<f64>,
    max: Size<f64>,
}

An AvailableSpace in Taffy is defined as:

enum AvailableSpace {
	MinContent,
	MaxContent,
	Definite(Size<f32>),
}

A SizingMode in Taffy is defined as:

enum SizingMode {
	ContentSize,  // Size ignoring explicit styles
	InherentSize, // Size including explicit styles
}

Xilem's Existing Layout System

/// Compute intrinsic sizes.
/// The returned sizes are (min_size, max_size)
fn measure(&mut self, cx: &mut LayoutCx) -> (Size<f64>, Size<f64>);

/// Compute size given proposed size.
fn layout(&mut self, cx: &mut LayoutCx, proposed_size: Size<f64>) -> Size<f64>;

Druid's Layout System

/// Max intrinsic/preferred dimension is the dimension the widget could take, provided infinite constraint on that axis.
/// Intrinsic is a *could-be* value. It's the value a widget *could* have given infinite constraints. This does not mean the value returned by layout() would be the same.
/// This method **must** return a finite value.
fn compute_max_intrinsic(
	&mut self,
	ctx: &mut LayoutCtx,
	axis: Axis,
	bc: &BoxConstraints,
	data: &T,
	env: &Env,
) -> f64

/// For efficiency, a container should only invoke layout of a child widget
/// once, though there is nothing enforcing this.
fn layout(
	&mut self,
	ctx: &mut LayoutCtx,
	bc: &BoxConstraints,
	data: &T,
	env: &Env
) -> Size;

Taffy's Layout System

fn measure_size(
    tree: &mut impl LayoutTree,
    node: Node,
    known_dimensions: Size<Option<f32>>,
    parent_size: Size<Option<f32>>,
    available_space: Size<AvailableSpace>,
    sizing_mode: SizingMode,
) -> Size<f32>

fn perform_layout(
    tree: &mut impl LayoutTree,
    node: Node,
    known_dimensions: Size<Option<f32>>,
    parent_size: Size<Option<f32>>,
    available_space: Size<AvailableSpace>,
    sizing_mode: SizingMode,
) -> Size<f32>

Analysis

Trivial Differences

There are a few difference which look like they might be important, but I suspect that they are actually not:

• Druid/Xilem use f64 and Taffy uses f32. Perhaps @raphlinus can comment on if/why he thinks f64 is needed, but in any case we can trivially convert between the two types (accepting the loss of precision), or if it came to it, it would be simple enough (if verbose) to extend Taffy to work with f64. I'm going to use f32 everywhere for the remainder of this post, but it could just as easily be f64.
• Taffy functions use Option<f32> where Druid/Xilem just use f32. However, where Taffy uses Option::None to represent an infinite/unset size (never using f32::INFINITY) Druid/Xilem use f32::INFINITY to represent this case. Again, this is a trivial conversion that could easily be handled as part of a widget or similar.

Extra data parameters

• Druid has data and env parameters which provide extra data. I don't quite understand env, but I think it's some kind of context. We will need something like that in Taffy at some point for allowing styles (particularly things like writing mode / direction) to inherit down the tree. But for now, I think we can ignore this.
• Taffy has the tree parameter which provides access to style information, the ability to request that children size themselves, and the ability to store the final computed size and position of nodes. I think this can all be handled by the widget implementation, so again we can ignore this (although this is one place where we might later get nicer DX with tigher integration with Xilem).

Comparison of functions

• Layout function: All three frameworks have a layout function that implements a full layout of that node and all children and returns a Size<f32> (modulo the aforementioned f32/f64 difference). Druid suggests that this should only be called once (but that this isn't actually enforced). Taffy only does call this method once in the usual case, but may need to call it multiple times to support baseline alignment (only if baseline alignment is actually used in the layout).
• Measure function: All three frameworks also have a measure function (this is compute_max_intrinsic in Druid) which allow child nodes to compute their intrinsic (content) size(s) under the provided constraints and hints and return it to parent nodes. However, they all work slightly differently:
  • Xilem's measure function returns:
    • Returns the sizes in BOTH the horizontal and vertical dimensions at once (as a Size)
    • Returns BOTH the min and max sizes (as a tuple)
  • Taffy's measure function:
    • Returns the sizes in BOTH the horizontal and vertical dimensions at once (as a Size)
    • Returns EITHER the min or max size depending on the available_space parameter
  • Druid's compute_max_intrinsic function
    • Returns the size in EITHER the horizontal and vertical dimension depending on the axis parameter
    • Returns ONLY the max size. Druid has no concept of a min content size.

I would suggest that the concept of a "min content size" is important and should definitely be included. I would also suggest that the function should not compute both the min and max sizes at once as Xilem currently does, as this could be expensive (e.g. for a text node) and at least for CSS layout it's relatively common that only one of the sizes is required.

Whether both axis are computed together or seperately I don't have too strong an opinion about. Taffy layout modes would probably compute both either way and cache the other one the other one for future queries. Update: I now believe that the single-axis-at-a-time model is superior.

Comparision of function parameters

Constraint paramters

Constaint parameters have a direct relationship with the returned size and must be respected by nodes' measurement/layout functions (and/or the sizes returned will be ignored/clamped if they are not).

• [Taffy] known_dimensions (Size<Option<f32>>) - it is often the case that a parent node wants to ask a child node to size itself in one dimension while treating the other dimension as fixed, effectively asking the child a question like "suppose your width is 100px, what would your height be?" (perhaps the width has already been determined in an earlier part of the algorithm). This parameter provides a way to specify those fixed dimensions.
• [Druid] box_constraints (BoxConstraints). Druid's box constraints offer a strict superset of this functionality (setting min=max=some finite number in a given dimension is equivalent to setting the known dimension in that dimension; setting min=0, max=Infinity is equivalent to not setting the known dimension in that dimension). The max contraint also seems useful in it's own right. It makes sense to ask a node to size itself within a certain bounding box. Taffy has it's own version of this in the available_space parameter

Hint parameters

Hint parameters provide extra information that nodes may use to help choose their size. These are merely hints and may be ignored in some cases. But will likely be very helpful to allow the parent and child node to cooperatively choose a good size.

• [Taffy] sizing_mode: This makes a distinction between the intrinsic (content) size of a child (ignoring styles like min-width and max-width that might override this) and the inherent size which does respect those styles. This is important for a 100% spec compliant flexbox implementation, but I think it can ignored here (at least for the time being).
• [Xilem] proposed_size (Size<f32>): Xilem uses a proposed_size: Size<f32> parameter, which seems to be used primarily by the v_stack component which sizes children in order and uses proposed_size to pass "remaining available space" which is equal to it's own proposed_size minus "the size of any already sized children" minus "spacing between children". I would suggest that this is replaced by a more general available_space parameter (see below).
• [Taffy] parent_size The purpose of this parameter is a size for the child to resolve percentage sizes against. The parent container can choose exactly how this is defined (for example in flexbox this is the size of the overall flexbox container, whereas in CSS Grid it is the size of the grid cell that the child being sized in placed in). I think is useful (because percentage sizing is useful) and should be kept.
• [Taffy] available_space: In Taffy this is always set to the same value as parent_size if parent_size is a finite definite pixel size. But if the parent size is unknown then this enum carries an additional hint: whether the content based size should be a "min content" or a "max content" size. Taffy doesn't have an hstack/vstack-like layout, but I think this parameter would be a good place to pass the "remaining available space" that Xilem's current v_stack widget calls proposed_size (in this case available_space would differ from parent_size). I think this is useful and should be kept, however I think it is potentially confusing to couple the min/max content sizing hint with this size, so I suggest that we split this into a seperate enum parameter.

Proposal for Xilem

The following type definitions are used in the propsoal below:

struct Size<T> {
    width: T,
    height: T,
}

struct BoxConstraints {
    min: Size<f32>,
    max: Size<f32>,
}

enum RequestedSize {
    MinContent,
    MaxContent,
    FitAvailableSpace,
}

I propose that the Xilem widget trait has the following two methods for layout, replacing the existing layout and measure methods:

fn measure(
    &mut self,
    box_constraints: BoxConstraints,
    parent_size: Size<f32>,
    available_space: Size<f32>,
    requested_size: Size<RequestedSize>,
    axis: Axis,
) -> Size<f32>;

fn layout(
    &mut self,
    box_constraints: BoxConstraints,
    parent_size: Size<f32>,
    available_space: Size<f32>,
    requested_size: Size<RequestedSize>,
) -> Size<f32>;

I believe this would provide a strong framework within which lots of powerful layout paradigms could be implemented. But I'm sure I haven't thought of everything and feedback and discussion is of course enouraged!

I suspect this is not intended behavior? All hover state should be discarded on cursor exiting the window.

Right now the assert_render_snapshot! macro and the underlying code are held together with duck-tape.

Some possible improvements:

• Better error messages when a snapshot test fails.
• Some sort of cargo-insta style UI tool that shows you the image and asks you to validate one or the other.

Follow the instructions to make a simple demo trainer, but there is a compilation error. Is there any information you can provide, thanks!

E:\MyProjects\rust-app>cargo build
Updating tuna index
Compiling masonry v0.1.2
error[E0609]: no field app on type RefMut<'_, AppRootInner>
--> C:\Users\liaoxuewei.cargo\registry\src\mirrors.tuna.tsinghua.edu.cn-df7c3c540f42cdbd\masonry-0.1.2\src\app_root.rs:217:19
|
217 | inner.app.quit();
| ^^^ unknown field

For more information about this error, try rustc --explain E0609.
error: could not compile masonry due to previous error

The goal of debug_logger is to serialize the state of the widget tree in real time, to a stream that can be read by an external debugging application which can inspect the stream and display what the widget tree looked like at a given time, and inspect internal values and stuff.

While said debugging application has proven extremely invaluable for me for debugging masonry, I don't expect anybody else to use it, as its UX right now is absolutely abysmal. I do intend to improve it eventually.

In the meantime, debug_logger is a pretty clunky, unwieldy tools. Among other problems:

• It requires trace calls in a bespoke format to be added everywhere in the code.
• It feels superfluous with tracing already in the codebase.
• Its performance for non-trivial cases is terrible.
• It adds a dependency to serde, whereas we want to trim dependencies (#10)

Now, I think debug_logger is an idea that has some real potential. In my ideal vision, it would be a record-replay framework of sorts, one that would produce a lightweight record of the widget tree and application state over time. It would probably use the tracing crate as the serialization layer, and it would do some fancy caching to only record changed information to avoid unnecessary overhead.

I need to write a full design document for that ideal logger, and then get started on implementing it.

The Julia ecosystem has a number of grid layouts optimised for charts and plots with titles, axes and so on. It has an extremely stateful and side-effect-driven API, but the underlying logic seems sound and results in very nice and flexible placement of elements. Once the widget set and layouts for this toolkit mature a bit, it would be nice to explore these kinds of layouts. They are more generally applicable than plotting. For example, flexibly attaching labels to input widgets while maintaining good alignment.

For example:

https://docs.juliahub.com/AbstractPlotting/6fydZ/0.14.1/makielayout/grids.html
https://docs.juliaplots.org/latest/layouts/

The Env class and the Data trait are vestigial by this point in the Masonry codebase.

There is currently no support for updating widgets when the Env is changed, and I have no intention of adding that support (Xilem seems to be moving in the same direction). This means Env is basically just a way to pass a singleton around in Widget code.

We should remove Env and data, and just hardcode the values that we currently get from Env, with future plans to pass them as arguments or something. Frontend frameworks might still use something Env-like to get style data, but that'll be their responsibility.

I think you can have a lot of power if you can implement only a subset of the DOM drawing and have a typescript/js interpreter in RUST that can leverage the xilem primitives.

Some things to improve:

• Multiple windows. Right now AppDelegate assumes there's only one window, and ignores all except the first one.
• Sending commands. Right now the AppDelegateCtx type doesn't have a method to send commands. That should be fixed (though it should be part of a large strategy, see #335)
• Implement for FnMut. If we expect most AppDelegate implementations to only implement a single method, we may just want to implement it for an FnMut taking the same parameters as that method.
• Maybe skip downcast. She should have enough type information to know the widget type of at least the first window. We might want to use it to skip a downcast.

Hi there! Really interesting project :) I wanted to try out the examples locally, but failed to compile them.

The scroll example requires hex and sha2, and the counter example can't resolve various imports from xilem.

Not sure about this one.

The requirement to call ctx.init() before any method call for all methods of SomethingCtx types was added to prevent the case where someone would get a widget ref and call Widget::some_method() on it instead of calling WidgetPod::some_method, which would be slightly buggy.

I'm not sure it's necessary anymore. Removing WidgetPod::widget_mut might be enough to remove all instances where that problem existed. And the ergonomic cost of panicking if you don't add ctx.init() everywhere is pretty huge.

Hi - I read the blog post and am interested in the concept. I was wondering if you had some kind of roadmap? Things look sufficiently in flux that I suspect you are not ready to accept random contributions at this time?

In current code, calling declare_widget!(FoobarMut, Foobar) will generate FoobarMut as a tuple of a WidgetState and a Foobar. It should instead be a struct with these two fields, with names like "state" and "widget", for clearer widget code.

Portals are widgets that are meant to display a view to a bigger area. They're used for Scroll widgets.

They are still conceptually unclear. We need to figure out what their semantics are and how they interact with other concepts.

• Portal should probably be renamed to ScrollPortal for clarity.
• We may want to remove the register_as_portal (the fact that it's only used by a single widget is a bit of a code smell).
• Right now, the Scroll widget needs mutable access to its child Portal when computing layout, which kinda breaks the core abstraction.
• We need to figure out how Portals interact with paint insets. Eg if a button inside a scrollable area has a drop shadow,, how is the shadow handled?

Some dependencies can probably be removed with minimal effort to reduce build times / sizes:

• serde
• im
• chrono
• fluent

Module structure

Virtually every module should be private. The only public modules should inline modules that gather public-facing re-exports.

Most items should be exported from the root module, with no other public-facing export. This makes documentation more readable; readers don't need to click on multiple modules to find the item they're looking for.

There should be only three public modules:

• widgets
• commands
• test_widgets

Module files should not be foobar/mod.rs. Instead, they should _foobar.rs (thus in the parent folder); the full name is for readability, the leading underscore is so these names appear first in file hierarchies.

Imports

We should avoid use super::xxx imports, except for specific cases like unit tests.

Most imports in the code base should use the canonical top-level import.

Prelude

Masonry should have no prelude. Examples and documentation should deliberately have a list of imports that cover everything users will need, so users can copy-paste these lists.

For example, if a button has a boolean state of pressed and I have a lot of them, do they have to become part of the global state now?

If so I think some way of combining the data tree with the widget tree could make this more flexible

Is it possible to add functionality to send a signal to a list view that the underlying data has changed, like for example that some rows were added, deleted, or changed? In addition to the straightforward use, this would allow trees to be implemented on top of lists: deleting rows can be used when collapsing, and inserting rows when expanding.

I think a derive macro for the view trait, especially on enums, would really benefit ergonomics.
Views that return multiple types on a condition, such as an if block or match statement, could instead return an enum with View derived.

For example, for conditional views we could return a custom enum like:

#[derive(View)]
enum UserForm {
  Login(LoginView),
  SignUp(SignUpView)
}

Widgets should be able to require paint-only animations, eg animations that don't change their layout.

When dispatching animations, you could then skip offscreen widgets. Animation event should include some kind of timestamp, so that widget that skip animation frames can display the correct contents.

The following tests are failing for me:

    widget::align::tests::centered
    widget::align::tests::left
    widget::align::tests::right
    widget::button::tests::simple_button
    widget::checkbox::tests::simple_checkbox
    widget::flex::tests::flex_col_cross_axis_snapshots
    widget::flex::tests::flex_col_main_axis_snapshots
    widget::flex::tests::flex_row_cross_axis_snapshots
    widget::flex::tests::flex_row_main_axis_snapshots
    widget::label::tests::line_break_modes
    widget::label::tests::simple_label
    widget::label::tests::styled_label
    widget::portal::tests::button_list
    widget::sized_box::tests::label_box_no_size
    widget::sized_box::tests::label_box_with_size
    widget::split::tests::columns
    widget::split::tests::rows
    widget::textbox::tests::simple_textbox

And the .diff.png files all only show the text, e.g:

(I'm on Debian.)

Masonry's pointer handling is inherited from Druid, and there are some gray areas that need to be better specified, documented and enforced.

Hot status

Hot status now roughly means "This widget is the one that will receive clicks from the mouse".

There are two things we need to specify:

• (1) What hot status means in a context with multiple pointers.
• (2) How hot status relates to layout changes.

(1) means we either get hot state for only the primary pointer, or the last active pointer; or we get multiple hot states, and is_hot() will take a pointer id as parameter, or we get hot state for every pointer without distinction of which pointer makes which widget hot.

(2) is especially annoying, because it breaks the unidirectionality of the "events -> layout -> paint" flow.

We may also want to rethink how hot status interacts with widgets overlapping one another; eg if we want a browser-like "mouseleave" event that doesn't trigger where the mouse goes from a container widget to one of its children.

See also: https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-hot_state-md

Active status

Currently, active is a flag which:

• Can be set by widget code, but never by framework code.
• Forces mouse events to be routed to the active widget, even when the cursor is out.
• Can be read by widget code to change how the widget is rendered (or laid out).

Aaaaand... that's it. Any other information it might represent (eg that it means "the user is in the middle of a mouse press on this widget") is purely down to widget implementation.

Ideally, I would like "active" to be better integrated in the framework:

• The framework should guarantee that there is only one active widget at a time (per pointer).
• The framework should force the widget to lose active state when some events occur; not just MouseLeave, but also Tab being pressed, the window losing focus, the widget being disabled, etc.
• There should be an equivalent to setPointerCapture in the browser; so that when a widget is active and the cursor leaves that widget, not only does the widget still get pointer events, other widgets do not (or at least in a way that lets them know the original widget still "owns" the pointer).
• Pointer capture should also be handled at the app level if possible, in case the pointer leaves the window.

We might even want to be more opinionated and have the framework guarantee that hot state and active state are correlated (eg when user clicks, active state is given to the hot widget).

See also: https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-active_status-md

Right now the framework only gives a children_changed() method when a Widget has changed its data in a way that adds or removes children.

Instead, it should provide three methods:

• child_added()
• child_stashed(bool)
• child_removed()

These methods should update semantic information in the widget tree, eg:

• The focus chain
• Active status, pointer capture (see #379)
• Keyboard focus

(Obviously, we'll need to add lots of unit tests for all that)

I was able to compile on my laptop, but both example apps fail to run. I could compile but not successfully run all the wgpu project tests. I was able to get some of the wgpu showcase apps to run e.g. the n-body simulation of galaxies. I was able to successfully compile and run the tests of piet-gpu.

Here's a stacktrace.

     Running `target/debug/xilem counter`
MESA-INTEL: warning: Haswell Vulkan support is incomplete
size = 80.09600067138672W×24.0H
layout size = 80.09600067138672W×24.0H
render size: 1024.0W×768.0H
thread 'main' panicked at 'Error in Surface::configure: surface does not support the adapter's queue family', /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/backend/direct.rs:274:9
stack backtrace:
   0: rust_begin_unwind
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/panicking.rs:142:14
   2: wgpu::backend::direct::Context::handle_error_fatal
             at /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/backend/direct.rs:274:9
   3: <wgpu::backend::direct::Context as wgpu::Context>::surface_configure
             at /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/backend/direct.rs:1017:13
   4: wgpu::Surface::configure
             at /home/nmrp3/.cargo/registry/src/github.com-1ecc6299db9ec823/wgpu-0.14.2/src/lib.rs:3715:9
   5: piet_wgsl::util::RenderContext::create_surface
             at /home/nmrp3/.cargo/git/checkouts/piet-gpu-adec1df56a2c5942/5718222/piet-wgsl/src/util.rs:70:9
   6: xilem::app_main::MainState<T,V>::render
             at ./src/app_main.rs:214:33
   7: <xilem::app_main::MainState<T,V> as glazier::window::WinHandler>::paint
             at ./src/app_main.rs:107:9
   8: glazier::backend::x11::window::Window::render::{{closure}}
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:568:13
   9: glazier::backend::x11::window::Window::with_handler_and_dont_check_the_other_borrows
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:505:31
  10: glazier::backend::x11::window::Window::render
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:567:9
  11: glazier::backend::x11::window::Window::redraw_now
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:673:9
  12: glazier::backend::x11::window::Window::run_idle
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/window.rs:932:29
  13: glazier::backend::x11::application::Application::run_inner
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/application.rs:700:25
  14: glazier::backend::x11::application::Application::run
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/backend/x11/application.rs:710:25
  15: glazier::application::Application::run
             at /home/nmrp3/.cargo/git/checkouts/glazier-3a172f69e2427c5a/6ef3bfd/src/application.rs:150:9
  16: xilem::app_main::AppLauncher<T,V>::run
             at ./src/app_main.rs:90:9
  17: xilem::main
             at ./src/main.rs:19:5
  18: core::ops::function::FnOnce::call_once
             at /rustc/897e37553bba8b42751c67658967889d11ecd120/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

and some vital stats:

$ cargo --version
cargo 1.65.0 (4bc8f24d3 2022-10-20)

$ uname -a
Linux persephone 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

$ prime-select query
nvidia

$ sudo lshw -class display
[sudo] password for nmrp3: 
  *-display                 
       description: 3D controller
       product: GM108M [GeForce 840M]
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a2
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:35 memory:f6000000-f6ffffff memory:e0000000-efffffff memory:f0000000-f1ffffff ioport:e000(size=128) memory:f7000000-f707ffff
  *-display
       description: VGA compatible controller
       product: 4th Gen Core Processor Integrated Graphics Controller
       vendor: Intel Corporation
       physical id: 2
       bus info: pci@0000:00:02.0
       version: 06
       width: 64 bits
       clock: 33MHz
       capabilities: msi pm vga_controller bus_master cap_list rom
       configuration: driver=i915 latency=0
       resources: irq:34 memory:f7400000-f77fffff memory:d0000000-dfffffff ioport:f000(size=64) memory:c0000-dffff

When porting from Druid to Masonry, I've removed the ability to trigger commands from links.

While I'm not sure how to handle that feature on the long term, on the short-term it should be fairly easy to re-enable. Most of the code is still there, just commented out.

Essentially, we just want Label widgets to send Commands during MouseUp events when the cursor is over a link. Ideally we'd want to handle active status and cursor changes and stuff, but we'll keep it simple for now.

IME events need to be reworked both in Masonry and in Glazier.

The current workflow is extremely complex, in part due to the inherent complexity of IME. However, I suspect it would be possible to make it simpler in some ways. In particular, I think it might be possible to pass glazier::text::Action directly to widget events.

Also, it should be possible for WinHandler::acquire_input_lock to return &dyn InputHandler instead of a Box.

The TestHarness type aims to emulate an actual GUI environment with as much fidelity as it can while staying performant. However, some of its internal abstractions are still a bit leaky, and should be improved:

• Skip downcast. Right now, the harness knows what Widget type it's built with, so the edit_root_widget method should perform a downcast internally and give the WidgetMut to the correct widget type.
• Handle ExtEvents. If the user starts a background thread that sends ExtEvents, these events will be ignored. However, in real use-cases, ExtEvents are used for IO operations and other stuff that lives outside of the window abstraction. The harness should provide options for developers to mock these leaky abstractions for unit tests.
• Fix painting invalidation. Ideally, for a given event-layout-paint pass, the invalidation rectangles given to widgets should be or similar in testing to those you'd get from a real platform. (modulo the fact that different platforms might handle invalidation differently)
• IME. Currently the handle only handles sending keyboard events. It doesn't actually send text events, which mean you can't eg mock typing text in a textbox.
• Animations. The move_timers_forward method should also move animations forward.

The project needs a logo, for all the usual reasons (to stand out, to give a professional vibe, to give a visual association that people will remember, etc).

I'm thinking the logo should probably be very simple, something like a SVG brick wall within a Windows-95-style window, maybe.

EDIT - Something like this:

Tooling

Masonry should have infrastructure to measure various runtime performance characteristics.

Because it's a GUI app, profiling is a bit more complicated than "run program 100x times, measure average performance". What we'd want to do is measure things like average FPS count, 99% latency, startup times, etc.

We need to look up industry best practices for profiling GUI and, if necessary, make up our own. At the very least, we should have tooling built into the pass system (maybe using tracing ?) to measure the targets listed above, and easy ways to dump the results. We might even want a "dump some core indicators to stdout on program exit" feature to be enabled by default in debug mode, to encourage thinking about these indicators.

Benchmarks

We should produce some benchmarks; again, we may want to both consider the state of the art (eg js-framework-benchmark) and make our own.

I'm especially interested in cases that stress-test Masonry: lists with millions of items, a widget hierarchy which is thousand of nodes deep, ridiculously high resolutions, etc.

For all those cases, we should consider not only the passive performance (eg how long does it take to render a frame), but also the performance given mouse events, keyboard events, etc.

Text handling right now is a huge mess inherited from Druid. Even the original authors of that code are a bit hazy on what some parts of the code do.

The text handling code should be refactored completely.

An important step in building out the widget tree is to get the trait right. Further iteration is possible, but getting it basically right at this time will save hassle later.

The high-level approach is to start with the Druid Widget trait and make some changes.

• The data argument goes away (as does being generic on T). This is a natural consequence of the Xilem architecture.

• The env argument goes away, I think. This is actually a much bigger question, as we rely on env to retrieve style data, so probably deserves some thought about what will replace that.

• The paint method evolves to generate a piet-gpu scene fragment rather than drawing into a Piet render context.

• The trait also needs to update the accessibility tree. In the prototype Druid integration for AccessKit, there an accessibility() method that generates the node, but that's based on generating the entire tree every time rather than doing incremental update. This needs to be worked out in detail.

As discussed in today's office hours, we will not be making significant changes to layout at this time, so those methods will basically be the same as Druid's existing Flutter-like mechanism.

The update method

Right now the update method isn't doing much, and if #6 gets adopted, it will do even less. In current Druid, one of its other roles is to respond to env changes (if theme or styling changes), but that may well go away as well.

A potential role may be to update the accessibility tree, but likely that should be a separate method, as it should be lazy (only called when a screen reader is connected).

So one of the questions which needs to be decided is whether to keep the update method or remove it. An excellent way to help resolve it is to go through the existing use cases in Druid and see if any really motivate keeping it.

The lifecycle method

We can also consider combining lifecycle and event, as the motivation for having them separate may no longer be compelling. I consider this a lower priority, as I'm sure it's fine either way, but again if we do decide to change this from Druid, it would be less work to do it now than later.

Masonry has pretty good unit testing tooling, but a lot of unit tests are still missing.

Ideally, we're aiming for 100% coverage (with opt-outs) for at least every widget and the WidgetPod code. Each widget should have tests covering:

• Creating the widget
• Displaying the widget
• Mutating the widget with WidgetMut.
• If the widget is interactive (eg a button, a textbox), interacting with it.

The first step would be to go over the code (potentially using cargo tarpaulin) and list all the places missing unit tests.

I'm on Ubuntu 22.10 (wayland) and after cargo build I get an error about missing cairo libraries.

I have several issues with how layout works in Druid (and thus currently in Masonry).

Flex footguns

I think this is more due to the Flex algorithm than Druid's specific implementation. But either way, it has quite a few footguns:

• Some widgets will return box_constraints.max() as their size, even though the Scroll widget will give infinite max constraints to its children.
• Flex::row(child1, child2) and Split::columns().with_flex_child(child1).with_flex_child(child2) render differently, even though they conceptually do the same thing.
• Buttons given a large minimum size (eg in a Split widget) will grow enormous, even having a button take half your screen makes no sense visually.
• Changing some Flex params sometimes does nothing, because some Flex params (or having Flex children) will "paper over" the effects of other params. This is not immediately obvious if you don't have a model of what these params do.
• Adding .center() to your widget will change not only its position, but also its size. Eg in my_widget.center().expand(), my_widget doesn't have the same size as in my_widget.expand().
• Adding .expand() to a flex container does nothing, because .expand() works by passing bigger minimum-size constraints to its child, and Flex calls .loosen() on all its children.

(note that some of these examples don't quite translate to Masonry because Masonry does not use WidgetExt, but the general idea is the same)

Overall, the Flex algorithm doesn't seem to be "continuous" enough. That is, local changes to the code may create non-local changes to the UI; sometimes changes may have no effects or invisible effects.

Ideally, I'd want the layout algorithm to be discoverable. The default case should give sensible good-enough UI, and it should be possible to implement common patterns by combining layout primitives with little guessing.

Handling invalid values

The base of the layout algorithm is this: each widget is given size constraints (a minimum and a maximum size) and has to return its size.

Currently, there is no specification as to what happens if the returned size doesn't match these constraints. There's also no specification for what happens if a widget passes invalid constraints to its children (NaN or negative values or something).

Which means that in practice, what happens is either "nothing" or "graphical bugs".

We might want to emit warnings in those cases. Though it's always possible that size constraints aren't respected because the user made the window too small or something similar, in which case warnings would just be visual pollution. But we do want a way to detect it when it happens.

We might also want a mechanism for detecting when creating a layout is impossible. For instance, if we have a box with 16px of padding, and only 10px of free space, there is no way to fit that box's contents in the given space. We could say layout() returns an Option<Size> and returns a None when layout is impossible.

What to do

The first step would be to work the concept of "optional layout" into the architecture. Widgets may have optional positions, and widgets with None positions would be marked as un-placed. (Not sure how clear that would be when debugging)

We should rework and/or remove these widgets:

• Align
• SizedBox
• Flex
• Split

Finally, we need to come up with better geometry primitives to represent UI layout and avoid doing math operations "by hand".

See also:

• https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-debugging_flex-md
• https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-layout_frustrations-md
• https://gist.github.com/PoignardAzur/c60cd72d790c083b7b55269bc91912db#file-safer_layout-md

I cloned the repository and attempted to run the default main application as a sanity check for myself, however my screen is presented with an empty button.

The only notable thing I could deduce while poking around the source code, is that the if let Some(fragment) statement on text.rs:40 always fails, and nothing is ever appended to builder. Could it be a failure in default font discovery?

My system is Linux Mint, running x11 with i3 as my WM and Plasma as my DE.

WebImage was removed for a few reasons:

• It was added for testing promises, but it wasn't really a core widget and it definitely wasn't important enough to justify the problems it created.
• It loaded its contents by calling reqwest::blocking::get(url) in a background thread, which isn't ideal for a few reasons:
• Bad performance.
• Adding an unnecessary dependency to reqwest that wasn't used anywhere else in the project.
• That specific function didn't compile on wasm.
• It relied on ImageBuf::from_data, which caused some build problems because that method wasn't reliably generated depending on feature flags.

I would like to add that type back eventually, with a few caveats:

• Wait for the transition to glazier (#24) where we'll hopefully have a better import structure.
• The type shouldn't use any network code itself. We should use dependency injection and pass a builder function as a parameter instead.
• We should implement running async functions in the background to avoid having to call reqwest::blocking.

Vello and Glazier are two successor projects to dependencies Masonry uses, respectively Piet and druid-shell. Both of these dependencies are unlikely to to be maintained in the near future (especially druid-shell), while Vello and Glazier are actively worked on by the Druid community.

As both these projects have APIs very similar to the ones they're succeeding, the switch should be a relatively straightforward affair (famous last words). It should not impact the semantics of Masonry.

Note: If I understood correctly, 2193b87 is where this was implemented in Xilem.

Dialogs, especially platform-created file pickers, are a feature from Druid that was ripped out when I wrote Masonry and haven't added back in yet.

Part of the code is still in the codebase, commented out, taunting me.

Ideally, I would want the Masonry implementation to use the promise feature, but this is a more complicated task than it might appear.

I want to make some pretty deep changes to the Widget trait. The ultimate trait would look something like this:

pub trait Widget2: Any {
    fn on_event(&mut self, ctx: &mut EventCtx, event: &Event);

    fn lifecycle(&mut self, ctx: &mut LifeCycleCtx, event: &LifeCycle);

    fn layout(&mut self, ctx: &mut LayoutCtx, bc: &BoxConstraints) -> Size;
    fn compute_max_intrinsic(&mut self, axis: Axis, ctx: &mut LayoutCx, bc: &BoxConstraints) -> f64;

    fn accessibility(&mut self, cx: &mut AccessCx);

    // Maybe some other arguments after the switch to Vello
    fn paint(&mut self, ctx: &mut PaintCtx);

    fn children(&self) -> SmallVec<[WidgetRef<'_, dyn Widget>; 16]>;

    // Got to figure out what argument the callback takes
    fn call_on_children(&mut self, callback: impl FnMut(&mut impl Widget));

    fn get_debug_text(&self) -> Option<String>;

    // --- Auto-generated implementations ---

    fn make_trace_span(&self) -> Span;

    fn get_child_at_pos(&self, pos: Point) -> Option<WidgetRef<'_, dyn Widget>>;
    fn get_child_with_id(&self, id: WidgetId) -> Option<WidgetRef<'_, dyn Widget>>;

    fn call_on_child_at_pos(&mut self, pos: Point, callback: impl FnOnce(&mut impl Widget));
    fn call_on_child_with_id(&mut self, id: WidgetId, callback: impl FnOnce(&mut impl Widget));

    fn type_name(&self) -> &'static str;
    fn short_type_name(&self) -> &'static str;
}

Notable changes:

• Accessibility support is baked in. I'm not sure this is the right way though.
• We add compute_max_intrinsic method to layout pass.
• We add call_on_children method. That method should replace the boilerplate code in container widgets that propagates events. I'm not sure how to make it work type-system-wise.
• Generally speaking, we give up on the idea of Widget being completely dyn-safe. This mean we'll need an ErasedWidget type, same idea as in Panoramix.

The AppRoot is the composition root of everything in a Masonry program; everything passes through an AppRoot instance.

AppRoot instances each own an AppRootInner, which it needs to lock to perform operations. However, some AppRoot methods may call WindowHandle methods which themselves may end up indirectly calling AppRoot methods; thus, these methods should be re-entrant.

Which methods need to be re-entrant and why should be documented (which will probably require familiarity with Glazier code).