Allow for an atomic define-able, composable global store

[duttaoindril]

Hey Crux team, after understanding a bit more about how Crux works and how the state management is defined, I can see history repeating itself. 1 massive key value pair serializable global store will not scale well. It needs to be distributed. For example, the event handler loop can be broken from a flat enum switch to a nested enum switch, where the first word is the first enum, and the second word is the second enum, and that determines how an event is handled in a nested fashion, allowing the app to scale.

That model doesn't work for state, and in js-land where we've gone through all these stages of the global state model, what we've arrived at as the best solution for scaling an app is an atomic store, like jotai. It would be really cool if Crux could support or let us opt in to coding that way instead of a massive key value pair store, that may not scale well.

[charypar]

Hey, thanks for your thoughts!

I'd like to understand the problems you see with the model a bit better. What scaling challenge have you got in mind? Is it in terms of app complexity? What do you mean by state being distributed, and how does it resolve the problem?

The event enum nesting you're describing is exactly the intended way for Crux apps to split into modules (or compose from existing ones), but I'm not sure I understand why that approach doesn't work for state, can you share a bit more of your experience with this?

I should probably add that we have been along for the (wild) ride for the entire history of state management in React, and I can see where you're seeing similarities, but they don't necessarily translate one to one. We're definitely not thinking of state as a single huge key-value store. In principle, we're not restricting the use of any types as or inside your Model. The only expectation from Crux is that the model implements Default, so Crux can construct it.

As an example, in the Notes app, the model holds a Note type which wraps an automerge document (which is quite a complicated piece of data with logic attached) and itself has a friendly API. Because we have the ViewModel to represent the state of the UI, we don't even need Model to serialise, it stays in the core and the only consumer of it is your app's update function.

We're planning a documentation guide which explains the details of how nesting modules works, but we haven't had a chance to write it yet, apologies. The best I can offer is a very noddy example of this in the cat facts example coee. It has a separate app module for fetching the platform description from the shell (it's a silly example). It's then used from the main app module like this:

match msg {
...
Event::Platform(msg) => self.platform.update(msg, &mut model.platform, &caps.into()),
...
}

and the model holds the submodule's state:

#[derive(Serialize, Deserialize, Default)]
pub struct Model {
    cat_fact: Option<CatFact>,
    cat_image: Option<CatImage>,
    platform: platform::Model, // <- here
    time: Option<String>,
}

I hope that's helpful context! Also if you'd like to talk about it more real-time, you can join our Zulip channel.

[duttaoindril]

I see, that makes a lot of sense, and it's much better than I initially thought. It might be possible to do what I'm thinking, but just to explain, here's what I believe the current implementation is missing:

Colocation of business logic and it's state: The state lives in this 1 variable, whatever it might be, and the business logic relevant to that state - any computations, any effects - live in the enum action handler, and potentially split into state class. This will not allow for colocation of the state and its relevant logic, which, from the advent of hooks as a pattern, really helps make codebases be more readable and scalable.

Fine grained reactivity that's only possible from distributed, atomic pieces of state: Instead of having each action updating state triggering a blunt render call to refresh the entire app, atomic state allows for only the reader of a specific piece of state to selectively re-render only when it's relevant state updated, improving performance across the entire app.

Colocated, reactive and composable state: I can take two or more pieces of data, compute something between them, and have it be fed back into the entire system like any other part. I can generate multiple (let's say a list of todos) reactive todo atoms that encapsulate the complex business logic and state of a task (maybe more atoms!), and put them into a composed atom of the Todo list, and have it be the only thing this Todo list atom is worried about: CRUD on a list.

The main way a library like jotai provides all these benefits is by "decentralizing" the responsibility for state. Each atom creates and manages its own data & subscribers, and when adding other atoms as a part of its definition - making a composed atom - it simply subscribes to those sub atoms. This automatically creates a sort of control graph of reactivity, where data holding atoms, the leaves, are subscribed to by other atoms, which repeats, until the UI subscribes only to the atoms that need to be rendered. Only things that are actually changed cause the UI to re-render, instead of a global "render" call. Using a variable to store an atom is really what allows this pattern to exist, and I don't know how to do that in a model that expects 1 variable for all it's state. This pattern is gaining a lot of traction, in react's jotai, solid's signals, flutter's riverpod, and as far as I can tell, is the pattern the industry has decided is the best way to manage client side local and global state.

It's not immediately clear how we could have all these traits in the existing implementation of Crux. I'm sorry I'm bringing very late level problems into such an early and promising framework, but I believe getting close to industry standards will help it become a full fledged mainstream framework. I really use jotai as a shining example of what these patterns could look like.

I'm also sorry for being very vague on parts of this, a lot of what I'm thinking just exists in jotai. I also just don't have the answers to them, since I'm pretty new to rust, and my mind was already blown from your talk about Crux. I'm really looking forward to this framework doing well! Definitely joining the zulip!

[charypar]

Thanks for taking the time to write that up. There's a lot of points to unpick, but I'll try to address the key ones.

Collocation of logic and state

In principle, all three of the main components of state management compose/nest – parent events can carry child events, parent update can call child update, parent model splits into child models. This should make it possible to collocate related logic and state. It also keeps the door open for not doing that, you might, for example, want an analytics update function which you call in the top level update to track the events, before then sending them to their respective update handler.

It's worth saying that in my opinion, a lot of the lower level logic and state should be unaware of Crux. As an example - moving from a list of contacts to a detail of a contact (e.g. a response to a Event::ViewContact) would be behaviour in Crux, but the detail of adding and removing phone number data and whether that data is kept in a vector or sorted set would likely be behind an API in an impl of a Contact type which is blisfully unaware of Crux.

Fine grained reactivity

I think this is where your preference and our preference of the overall approach diverge. Nothing particularly wrong with either of them, but we're not going in the direction of Functional Reactive Programming.

FRP is useful to to manage complex flows of event data through a pipeline of transformers containing bits of business logic, to a number of sinks. It's certainly possible to model user interface state management that way, but at least in my view, it always ends up being quite complicated, and hard to understand where changes originate and how they propagate. The code ends up being separated into distinct "construction" and "execution" phases and sections, and reading the execution section is difficult without keeping an accurate mental model of the construction phase in my mind the whole time. I find it hard.

In the Crux (Elm inspired) model, we prefer to think of the application behaviour as a state transition function with a signature along the lines of:

update(event, state) -> (state, view_state, effects)

Things are simpler to understand when events can only happen sequentially (they go through a single entry point, one at a time), and all of them are actual values that can be logged, inspected, etc. That calculation does decompose into smaller pieces, arranged in a kind of natural call tree, where the functions closer to the root serve largely as dispatch logic for the lower level functions. We're hoping this tree neve needs to get too large and unwieldy, but nothing should really limit how far it can scale.

UI as a function of view state, not a large stateful object

One of the outputs of the above transition function is the user interface state, which contains a simplified description of the state of the screen - the view model. It should only ever describe about a screenful of content. This gets created after every UI state transition. The actual UI itself, in our minds, is a projection of this data description, something like:

view(view_state) -> ui_component_tree_or_drawing_instructions

This translation happens in the shell. And the declarative UI libraries like React, Swift UI and Compose allow this abstraction to perform well by doing various optimisation tricks (like diffing) to skip work for things that didn't change, so that we don't have to.

This is where in some cases, some level of hand optimisation with elements of explict reactive programming might be necessary, but given the mental overhead of working with those tools, this should, in my opinion, be treated as opt-in, and a trade off between performance and simplicity.

I personally want to think about user interface as a function of data, not as a complex thing I need to orchestrate careful localised updates for, by hand.

You mention that the industry seems to be rallying around observable atoms or similar reactive ideas as best practice, and this is actually where I see history repeating itself. This happened back in the day in Angular with RxJS, and then again in mobile apps with things like RxSwift and Combine, and I'm not sure people were very happy with the outcomes.

Interestingly, Elm started there, with very FRP-like signals, and ende up with the current architecture, which inspired Redux and Crux as well.

As I said, this is largely a philosophical question of how one prefers to think about event driven UIs. We think this way is simpler, but we may well be proven wrong, we shall see 😅

[matthewpflueger]

I agree with the simplicity/elegance of the update(event, state) -> (state, view_state, effects) and view(view_state) -> ui_component_tree_or_drawing_instructions architecture.

However, I think what @duttaoindril is worried about is a situation where you have a very large view_state. After it is deserialized in the shell, it is a whole new object which in many cases/frameworks will cause the entire shell to re-render. I think this would be true in the case of React. The render itself may not cause a DOM update but a full re-render of a large table of values, for example a spreadsheet, could take enough time to cause issues for the user.

Some frameworks like Leptos use fine-grained reactivity however with a whole new object on deserialization of the view_state in the shell I am not sure even fine-grained reactivity matters (I think Leptos would update every DOM element that listens to changes to the view_state).

How do you use Crux with an app that has many screens and/or a large view_state?

[charypar]

Yes, I can certainly see that there are scenarios where a wholesale re-render isn't really feasible.

We haven't tried this in practice, but the way I'd go about the optimisation for those cases is probably by avoiding the view API altogether and making a custom render capability - one which takes some payload describing the intended semi-imperative view update, keeping with the thinking that UI is a side effect.

I'm picturing something along the lines of:

caps.my_render(Screen::Profile(Panel::Sidebar(profile_sidebar_view_model))

Which would support sectioning the app into smaller chunks. This can obviously go as granular as it needs to.

Does that sounds like it would address the scenarios you're thinking about @matthewpflueger?