Tuesday 4 March 2014

GUI Event Handling with a Functional Hierarchical State Machine


A journey toward a state-machine without mutable state, but enough flexibility to express GUI behaviors. This is nothing new in it's parts, but maybe as a whole. In scouring GUI lore there is very little in a functional style.

I had a "feel" for how I wanted event handling to be, but it wasn't clear. Initially I wrote some wishful thinking pseudo-code:

  let tooltip ?(delay=1.) ?(show=5.) ?(style=tooltip_style) text =
    on_enter (fun id ->
      after delay (fun id ->
        let tip = make_tooltip style text in
        after show (fun id -> delete tip (* and what is the active state now? *))
      )
    )

I like the style of chaining functions which are awaiting events to continue. Another variation I considered was with a yield-like operation, but that looked like a vertical stack of statements without clear structure.

Anyway, the above example was really declaring a state-machine with each state able to return a replacement state.


Playing with State Machines


I drew some state diagrams, and they were as messy as they always seem to be -- special cases and redundant states to deal with "one thing is different, even though mostly things are the same as this other state". After a lot of futzing about I realized some details: to avoid a redundant explosion of states I could leverage parent states which have child states and the parent transitioning also cleans up the children. This is because state machines often have exception-like transitions. Here is an example of the full tooltip:

        start:(on_enter)
                  |
                  v
    (on_leave (on_use (after delay)))
        |         |        |
        v         v        v
      start      ( )  (after show)
                           |
                           v
                          ( )

So, at "start", we are awaiting on_enter. Once that triggers, we replace the current state with one having a (parent (parent (child))) chain of on_leave, on_use, and after delay. This means the delayed show and hide of the tooltip is on its own, but can be interrupted by either on_use (say, clicking the button) or on_leave. For example, if the initial delay is passed, the tooltip is displayed and the current state is:

    (on_leave (on_use (after show)))
        |         |        |

Here, after show has replaced after delay. Now if on_use occurs, it transitions (to nothing) and also removes its child state, leaving us with:

    (on_leave)


And in Code


First, a general n-ary tree, and a type consumed to specify whether a matching event terminates at this handler, or propagates onward.

  type 'a tree = Child of 'a | Parent of 'a * ('a tree) list

  type consumed = bool

Then the recursive types to express these hierarchical states and transitions:

  type state = {
    handler : handler;
    cleanup : unit -> state tree list;
  }
  and handler = id -> event -> consumed * return
  and return  = Retain | Stop | Follow of state tree list

It might be easier to see how it works with an example. Here is the corresponding code for the "tooltip" handler:

  let tooltip ?(delay=0.6) ?(show=5.) create_fn =
    let open Event in
    let rec start () = (* start needs an argument to constrain recursion *)
      on_enter (fun id -> Follow
        [ Parent (on_leave (fun _ -> Follow [Child (start ())]),
          [ Parent (on_use (fun _ -> Stop),
            [ Child (after delay id
                (fun id ->
                  let tip = create_fn id in
                  let cleanup () = delete tip; [] in
                  Follow [ Child (after show id ~cleanup (fun _ -> Stop)) ] )
            )] )] )] )
    in start ()

A little clunkier than my wishful thinking at the start, but I prefer this because it works! Also because it's clear about some things that the first-guess didn't cover. It deals with "on use" (interacting with something should make the tooltip go away), and has proper cleanup. The optional cleanup function is called when a state exits, including due to a parent expiring.

Note the square brackets everywhere: a bit noisy, but the returned replacement state is actually a list. An event handler is really a list of handlers which can each have children... so it's a full tree. Even the cleanup function returns such a list, empty [] in this case.


Composing Event Handlers


With my initial wishful thinking I didn't know if I could practically declare an event handler "tooltip" and simply attach it where I want it... but that's exactly how it works:

  Event.set button
    [ Event.Child (click_to_toggle ());
      Event.Child (hover_fade (0.85,0.95,1.0));
      Event.Child (tooltip (label "Toggle me on or off")) ]

Tooltip in action
This is attaching three root-level event handlers to button, which each work independently and go through their state transitions. Processing an event returns a new list of handlers which replace the current handler (if there was a change).

I think what I ended up with is a fairly standard hierarchical state machine, implemented functionally (except where I bind a list of handlers with an ID, but that's part of the nature of the component database).



I don't have experience with GTK, Qt, Windows-anything. I've written simple hard-coded interfaces for some games, but usually avoid even that -- linking character behavior to (abstracted) device inputs. Now I'm working on a game which needs more mousey interaction. I know I should have looked into existing systems at least to inform myself, but those libraries are huge and I never liked the first taste. Maybe I'm also fearful that if I got to know them I'd want to have all the features they have... Experienced comments are welcome!

10 comments:

  1. Have you considered FRP?
    http://erratique.ch/software/react/doc/React

    ReplyDelete
  2. Yes, I sometimes try FRP, but I haven't figured out how to make favorable use of it yet. The ideas are very appealing. I suspect I need more familiarity with it -- that's why I keep trying. Alternatively, I hope someone will develop rich examples of FRP in practical use. A few of my modules have "open React" commented out, evidence of occasional experiments!

    ReplyDelete
  3. Tony, you have a mind that is charmingly untainted by the "classic GUI paradigm", and my wish is -- please keep it that way!

    ReplyDelete
    Replies
    1. That is fantastic encouragement! There's value in "not knowing" things -- A few rubes in every field to help keep a fresh perspective. ;)

      Delete
  4. Your RSS feed https://www.blogger.com/feeds/9108979482982930820/posts/default/-/OCaml changes the date of the posts (03/09/2014 ≠ 4 March 2014). If you can, please fix that.

    ReplyDelete
    Replies
    1. Hi ChriS, I'm vaguely aware of RSS, but have never used it, so please bear with my ignorance. I now did some reading on it... could it be that updates to the post are caught by RSS, and is there a way to control that? I updated the post today (03/09/2014) with minor readability improvements. I certainly don't want to feel shy of making edits, so if this is the issue, and you (or anyone!) have advice on how I might indicate to RSS that this is a minor update not worthy of broadcast... that'd be great!

      Delete
  5. The quality was great, and the seats were pretty comfortable, and because the hall is a bit small, you're in and out in a flash. So I’ll say I’m a fan, and look forward to visit again.
    venue Houston

    ReplyDelete
  6. It seems you'd like Elm :) Look at how it manages mouse etc. events in its examples! http://elm-lang.org/

    ReplyDelete
    Replies
    1. Yes, I like a lot of Elm. My OCaml code looks a lot like Elm code, so I figure a similar aesthetic sense must be driving Evan Czaplicki. :) I don't make browser content -- but it's nice to know I'd have a pleasant option if that changes!

      Delete