Have Clojure UIs Taken the Wrong Path? Part 1

    Architecture Clojure   engineering hypermedia rest

The Clojure community has focused on React-based solutions for complex front-end clients such as Reagent, Rum, Om, Re-frame, and Fulcro.

For all their differences, they follow a very similar architecture, making heavy use of client-side state and using RPC for client-server communication. We will call this the “React+” approach.

But is this the right choice?

I will suggest the answer may be negative. My suspicion is that as web UI libraries advance, the problems they solve are not essential. Rather, the problems are accidental; they are generated by the React+ architecture.

A more fruitful foundation for interactive web UIs in Clojure is, I will argue, an extended Hypermedia approach. I will clarify what I mean by the extended hypermedia terminology as we proceed. To give some initial indication, think of HTMX on one end of the spectrum and Phoenix LiveView on the other.

Over the course of this blog series, we will envision a future where we don’t have to worry about huge node_modules/ directories, JavaScript dependencies, a build process, externs, code splitting, front end routing, complex synchronization between back and frontend, hydration, running a CLJ and CLJS REPLs, shared cljc files, etc, etc.

First, let’s consider a brief overview of the main ClojureScript UI libraries.

React+

Reagent provides a very simple model. Store your state as ClojureScript data in an atom, provide a function that returns your view in hiccup.

But state management is complicated. Rum took the approach of entirely separating state management, enabling its users to choose whatever state management approach they liked.

Re-frame manages this complexity by introducing views on the state (via subscriptions) and using an event system to manage changes in state. Re-frame represents state changes and effects as data.

But as the size of the app-db grows, it can require significant discipline to maintain its coherent structure. It is very easy to get view components that are tightly coupled to the app db structure, and an event system that is tightly coupled to backend APIs. Local reasoning can be quite difficult, as things become spread out among event handlers, subscriptions, and the backend service.

Fulcro takes a few steps further, and improves on the ability to reason locally. Fulcro defines queries on the view components and composes those queries together to make API calls to the server using EQL. It then normalizes that data into a graph database.

Fulcro does a lot more: it has a built-in way of setting up the initial state. It provides a nice way of doing mutations. Its tight integration with EQL provides powerful mechanisms for graph queries. Fulcro also provides a rapid application development framework.

The proliferation of different UI libraries and frameworks does not result from fundamental differences on presentation. The complexity is not how to render a view; the complexity is managing state. React itself provides a simple model: application state → view.

It’s Really RPC

Let’s back up a bit and think about the data interchange between client and server.

Which of these looks more like a RESTful response?

Example 1:

{
  "tasks": [
    {
      "id": "TSK001",
      "status": "Assigned",
      "project": "PRJ0001"
    },
    {
      "id": "TSK002",
      "status": "Accepted",
      "project": "PRJ0002"
    }
  ]
}

Or example 2:

<tbody>
  <tr>
    <td><a href="/task/TSK001">TSK001</a></td>
    <td><span class="assigned-icon"><a href="/task/TSK001/accept">Accept</a></span></td>
    <td>PRJ0001</td>
  </tr>
  <tr>
    <td><a href="/task/TSK002">TSK002</a></td>
    <td>Accepted</td>
    <td><a href="/project/PRJ0002">PRJ0002</a></td>
  </tr
</tbody>

The answer is the latter. However, in a perverse irony, many developers today would confidently say it is the former.

Roy Fielding (creator of the term REST), expressed his frustration with JSON-RPC APIs being mislabeled as REST:

I am getting frustrated by the number of people calling any HTTP-based interface a REST API. Today’s example is the SocialSite REST API. That is RPC. It screams RPC. There is so much coupling on display that it should be given an X rating.

What needs to be done to make the REST architectural style clear on the notion that hypertext is a constraint? In other words, if the engine of application state (and hence the API) is not being driven by hypertext, then it cannot be RESTful and cannot be a REST API. Period. Is there some broken manual somewhere that needs to be fixed?

A key insight emerges when contrasting REST with RPC. With RPC, the data interchanged between the client and the server requires the client to have implementation time knowledge. That is, the client application must be custom-built for your particular application.

Contrast that with a browser. Do the makers of the browser need to know anything about our business’ app? This is the key difference between a hypermedia and RPC response.

Carson Gross puts it this way:

HTML encodes both the data about [your domain entities, e.g. tasks] as well as the actions available on that data.

Contrast this with a thick client, such as a standard mobile application. For a standard thick client a specific [business entity’s] screen must be built, with the actions on that data already encoded into the UI. The UI simply retrieves the data and then renders it locally, with the actions defined locally. To do something new you will need a new version of the [client] application.

[With HTML,] the client (a browser) doesn’t know anything about the data, it just knows how to render hypertext. A technical way to say this is that we are using Hypertext As The Engine Of the Application State.

This principle is essential to REST. Fielding again:

A truly RESTful API looks like hypertext… Resource representations are self-descriptive: the client does not need to know if a resource is from OpenSocial [substitute your domain entity here] because it is just acting on the representations received.

But why does it really matter? Is this really a question of labels?

No. The RPC architecture is neither easy nor simple. It results in a massive amount of unnecessary code and fragility as applications grow in size.

Unnecessary Code

Look at our examples again, using edn this time.

RPC example:

{:tasks [{:task/id "TSK001"
          :task/status "Assigned",
          :task/project-id "PRJ0001"}
         {:task/id "TSK002"
          :task/status "Accepted",
          :task/project-id "PRJ0001"}]}

HTML example:

<tbody>
  <tr>
    <td><a href="/task/TSK001">TSK001</a></td>
    <td><span class="assigned-icon"><a href="/task/TSK001/accept">Accept</a></span></td>
    <td>PRJ0001</td>
  </tr>
  <tr>
    <td><a href="/task/TSK002">TSK002</a></td>
    <td>Accepted</td>
    <td><a href="/project/PRJ0002">PRJ0002</a></td>
  </tr>
</tbody>

What can I do with TSK001? Where can the user find more information about it?

It’s impossible to tell from the RPC example. In order for the user to be able to do something with this response we need to ship (sideband) custom code. The React+ approach is to ship an entire custom application on top of the existing hypermedia client application. Only to ultimately generate HTML on the frontend.

By contrast, the hypermedia response contains, as data, both the information the user needs to see, and the controls the user needs to act on the information. A user can navigate to a task or project, accept a task, etc.

Consider the table below, where we ask what features we would have to write or modify a custom client for:

Feature RPC Approach Hypermedia Approach
Add the "created at" for each task Custom client code required No custom client code required
Add a "Decline Task" action Custom client code required No custom client code required
Remove the ability to Accept a task Custom client code required No custom client code required
Show the user that created the task Custom client code required No custom client code required
Show the current assignee for a task Custom client code required No custom client code required
Put a project on hold Custom client code required No custom client code required

You get the point: RPC requires a custom client with extensive implementation-time knowledge of the application. This is expensive! And what do we get for it?

The same HTML, usually buried in div soup: <div><div><div>....</div></div></div>

The innocuous terminological questions “REST” and “hypermedia” turn out to be the small cracks that opens down into a deep, yawning chasm: all the custom code we do not need. The custom code we may never have needed.

Separation of Concerns

One of the myths that appeas to Clojurians is that this is data:

{:task/id "TKS001"}

While this is not:

<a href="/task/TSK001">TSK001</a>

But this is false. Both the former and the latter are data. But can the EDN claim to at least separate concerns?

In fact, the exact opposite is the case.

Let’s return to our original example with the RESTful response and the JSON response. An approach that separates concerns will allow the client to be focused on UI concerns, excluding any special knowledge of the higher level concerns (such as business logic).

Suppose that the user should be able to accept a task. How does the client know, given this response?

{:tasks [{:task/id "TSK001"
          :task/status "Assigned",
          :task/project-id "PRJ0001"}
         "..."]}

The answer is to put business logic into the client. There is no way for the browser client to know how to represent this to users so that they can see the information they need to see, and take the actions they want to take.

With React+, we build a custom client and stick the business logic in it. The client must know the business rules that, given this user (using sideband information), and this task status, the the accept action is available.

Far from separating concerns, the data API approach requires violating the separation of concerns. It involves creating a custom client, coupling it to EDN or JSON API, maintaining large amounts of state in the client, and spreading our business logic across both the server and client.

By contrast, consider whether the client needs to have any special knowledge of the business rules with the RESTful response:

<tr>
  <td><a href="/task/TSK001">TSK001</a></td>
  <td><span class="assigned-icon"><a href="/task/TSK001/accept"><i class="fas fa-check"></i>Accept</a></span></td>
  <td>PRJ0001</td>
</tr>

Of course not! No client code of any kind is needed to understand this.

Coupling is not determined by how we feel about our code. It has a strict definition: A and B are coupled if, for a given change Δ, changing A requires changing B.

Applying that to our case, the client and server are coupled if and only if, in order to change whether the user can accept the task, changing the code on the server requires changing it on the client.

Fortunately, hypermedia allows us to make these changes without opening a PR on Chrome.

Further Reading

The Hypermedia Systems Book is an excellent contemporary discussion of the significance of hypermedia.

Coming up…

In the rest of this series we will debunk some myths:

  • Following a REST or Hypermedia approach is incompatible with rich client interactivity
  • Hypermedia == Plain HTML
  • React+ is required for things like infinite scroll, inline edits, and offline functionality

And we will discuss an alternative path, extending the hypermedia approach to achieve the same (or better) functionality with vastly less complexity.

Thomas CothranSoftware Developer

FP, Clojure, Domain Driven Design