What do I want from a codebase?

Carson Gross’ “Codin’ Dirty” essay is designed to provoke controversy. Although I will critique the essay, I want to make it clear that I am (on this Thanksgiving eve) grateful for Carson Gross’s work.

Gross writes:

I’m … not trying to convince you to code dirty with this essay. Rather, I want to show that it is possible to write reasonably successful software this way and, I hope, offer some balance around software methodology discussions.

And:

Some people ship successfully while strictly following Test Driven Development, others slap a few end-to-end tests on at the end of the project, and many people end up somewhere between these extremes.

I’ve seen projects using all of these different approaches ship and maintain successful software.

This is not a good way to determine what methods tend to produce the best results. As I’ve written before, we have a growing body of industry research that allows us to go beyond having opinions to judgments informed by empirical evidence.

Our choice as developers is like that of Cesare Cremonini when Galileo offered him the telescope. We can take an empirical approach, or we can rely on those anecdotes that (conveniently) support our opinions. I don’t think this is a lost cause. Evidence-based medicine won the day. I’m optimistic evidence-based software engineering can do the same.

But I want to ask a different question than whether the opinions in “Codin’ Dirty” are consistent with the best available evidence. What do I personally want out of a code base? And does “codin’ dirty” get me there?

What I don’t want

If I imagine I’m starting a project for the first time what do I hope to find?

It’s easier to say what I don’t want. I don’t want a code base that:

• Is harder to adapt to new requirements over time
• Doesn’t tell me immediately about a regression
• Tells me that there is a regression but not where exactly it is or why it matters
• Doesn’t express the core business domain in a unified place
• Requires that I understand everything to do anything
• Expects me to learn the business domain independently of the code base
• Requires alterations throughout the code base to change a requirement
• Fails to discourage bad design choices

A few of these are obvious. Some need more elaboration.

Clear Expression of the Core Domain

The term ‘core domain’ might sound like jargon. For most developers working on line of business applications, the core domain is a concrete business process. It’s what people actually do, independently of whether it is implemented in a system we own, or even on a computer.

I want to be able to sit down at the code base and, for each business process, find code that clearly expresses:

• What the relevant actions are in a domain (e.g., submit a claim, receive a deposit)
• Under what conditions those actions can be performed (e.g., receipt of an application, account active),
• Who can perform that action (whether a human with a certain role or a machine)

Here’s an example from limn, a library of mine that implements artifact-oriented business process modeling in Clojure:

(def mow-lawn-spec
  {:workflow/name "Mow the lawn"
   :workflow/actions
   {:get-gas
    {:action/requires #{}
     :action/produces #{:mower/fueled}}

    :stop-running
    {:action/requires #{:mower/fueled
                        :mower/running}
     :action/produces #{[:not :mower/fueled]
                        [:not :mower/running]}}

    :start-mower
    {:action/requires #{:mower/fueled}
     :action/produces #{:mower/running}}

    :don-safety-glasses {:action/produces #{:worker/prepared}}

    :cut-grass
    {:action/requires #{:mower/running
                        :worker/prepared}
     :action/produces #{:grass/cut}}}})

You may not understand Clojure. But do you have any trouble understanding what this code is about? Any doubts as to what actions can be taken? Any confusion about the conditions for those actions?

You might need a few hints: {:key :value} is a map, #{:a b} is a set and so on.

What I don’t want is to have to:

• Fire up the application and poke around to try to figure out what the business logic is
• Depend on asking other developers or product owners about the business so I can understand the code
• Execute the code in my head to try to work backwards to the business rules

A good test is to ask a product owner to look at the business logic. If they can’t tell what it is, and have a decent idea of whether it matches their understanding, it’s probably not “business” logic – it’s whatever happens between the UI and the database.

What about non-enterprise applications?

Sometimes the core domain is more technical. For example, I once worked on a browser based telestrator – essentially a video editor tailored to add special effects to sports videos. This isn’t the kind of thing that could be implemented in any other way than on a computer. But how it is implemented – whether through WebGL, a helper library like three.js, direct manipation of a canvas – is a detail.

Even in this case, I should be able to tell what users can do (e.g., add a spinning circle under a player’s feet) by looking at one part of the code base, without firing up the app. And certainly without having to understand the details of an OpenGl implementation.

Easy to Fit in My Head

For any given part of the code, I want the problem space to be quite small. I don’t want to have to understand everything to do anything.

This means that I want to be able to understand what a function/method/class does without needing to know how it does it. If I want to understand the business rules for how a business entity can change in response to some event, and I need to worry about cache invalidation – not good.

In a business application, problems will typically be subdivided along categories like:

• Business rules
• UI logic
• Persistence
• Reports

A classic anti-pattern is embedding a business rule (e.g., users without role X cannot do Y) directly in the UI, such as disabling a button for action Y when the user lacks role X.

Changes are localized

If I have a change to a business rule, I don’t want to have to go hunting throughout the UI to find all the places that need to be changed. If I have a change to the UI, I don’t want any risk of breaking some non-UI concern.

Changes to an application typically fall into distinct categories. For example:

• Pure UI updates. The business rules remain the same, but how information is displayed or how users take actions change
• Persistence optimizations. In order to improve performance (e.g., for a list view) we might need to create a specific materialized view. No business logic or UI changes occur.
• Use case adjustments. Perhaps a use case changes, while the invariants expressed in a business rule do not.

In each case, there should be a part of the code base that changes together. If I have a change to the business rules, for example, it should be easy for me to find and change one place. Again, if I make a purely UI change, I should not worry that I have broken something that may be called in, say, a function invoked by a Kafka consumer.

What I don’t want:

• I make a change in one place to fulfill a requirement, and then track down the other places that need to be changed
• I make a change in one place that has unexpected consequences

Make Poor Designs Painful

I want it to be hard to write a codebase that is hard to change, fails to minimize the problem space, propagates changes throughout the code base, etc. And I want the difficulty to show up as I’m writing the initial implementation.

In other words, I’ve said what I don’t want – but how do I make it painful to write a codebase like that? Should I depend solely on my skill and that of my team? Should I make daily affirmations like “I solomnly swear to practice information hiding”?

This is where practices like test-driven development (TDD), within the broader context of Continuous Integration and Continuous Delivery, help enforce good design.

TDD makes writing bad code bases hard. Coupling business logic to the database is going to require extra setup. It’s much easier to abstract business rules and test them with unit tests. Writing integration tests to cover the combinatorial possibilities of use cases is both tedious and error-prone. Writing end to end tests (perhaps because the business logic is stuck in the UI code that enables a button) is not only painful in the short term, these tests take forever to run in the long term. In my experience, maintaining them is awful.

The One True Way?

I do not think TDD is the one true way necessarily. But I’ve never found something as effective at fostering the kind of code that I’d want to work in over the long haul.

Does slapping some end to end tests together do the same? I have never found this to be the case. End to end tests are not fast (certainly in any reasonably complex business application). They cannot compete with unit tests when it comes to isolating the locus of the problem. Nor do they discourage coupled implementations. My experience here dovetails with DORA’s research:

CI requires automated unit tests. These tests should be comprehensive enough to give you confidence that the software works as expected. The tests must also run in a few minutes or less.

Creating maintainable suites of automated unit tests is complex. A good way to solve this problem is to practice test-driven development (TDD), in which developers write automated tests that initially fail, before they implement the code that makes the tests pass. TDD has several benefits, one of which is that it ensures developers write code that’s modular and easy to test, which reduces the maintenance cost of the resulting automated test suites. Many organizations don’t have maintainable suites of automated unit tests and, despite that, still don’t practice TDD.

What about “integration tests”? The term is more ambiguous than “unit testing”. Sometimes people mean by integration tests the same thing as Fowler does by sociable unit tests.

It’s unclear how Gross’s recommendations encourage the qualities I value in a codebase. Nor that they do much to actively discourage a tightly coupled code base. Is it the case we want different things out of our code? Or is there a further story to tell about how “Codin’ Dirty” gives immediate feedback driving us toward maintainable, expressive code bases?