The Potential of Patterns

All software development is design. It consists of decisions. Whether it’s a payroll app or a next-generation computer game, the value of code and data is its information content. All our efforts to create software come down to creating new valuable information by deciding how to act upon existing information. This simplifies the process somewhat. There are many stages to decision-making, but at its core, this is the truth of all designs.

Decisions are affected in different ways as you take the steps to a conclusion. When you design, you turn the data and evidence into information by collecting, filtering, and interpreting. Once you have that information, you combine it with other information you hold: your memories, analytical processes, biases, and tendencies. The combination is your assumption of the true state of things and often includes what things should look like. You then generate a tactic to resolve that gap. That tactic is the action you will take, and the outcome is the decision made real. Changes are the realisation of decisions.

These steps can often happen so fast that we don’t even notice ourselves stopping at any of them on our way through to acting. When you want to get the largest value from a collection, you select a max function because you interpret the largest as the maximum without thinking about it. However, the route to action can sometimes take a little more time and investment.

Deliberate decisions

What slows us down is having to expend mental effort on any of these steps. In Daniel Kahneman’s words[TFaS11], it’s when we use the slow thinking part of our brain to make a decision. Many things can snag our minds into slower thinking.

• Unexpected data: When you come across data you don’t know how to interpret, you need to read up on how to analyse it or find new tools that can. When data seems missing or you can’t detect it, you need help getting what you need.
• Unexpected interpretation: If the data is familiar, but the interpretation seems odd, you seek causal relationships or advice from a person or other resource.
• Lack of assumptions: We might need to think more carefully about what data means and what we can assume because of it. Your findings may break existing assumptions, asking you to revisit past decisions.
• No default tactics: If conclusions are unexpected, you need to discover new tactics to employ. When an action is tedious or takes a lot of energy, that also slows us down, as we have to expend willpower to commit to the task.

But at some point, you can’t decide any more. There is a limit. Medical practitioners recognise decision fatigue, indicating, at least in my uneducated reading, that we can make people ill by creating environments which demand of them a high number of decisions. So, if we can’t increase the number of important decisions we can make well per day, we need to find out how to filter the decisions to only those that are essential.

Patterns help by making decisions easier or making them for us the same way coding rules and auto-formatting tools decide some of our actions for us. We can begin to find new processes to help us by measuring our productivity by counting valuable decisions made per day.

Reducing cognitive load by limiting the number of decisions that must to be made could be a positive effect of design patterns. They help frame data into interpretations and offer guidance on how to reveal the tensions in a situation. Tension is an is–ought state. Something is amiss. You can plainly see a violation, something which is but ought not to be or something which is not and yet should be.

Patterns can teach us tactics to use when resolving the gap. The framing is usually the problem statement, the context, and the forces. The body of the pattern guides the interpretation and suggests tactics to close the gap. The GoF patterns[GoF94] missed these steps. They did not structure patterns around what we would perceive before the solution was in place.

Discernment

Your ability to make decisions relies on information, but acquiring that information isn’t always a simple case of looking for it. Sometimes, you lack the skill or discernment to turn your observations into information with the required accuracy. Other times, you might need to learn how to recognise the information. And then there are times when you don’t even know you need the information.

Our ability to see things comes from our ability to discern—to tell things apart. The difference between one thing and another is purely in the eye of the beholder. We build tools such as microscopes and telescopes to give ourselves a way to extend our ability to see. Nonetheless, it still comes down to us using these instruments to augment our senses to detect meaningful differences.

The problem of discernment is a problem of training. We must attain a threshold level of ability to detect differences between materials, actions, and environments. Without it, the best decisions are beyond our reach. Good decisions are based on good information, and we define good information as allowing us to progress towards a good decision. It sounds circular, but it’s not. It hinges on good decisions—those which, given the constraints, introduce less waste or more benefit than any other alternative.

Wisdom and, by extension, a pattern, allows you to make good decisions where others may not even see options. If we only think of wood as a singular material, we won’t be able to decide what type of wood to use when building a structure. Someone with deep knowledge of the various qualities of different hardwoods and softwoods could make an educated decision about the best size and selection of materials for a particular task. Someone with a good grounding in concurrency patterns can better decide how to resolve distributed services.

Patterns might not always give you an answer or solution. In fact, wisdom can let you know when what you want is impossible, such as when someone asks you for a consistent, always-available service that handles network splits.

When we only see the surface level, we are blind to systemic problems. Being unable to derive an intent or detect the system behind a set of attributes often means it does not exist to us. And you cannot react to that which you cannot sense. Think about how you would not feel safe putting your hand into a drawer full of knives if your hands were numb with cold. When you have problem-centric patterns, even if they don’t provide solutions, they might still save your fingers.

The potential for patterns to replace discernment training is very high. Many organisations recruit for or train their members in skills to fill gaps in the ability to act but not in their capacity to sense. It is important to be able to make changes, but it’s much less valuable if you don’t know which change is best to make. Or whether a change needs to be made at all.

I have experienced three ways to gain discernment skills. You may have experienced others. There’s learning from experiencing it yourself. This is the slowest but the most memorable form of training, as the problems are personal. Then there’s official training, which doesn’t tend to stick, but at least it often comes with checklists to augment your senses. This applies to safety and security training, among other areas, as there is frequently so much to keep in mind. The last is via anecdote. I put patterns in this category. It’s someone else’s experience delivered in a consumable form. This is probably the best way to transfer discernment skills. You don’t have to learn from your mistakes, but the drama of following the story makes it stick much better.

Information

We can only make decisions based on information. Too much information hides valuable information in the noise. This is why we say timely, relevant information is valuable. We only want what’s necessary. But what is necessary?

• Anything that answers a question we actively ask.
• Anything that answers a question we didn’t know we needed to ask.
• Anything that reveals a new important question.

We can only make decisions based on processed information, which implies that some actions you take will cause a future decision to become available to make. This indicates there is an order in which decisions must be made—a good sequence.

DevOps seems to have emerged from an extended period when management looked at the wrong information to make their decisions. They looked at costs, not profits, as they were easier to measure and estimate. They had narrowed their attention to making sure everyone was working efficiently. It took Eliyahu Goldratt to shake them up and make them realise they should have heeded other metrics.

Eliyahu Goldratt’s book, The Goal[Goal84], demands that everyone scrutinise the flow of work, not the busyness of the machines. DevOps thinking is meant to capture this thought process, but we still find managers making sure everyone has work to do, not making sure important work is making progress.

Patterns have the potential to help by reminding us of what we want to achieve, which metrics to look at, and how to measure them. The wholesome design of a neighbourly street is one where it is inviting for children to play and sitting outside is pleasant. Metrics for this would be things like how fast cars are likely to travel down the road and how much shade you have in summertime. Planting trees gives shade, and making the streets narrow and corners tight slows the traffic down. The problem is defined, the metrics are known, and the tactics for improving them are offered—the solution is up to you.

Revelation

You cannot make decisions without some level of discernment. Decisions are based on information, and information presupposes a capacity to interpret data (the observable) into information (phenomena).

I prefer when the details of a decision are explicit and the consequences identifiable. These features allow me to be confident about my decisions and judgement calls.

When I think about design, I think about simulation and experimentation in the malleable fabric of the mind. In this sense, we can make decisions without having to do all the groundwork to get to the point where we have all the details. You mentally simulate the groundwork and have an idea of the actual situation before you get there. This gives you the context you need to make a reasonable decision. But a simulation is not the real world, and you are a human who can readily hold two contradictory beliefs simultaneously. Only the real world, with its consequences for contradictions, can give you the feedback you need. Only the real world can reveal the problems with your design. This is the reason why plans fail.

Design is a verb and a noun. The verb creates the noun the same way structure is a verb that creates structure the noun. Both are acts of connecting things to complete a balanced final form. Both are about resolving the forces in systems across multiple domains. But a lack of revelation limits design on paper and in the mind.

Christopher Alexander’s methods rejected theoretical design work—design purely on paper or simulated in the mind¹. Instead, his paper processes only existed to support organising his active methods—those methods that revealed the limits early in the process and found beauty in working with them. His buildings were not made for postcards or presenting to committees to get them to sign off on a budget. They were made more than they were designed, and made to work well in their environment for the people who were already part of it. In effect, he avoided compromising his vision by finding the right decisions to make early.

So, how do we avoid compromising our vision? How do we bring the right decisions to the front and make those good decisions early? Astute readers will have already made the leap at this point to the iterative approach of agile development, where working code is more important than following a plan. Working code is the same as physical examples and helps discover unexpected constraints and limits. It prefers smaller working pieces over making a beeline for the final destination, which is precisely the difference between Christopher Alexander’s work and the general process of modern architecture. This is why some saw Agile as the silver bullet.

We want these revelations in our design. We can use agile approaches, but is there an even faster shortcut?

In Pattern-Oriented Software Architecture Volume 5[POSA5-07], the foreword is by Richard P Gabriel. In it, he reiterates, ‘Design is the thinking one does before building’. The foreword leads down a path of thought touching on many aspects of design as an activity and an artefact, leading us to conclude that patterns help us avoid getting stuck in a rut with our biases and expectations. It’s a wonderful foreword I have read multiple times. Towards the end, it talks about AI-generated solutions for problems in response to problems that are intractable from both the outset and outcome.

This does not refer to the contemporary understanding of AI but to the genetic algorithm (GA) approach to discovering new designs. The approach of putting a generator and fitness function into a space to emulate random mutation and combination, as well as the selection process.

The core engine of a GA AI doesn’t have our biases. Training data brings that along later. It does not have our genes and therefore does not have our sense of quality. In the real world, evolution also does not have a bias but selects only as appropriate. Selection is not random and the candidates are, within bounds, arbitrary and free of intention. In effect, it is goalless and naïve.

Design is the result of the creation achieved while in the conceptual domain. That is, to design is to iterate mental constructions. Running a dream truck across a dream bridge is testing, if somewhat abstractly. When we design within our biases and expectations, it’s fast and safe. When we have a novel thought taking us away from our expected design, we call it a revelation. The AI way of designing is all revelation because it doesn’t have a normal approach, but it does have an externally provided means to recognise what is good—us, watching. Because it is so fast at designing, we add metrics as a procedural stand-in for our awareness.

It’s all revelation because a GA AI has no inhibitions. It will drive a truck over the bridge, sure, but it will also attempt to drive the truck backwards, under, through the water, and anything else we would have immediately dismissed. We see such assumption breaks in many examples of GA where people build fitness functions without laying down principles. One of my favourite examples² of this is a GA that was tasked with creating a walking human, but instead of making progress by walking, it rotated the feet beyond their natural limits, effectively turning them into wheels.

Design patterns cannot replace the childlike naïvety of an AI, but they give us something else. They provide someone else’s experience with their biases, not ours. Someone else’s rut can be just as revelatory. Patterns can offer a way to find other points of view for inspection. They could be a way to break us out of our rut because they reveal the thought process of another mind on a similar journey.

In summary, a design pattern has the potential to reveal the set of consequences and new decisions that must be made. It’s valuable because we avoid the extended physical and mental costs associated with gaining these insights first-hand. A well-written pattern identifies your next position for deciding the subsequent action. A pattern can tell you all the decisions you must make and the choices that are already determined for you.