Unresolved Forces - Copyright Richard Fabian 2024

Use systems theory

The only way to beat a system is to understand it. If we want to develop our processes into healthier ones, we need to understand the system reinforcing those that harm us. There are many elements inhibiting a better way of developing software. Understand what supports those elements and work on changing the feedback loops. Only then can we control how we are affected by them.

This is true for not just the large but also smaller things. Not just for processes that create problems for people but also for the tiny details of deciding what hardware to buy and when to migrate to a new programming framework or language.

Look at how each stock affects the flow

Stock is anything you can measure: staff wages, toner, office space, products sold, or electricity bills. Many of these stocks will affect the rate of flow of the other. If you’re unaware of the flow between stocks, it will be like trying to optimise code without profiling it.

• Balance hardware costs against other costs. Bigger hardware costs more non-linearly. Cheaper hardware makes iteration times longer, and productivity increases non-linearly with iteration frequency.
• Balance features against maintenance. In some industries, features take priority over bugs, but even when they do not, we must keep a balance. Without balance, features are slower to develop, and a contradiction occurs.

Look at how the sensors in the system affect the flows

When you look at profit first, you maximise productivity. When you look at costs, you maximise efficiency. Which makes the company grow? Which gains you an outstanding reputation? Which makes staff happier and is more inviting for top talent? What organisational patterns do you see reinforcing these response mechanisms?

Sometimes, a sense is poorly calibrated. When a team is not performing well, it sets a standard. Poor previous performance can lead to ‘it’s the best we can expect’. Now, we have a drift to low performance. In effect, standards are affected by history, and expectation is affected by standards. So, keep performance standards absolute, or let them rise with each success. Move to a drift towards higher performance. What patterns of behaviour in your community lead to a negative drift and which to a positive?

Dangers of metrics

The environment will always define what is the fittest. If the environment includes some perverse metrics, then the fittest will be equally perverse. Societies valuing wealth and fame create beings with interest and skill in acquiring both. A workplace showing appreciation for people making a visible effort to do more work and praising people who push harder to get through over-commitments will create an overtime culture, not a culture of productivity. People willing to commit to working longer hold each other accountable to do the same. What patterns can you introduce to reject detrimental metrics?

Rewarding the wrong thing can be dangerous. A health professional who does not maintain the right amount of downtime becomes sloppy at work due to overwork or too little rest and causes more damage than is repaired by the work they do. Working hundreds of hours a week as a psychologist, judge, or doctor is irresponsible. Working longer hours to do more work makes little sense for air traffic controllers, video surveillance technicians, or any other job where a lack of concentration on the task at hand can cause cascading failure. But programming doesn’t require concentration, so 60-hour weeks are fine for software development, right? What pattern of forces and context led us to this state?

Reinforcement and re-evaluation

Organisations are systems. They drive what we do, what we can do, and who we are when we are part of them. When introducing a new rule into a system, we need to evaluate whether it makes things better. That much is obvious. But we also need to assess whether it makes ‘making things better’ easier or harder in the future. Some flawed rules can be easy to spot. A good rule that halts growth and adaptation is dangerous. Like evolutionary dead ends, they might be good right now but eventually cause problems. Pinhole camera eyes and employee timekeeping rules share this trait.

If your fire register turns into a timekeeping exercise and Dave notices John isn’t turning up on time, they may decide there needs to be a policy about it. If the policy is enabled, everyone knows management is watching them for failure to comply. Knowing your management is watching employee timekeeping leads to some people believing performance isn’t measured. Disengaged employees will be rewarded for arriving on time and won’t get the support they need to regain that sense of purpose. In contrast, those who care about the company and what they are working on may get chastised for being late or going home early, even if they’re producing 10 times the output of the ‘warm-body-on-a-seat’ worker.

Rules can and should be replaced. Make sure your rules are set up so they can be re-evaluated. Guarantee the criteria against which they are evaluated can also be re-evaluated and changed. Without adaptation, there can only be revolution or annihilation.

Understand feedback

Feedback comes in many forms, but a lack of feedback is the most dangerous. Some people just want to get on with their work. But ‘get on with their work’ is anti-team. Checking for null and returning to get back to work hides a problem and smothers the feedback. Seeing a garbage value in some data and simply clearing it out without checking is just as bad. Did they not ask why there was garbage? Were they even sure it was garbage? One person’s garbage can be another person’s hidden side channel for unsupported data. Do not shy away from errors by hiding them behind validity checks, as bugs will come and triple any pain you feel. If feedback is essential, then a null check is a design error. Numbness kills.

An error is a poor friend but a great teacher. So long as you recognise that errors are lessons and not punishment, you will naturally get better. It is feedback you can act on. If you cannot see your error, you will not learn. If you do not see the error as a lesson, you will not learn.

There’s a good pattern for error codes. State the type of error so that it shows which way to proceed. Keep the root cause and message’s source away from the error code and expose that extended data through a different mechanism. Some systems use error codes with message objects, such as exceptions. Others use a numeric code and a global reason-string-getter. The solution is up to you. The pattern is only the wisdom to return a code that indicates the direction to success for the user and raise it as soon as possible.

However, errors mean different things for users and developers. Developers need details and information about the right way to handle an exceptional situation. Users need indicators to help them get back on track. A developer needs a relevant documentation page and error code. A user needs visual feedback, live checks, on-screen rules, an indication of what was wrong (think about password entry boxes), and the ability to recover without rework (think about partial undo mechanisms).

In case it’s not clear, a programming language has developers as users, so the language should provide live, clear, visual feedback on screen that the language syntax or structure is not currently correct. Many people love the Rust programming language because it treats developers as users.

Some thank you for what you do. Some blame you for what you fail to do. Many will thank you for fixing what went wrong. Few give thanks for preventing problems in the first place. Some will blame you for not achieving the impossible. Some will praise you like a hero for doing what must be done. There are many asymmetries like these.

Make your application’s document states visible and provide relevant feedback on future actions. As advertised by Brett Victor¹, visualise the present and future state. Give the user predictive powers and perfect memory.

The ‘Ship early and ship often’ pattern can be applied to internal projects. Stakeholders can provide feedback quicker (less waste and more confidence). Only build up small sets of changes and try to commit them in small packets. Choose between feature flags and feature branches but have a good reason to choose the latter. If you’re trying to change how you work, making changes and evaluating frequently is essential.

Journalling is a great pattern. Writing reports makes you smart. Saying what you did helps you learn from your mistakes and allows others to help you by providing feedback on things you’ve misunderstood.

The same thing can have different feedback if presented in different ways or at different times. What an observer can grasp, they will evaluate. For example, try to avoid adding visual style to a mock for layout or function. Otherwise, that will be evaluated, and you will get feedback on the wrong aspect. There is a hierarchy in how we assess things. Anything that is considered aesthetic will take centre stage during evaluation. This is why there are page formatting and layout rules for submitting screenplays; the approach is an equaliser.

Fixing anti-patterns

Anti-patterns are the same as patterns in that they self-form, indicating their stability. They are negative feedback loops in equilibrium. That means they push back when you try to fix them.

When a corrective measure doesn’t fix an anti-pattern, it’s usually because of policy resistance. The effort doesn’t fix the problem but only affects the flow in and out of the system. Real-world examples include opening a window when the room is hot, but the heating kicks up a gear because you didn’t change the thermostat.

The idea is simple. If you know what is wrong but attempt to address the problem directly, the system will maintain the state despite your interference by adjusting different flows elsewhere. For example, when governments introduce measures to reduce the number of fishing boats to help replenish fishing stocks, what is to stop the boats from getting bigger or the trawlers from fishing longer? Another example would be spaghetti code. Attempts to fix it make things better, but things revert quickly without addressing the culture around the original code. A quick change to get a feature complete or a bug resolved will likely require some additional unwanted pasta to get the job done. This regression wears out the programmers who tried to undo the damage—ultimately bringing the anti-pattern back into equilibrium.

To stop unacceptable behaviour, you must understand and address the system, not the symptom. If you don’t like a particular behaviour but don’t understand what stimulates it, you leave the original trigger open to find an alternative source of satisfaction.

An example I heard while listening to the radio one day was about drug testing in prison. Bringing in a test for drug usage caused the prison population to move to a harder variety. They explained² that harder drugs were less detectable by the tests. Those in charge did not address the tendency of the prison population to use controlled substances in the first place.

The basis of any approach must be to avoid fighting the system and instead coax, coerce, and convince it to follow a new path. When you fight a system, the same mechanisms that built and stabilised it activate and defend the status quo. These constructive, repairing forces are usually more effective than any individual pushing against the system.

If you say you believe test-driven development (TDD) is good but find yourself writing code without writing the test first, analyse the thought process behind your actions. What triggered writing without a test? Did you want feedback from the result of the code faster? Or did you feel the test would be hard to write? If it’s the former, perhaps you’re in an exploratory stage and don’t believe TDD applies there. If it’s the latter, the problem may be that the design needs refactoring so that writing tests becomes easier.

Remember, systems protect themselves by being invisible, so detection requires good metrics. Observe actions, analyse triggers, and actively decide what and how to change. I have found that journalling helps me reflect on triggers and how I responded. You might find coaching sessions or other forms of guided self-feedback help you discover what systems of feedback limit you.

Systems work whether we understand them or not

It takes great character to admit we don’t have direct control. Rather, we must accept we need to understand the systems controlling us before we can appease them. Systems don’t care if you aren’t aware of the flow of logic behind them and how forces propagate; they will affect you regardless. Many claim evolution doesn’t work because they don’t understand how it could work. Unfortunately for them, the evolutionary process doesn’t care whether or not you comprehend it. Natural selection will carry on adapting organisms to the environment either way.

Additionally, none of the systems we inhabit can be correct overall. Some aspects of each system are good for some inhabitants, but pleasure for some is invariably pain for others in any zero-sum system. If you want to make a new system work well, you cannot suggest or attempt to apply a change that makes it worse for those currently in power without a revolution. You must migrate slowly towards it via changes that have no apparent direct negative effect on those in power, so they can’t or won’t fight back against the change. You change a river’s course by digging a better route, not by building a wall.