With a few notable exceptions, software makers hate dealing with preexisting code. Sure, they’re happy to have tools that solve problems they have, but give them a bunch of code that “mostly solved” an existing problem or the opportunity to start over from scratch, and many would blindly pick starting over from scratch.

Easy-to-work-with software is straightforward, easy to comprehend, easy to add new features and capabilities to, and quite hard to break. And very little software has all of those qualities.

The reason for the greenfield preference is that most pre-existing software isn’t very good. It’s not that it’s not serving a business or that it’s not functional, but that it lacks qualities that make it easy to work with. Easy-to-work-with software is straightforward, easy to comprehend, easy to add new features and capabilities to, and quite hard to break. And very little software has all of those qualities.

Most in-use software in the world is significantly lacking in at least one of those areas. I’ll pick an example with which we here at Press Up are intimately familiar: WordPress. It’s used a lot, and performs many web publishing duties as well or better than anything else. And because of its heavy use of “hooks,” it’s relatively easy to extend (once you wrap your head around the whole event-based model). But it’s not very easy to comprehend — it’s a mess of unstructured flatness without much rationality or meaning to its structure. A function may or may not use certain verbs and conventions in its name or functionality. The parameters of a function call aren’t always easy to guess. And custom inscrutable functions call other custom inscrutable functions to some great depth in a large number of cases. WordPress has a bit of a reputation for this — the complaint often gets shortened to “spaghetti code” — but it’s hardly uncommon for any existing software project to be similarly problematic, if only in different ways.

Inherent Complexity: The Ways the Problem You’re Solving is Hard

This is inherent complexity: the difficulty of the actual problem that the software in question is trying to make simpler and better.

Part of the reason software projects can become hard to work with and not fun is that the problems they’re solving are hard-to-work with and not-fun. This is inherent complexity: the difficulty of the actual problem that the software in question is trying to make simpler and better.

All software has some inherent or essential complexity, this is why the software is being made in the first place. If the task in question is so simple that a person can thoughtlessly do it, the inherent complexity is low and it’ll probably be rather easy software to write. But, if a task is related to an area so complex whole industries exist around its periphery, you can bet there’s a lot of inherent complexity there. In either case the essential problem that the software needs to solve is the complexity “inherent” in the problem.

The prototypical example of a field with high intrinsic complexity is financial-regulatory-compliance. Because two large and well-paid professions — lawyers and accountants — are intimately involved with this, we can bet there’s a high level of inherent complexity. Figuring out an individual person’s tax return in the US will drive some professional accountants crazy, even with good records. And individuals untrained people encountering the problem justifiably feel overwhelmed. The whole area is a mess of rules and exceptions and exceptions to exception when this or that special case exists. We’re talking about a problem with high inherent complexity. It’s likely that the software here will necessarily have a high degree of complexity, especially when compared to something as “simple” as web publishing. (As a practitioner, I have to accede to the deprecating joke that web publishing systems are really just fancy ways jamming a bunch of strings together).

Incidental Complexity: When Programmers Make Their Lives Worse

Incidental complexity is anything about software that is hard but doesn’t really have to be.

I think I initially heard the incidental and inherent complexity frame from J. B. Rainsberger. In explaining it, he said something like: “inherent complexity is that the problem is hard, incidental complexity is because we [the makers of software] are bad at our jobs.” It’s maybe a bit bluntly stated, but it’s got a simple (painful) accuracy. Incidental complexity is anything about software that is hard but doesn’t really have to be.

All complexity in software can be accurately thought of as falling into one of the two sides of this dichotomy of inherent and incidental. Take everything that sucks about your software, subtract out anything that sucks because it’s a hard problem, and you’re left with the (molehill or mountain, I’ll let you choose) of incidental complexity that your project has accreted because the people working on it made mistakes in the past. (Though the analogy falls apart a bit, there’s a rough equivalence between this incidental complexity and the idea of “technical debt.“)

Incidental or accidental complexity‘s many many possible causes are beyond the scope of this introduction. But to quick-list some causes of incidental complexity I’ve been thinking about recently:

• Choosing the wrong tools. Sometimes programmers, either in a hurry to get started, or too in love with a specific tool, will use a screwdriver to pound in nails. Maybe it’s a three-page website that lead someone to use Ruby on Rails, or a RESTful web API that someone insisted needed to be built with Objective C, in any case it’s not the the tool isn’t good, just that it was applied to the wrong problem.
• Choosing the wrong abstractions. This is a subtler but still pretty easy one. Sometimes you’ll find software that’s all about specific vocabulary — perhaps library records — but was written using some other vocabulary, say a CMSes “posts” datatype. There’s an abstraction, but it’s not the right one, and everything was made much harder by the need to massage the disparity between the two concepts.
• Choosing no abstractions. This can initially seem subtle, but it’s similar to the case above. The difference here is that someone has created an activity logging system, but never created and sort of “activity” idea, which is aggregated into a “log”. Instead, they’ve got a very literal CRUD (create, read, update, delete) application without any abstraction at all. Database calls are hiding all the business concepts of it, and so the system is hard to reason about. Without any useful abstraction of the high-level concepts all behavior is expressed in terms of low-level of code and making changes requires thinking on many simultaneous levels of abstraction, which is quite difficult.

These aren’t the only causes of incidental complexity — failures to assure the quality of all element of your software as your create it, and failing to isolate discrete units in it are two huge areas that immediately come to mind. But the point about almost all incidental complexity, no matter what it looks like, is that it’s not strictly needed to solve the problem.

So, what does software complexity mean to me?

Obviously software makers, just as other people, are fallible humans who make mistakes. Some incidental complexity is almost guaranteed to creep into any software. But minimizing it is a powerful concept to understand and think about in the quest to build good systems and software that work not just on delivery, but long term for clients and partners. By understanding when your software is hard because it has to be, and when it is because it’s not built effectively is a powerful step toward being one who can regularly deliver great long-lived results.

Image Credits: dmjarvey, aidanmorgan, hikingartist.com