Great designs usually emerge from iteration (which sounds much nicer than trial-and-error). The engineering team moves from an initial design through the life of the project performing several more or less painful changes that naturally lead to bigger entropy, while at the same time push hard to get it back on track through a series of refactors. On this way there are several traps, and it is not always obvious or even possible to avoid them, because of all the conflicting forces acting on the project.
In the beginning, you come up with an awesome initial design/architecture. There are lots of traps to be made here, like over-engineering by adding lots of useless abstraction layers that allow you to change even the implementation of the floating point numbers without affecting the rest of the code. On the other extreme, you can also forget about the fact that you will probably need to extend it in unexpected ways, and therefore not make it flexible enough to survive a couple of versions. Or you can think that you know better than your user and design for yourself, which is even worse.
Let’s focus just in one of your architectural layers. One of the trickier traps is that your design is going to impose specific restrictions, but you won’t realize their impact yet, because by relying just on the knowledge that you have so far, there is no way you can think of a reasonable use case that won’t fit this design. In fact it is very likely that you are aware of some of those restrictions, and you even find them desirable, because they are in place to prevent bugs.
As you keep having new requirements, you start patching the software to introduce new awesome features. And after a bunch of patches, the trap becomes evident. Suddenly, you find a requirement that can not be met with the existing design due to the restrictions. Sometimes it is some fundamental axiom that suddenly happens to depend on new context, and some times your taxonomy turns out to be imprecise.
Here comes the second tricky trap. Since changing the design is a lot of engineering effort, and you have a deadline to meet, you decide to use some hacky way to adapt the requirement to the existing design. You might even find a way to use the design that was not intended when it was conceived. This code is going to be painful in various ways, not only during maintenance of the hacky code itself, but also during maintenance of the design it hacks.
One of my favorite hacks is to abuse the strings. You need to add some extra piece information in a way that all the tools will preserve it even when serializing your objects, and you decide to encode it in some string, like a name or a description. It works, but it is slower because you have to encode and decode it even for in-memory operations, and suddenly even the good restrictions that were introduced to prevent bugs, are rendered void.
At some point you have so many hacks that it becomes evident that you need a redesign. From this point, you are probably going to iterate between simple refactors, complete redesigns and new hacks several times.
This might eventually create another trap, because the design is going to become too complicated in order to handle all the weird use cases that you keep finding in your way. This will become a barrier for anyone willing to adopt this design in new code. As you may already know, it is harder to read code than to write it, so you can easily imagine what comes next…
As the project grows, changing the design becomes harder, the pressure to build better designs from scratch grows like crazy, and new designs need to be tested in a small portion of the project before going to prime time. Therefore, you end up with two (hopefully only two) different designs running at the same time. One of the designs might appeal more some members of the team, while the other might appeal more other members, perhaps because each design fits better their respective areas of expertise. During the conflict, you can have some degree of fragmentation, with the design that is not yet ready for everyone, and the one that doesn’t want to die.
The simple yet complete design
Over time, you might figure out that the only way to win this battle is to have a simple yet complete design, and let any other part of the project wrap/extend it with the abstractions that better fit them. Eventually some of these abstractions could turn into a software layer on top of the basic design because more than one part of the project use them, but in order to avoid falling into the same traps again, you will not hide the underlying simple layer on those upper layers, which means that any possible feature will still be possible by just accessing the simple yet complete layer.
An example of a simple yet complete design is jQuery, where the base library is as simple as possible, with a set of operations that allow to do any possible manipulation to the HTML DOM tree. There are tons of extensions and wrapper libraries that target specific use cases, like jQuery UI, but you will always be able to use the base simple layer to do anything that is beyond the restrictions of those upper layers.
Making a software layer simple yet complete is not about making it draw windows, index data and make coffee, it is about doing a simple thing, in the most simple way, and doing it very well.
However, don’t think for a minute that you can skip the whole process and just do the nice simple yet complete design from the beginning, because it is not always evident how simple the design can be while being complete, and it is very exceptionally evident in the beginning what is the definition of complete. You have to go through some pain to get there 😉Thanks to Ricardo Rodríguez for reviewing this article.