On Choosing Boring Technology

Note: the ideas in this post have already been covered numerous times in the past. One article that has greatly influenced my perspective over the years is McKinley's Choose Boring Technology. Below I'll explore the topic from my own experience, and how I ended up using Kubernetes for a recent project.

Over the years I have observed that many engineers tend to attribute much of the success or failure of a company to the technical choices made. I know I’m often guilty of this too. And while it is often justified, I would argue that for the vast majority of startups out there, the choice of programming language, framework, or even database doesn’t matter that much. This seems especially true during the early stages.

Through the engineering lens

This perception is understandable, we as engineers tend to look at the world from a specific lens, and are often biased by what we know best. Our daily activities may include things such as debugging CI pipelines, implementing new features, pairing with colleagues, or migrating the always present legacy codebase. The environment that surrounds us makes it easy to believe that it all boils down to those things that we see and understand. It’s an illusion that makes us feel like we’re fully in control of what makes or breaks the product.

Don’t get me wrong, it can be a huge advantage for many companies to make their product 3x more efficient than competitors, or to have elegant, composable code. But you might be focusing on the wrong problems if nobody cares about the product you’re actually building, and sooner or later your business will hit this wall.

I’m not saying that software doesn’t matter. A solid foundation for your startup goes a long way. If investing in this allows you to build better features faster than your competitors, more power to you. But finding the right balance is highly dependent on what you’re trying to solve and the resources you have at hand. There’s no right or wrong way to do it, and as usual, it mainly comes down to tradeoffs.

Boring is less surprising

I believe aiming for a healthy balance of risk vs reward when it comes to your technical choices is something to strive for. In particular, if it decreases the chances you get stuck on the wrong problems down the road.

Boring is often interpreted as “picking old technologies over newer ones”, but it doesn’t necessarily mean that. For me, this comes down to using proven technologies in which the ways it can fail are mostly known, but occasionally experimenting with different, possibly newer tools that might suit the problem at hand better.

Maybe you want to gain more experience by using the latest framework or programming language, or you just want to have some fun. You do what makes you happy. But if you’re trying to make a decision to increase the odds that your product or business will succeed, it’s worth stepping back and considering your options.

For me, mainly choosing software that has been around for longer is not about it being boring or older, it’s about the fact that the ways in which it fails are better known. There are fewer unknowns for you to deal with and this maximizes your chances of actually shipping the project.

For example the other day I had an issue with my Django app, and a quick search led me to tens of answers to this problem in various forums and websites. It took me at most 10 minutes to get back on track and that was the end of this issue.

I experienced the exact opposite a few years ago with a popular, but not so battle-tested Scala library my team had been using for a while. We were probably among the first to encounter the issues we were facing, and it seemed nobody had walked down this path before. Maybe it sounds like a fun challenge or a great chance to contribute back to OSS (which I’m happy to), but once you solve it, do your customers really care about it? How many days, weeks, or even months are you willing to invest in such issues? In my case, I’d rather use that time to ship new features or improve the existing ones.

Exploit vs explore

I try to follow an 80/20 distribution when it comes to my choice of tools. This means my stack consists of about 80% software I already know well, but I do allow myself 20% of the stack to explore tech I have less experience with. The exact ratio is not what’s important here, it’s more the fact that you should lean towards using proven technologies.

This also resonates with how Multi-armed bandits work. You try to maximize your expected gain by taking advantage of what worked well in the past, while sometimes exploring new things to avoid missing out on a possible goldmine.

A more recent example of mine started as an embarrassingly simple Django app with no charts, all metrics were rendered on a plain HTML table, and all data was stored on a SQLite database. Took literally a weekend to get it up and running including manually deploying to a $5/mo VM. Low risk and high reward for my needs at the time.

Fast forward and as I added more features and began handling more page views for various websites, I started to notice that the codebase could use some refactoring. It also became increasingly repetitive to do things like deploying to new instances, issuing SSL certs, and keeping the DNS records up to date in case the IP address of my instances changed.

As a second iteration, I upgraded to a docker-compose setup plus lots of glue code. But soon enough I found myself reinventing what other tools already do well. There are multiple ways to solve each of these pain points, but in my case, it came down to using a tool I am very familiar with from my full-time job: Kubernetes.

Yes, I am well aware it can be overkill for a lot of projects out there, and I could have gotten away with a more traditional solution. But it works well for me, and I have been running production workloads with it for several years already. Please do not just blindly follow my path. Use what you already know best.

Kubernetes allowed me to simplify the operational aspects tremendously, and I feel comfortable debugging issues with it after having the pleasure of putting down multiple production fires for my employer over the years. It has also been around several years, there's lots of documentation, and a huge helpful community who can help. It'd argue that there's more documentation available than for any home-grown deployment system, or even EC2/Lambda/DigitalOcean for that matter.

As an added benefit, it also made it trivial when I migrated from DigitalOcean to Linode, and most recently to AWS (each migration took mostly an evening of changing my Terraform files and deploying them - I’m being serious). But that’s for another post.

Let complexity come over time

My point is, I moved into these technologies as the pain with the previous solution was higher than dealing with the new tech. But more importantly, it helped me ship features even faster to my customers while reducing the operational overhead for me.

If I had started with the more advanced setup from day one, I might have lost all motivation before I would have had the first version of Panelbear. The key is to solve the problems that are getting between you and your goals, not potential issues you believe one day will be yours.