The Myth of Scalability

It often comes up as a question: how do you build a scalable app? The simple answer is that scalability isn’t a thing. Scalability is a much more complex beast than that.

First of all, and as usual, I need to clarify what I’m saying and what I’m not. I’m not saying there is no such thing as scalability. When I say it’s not a thing I mean that it’s not a thing. It’s a number of different things, some of the least of which are used like they’re the exclusive meaning, while vastly more important factors are often ignored.

Dealing with Large Volumes of Users

So how do you develop for that?

At best, trying to build scalability into your application is a series of premature optimisations. But more importantly it’s pointless and almost certainly doomed to failure.

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming. — Donald Knuth in The Art of Computer Programming

At this point, you don’t actually know what the future bottleneck will be. Is your frontend availability limited by the CDN? Is your database slowing down at peak write periods? Are your API entities inefficient, requiring excessive HTTP calls? Does an important user-facing process require a slow disk write? Does your homepage have a crazy number of SQL queries?

The most common (and easiest by far) solution to these things is to throw servers at the problem. Maybe a load-balancer here, split read/write database servers, a varnish server there, more high availability instances, some Redis. More servers, more caching.

But more cost. These solutions inevitably come with a significant financial cost, albeit one that is way cheaper than ongoing development costs aimed at extreme automation. But why incur that cost before you need it? Moreover, why incur that cost to solve a problem it might not actually be a solution for?

That’s not to say some low-hanging-fruit isn’t worth picking early. For example, ORMs notoriously make bad queries in inexperienced hands, and the number of executing queries can often be reduced by an order of magnitude with a single tweak. By all means, farm off that slow user-setup process to a Lambda call and return a working session to the user in the meantime. Sure, a nice containerised application might well be easier to scale auto-scale later while also providing an easier deployment process now.

But too often I’ve seen people start off the development process by asking themselves how they can operate their not-yet-launched app at some sort of absurd Google scale. And often this means making bad technical choices for bad reasons. For example, I’ve seen people dismiss Ruby on Rails because it wasn’t fast enough for Twitter. Or dismiss PHP because Facebook had to replace it. But let’s be honest — they were fast enough for them to become Facebook and Twitter, right? And as we all know, only MongoDB is webscale.

For the most part, scaling (rather than scalability as a thing) should be a process, keeping an eye on what has near inexhaustible headroom and what’s under pressure. Systems rarely break from load suddenly. The key to technical scalability isn’t any some sort of magical architecture or approach — it’s monitoring.

Other kinds of scaling

Feature Addition

Designing for this (or against it, depending on your point of view) is difficult and requires experience. One big factor is technology choices. Certain technologies are inherently difficult to scale. A classic example is jQuery. While it’s great for a button that shows a box, these sorts of interactions multiply exponentially, eventually creating “spaghetti code” that inhibits further development.

This isn’t to say jQuery has to create spaghetti. Careful and disciplined use can indeed keep clean jQuery code, but there’s little inherent to the library to prevent the mess. Likewise, certain types of technologies and abstractions can help. As an example, a good ORM is a lot more scalable (in this respect) than raw SQL queries scattered through the application would be.

This type of scalability is an area where monolith frameworks like Rails or Laravel excel. The same applies to frontend tools such as Ember or Angular.

These frameworks provide architectural patterns and designs that help to maintain a relatively constant (if sometimes fairly high) level of complexity, regardless of the addition of features or the expansion of scope. This is where the term framework really becomes meaningful.

Company or Team Growth

Again this often comes down to technology choices. Learning a new commercial library or framework isn’t necessarily that difficult, even for a junior developer. What often makes things hard is learning your organisation’s idiosyncrasies. Your strange build tools. Your custom framework. Your undocumented dev tools setup or your “unique” testing approach.

Again, this is something monoliths help with. Being able to point to the documentation for Phoenix, .Net or Laravel and say “We do it like that” is a significant blessing.

New Markets and Products

Premature Optimisation is the Root Of All Evil

Senior Web Developer based in Bangkok, Thailand. Javascript, Web and Blockchain Developer.