The Myth of Scalability

It often comes up as a question: how do you build a scalable app? The simple answer is that scalability isn’t a thing. Scalability is a much more complex beast than that.

First of all, and as usual, I need to clarify what I’m saying and what I’m not. I’m not saying there is no such thing as scalability. When I say it’s not a thing I mean that it’s not a thing. It’s a number of different things, some of the least of which are used like they’re the exclusive meaning, while vastly more important factors are often ignored.

Dealing with Large Volumes of Users

In fact, I’d argue that the meaning most commonly given is by far the least interesting and important. The most common usage would be that an application can be used by increasingly large numbers of users at a time.

So how do you develop for that?

Would it be unreasonable to suggest you just… don’t? If you’re sitting at the start of an application now, scaling is the least of your problems. If you have no customers, no users, and little or nothing of a product, why are you focusing on size?

At best, trying to build scalability into your application is a series of premature optimisations. But more importantly it’s pointless and almost certainly doomed to failure.

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming. — Donald Knuth in The Art of Computer Programming

At this point, you don’t actually know what the future bottleneck will be. Is your frontend availability limited by the CDN? Is your database slowing down at peak write periods? Are your API entities inefficient, requiring excessive HTTP calls? Does an important user-facing process require a slow disk write? Does your homepage have a crazy number of SQL queries?

The most common (and easiest by far) solution to these things is to throw servers at the problem. Maybe a load-balancer here, split read/write database servers, a varnish server there, more high availability instances, some Redis. More servers, more caching.

But more cost. These solutions inevitably come with a significant financial cost, albeit one that is way cheaper than ongoing development costs aimed at extreme automation. But why incur that cost before you need it? Moreover, why incur that cost to solve a problem it might not actually be a solution for?

That’s not to say some low-hanging-fruit isn’t worth picking early. For example, ORMs notoriously make bad queries in inexperienced hands, and the number of executing queries can often be reduced by an order of magnitude with a single tweak. By all means, farm off that slow user-setup process to a Lambda call and return a working session to the user in the meantime. Sure, a nice containerised application might well be easier to scale auto-scale later while also providing an easier deployment process now.

But too often I’ve seen people start off the development process by asking themselves how they can operate their not-yet-launched app at some sort of absurd Google scale. And often this means making bad technical choices for bad reasons. For example, I’ve seen people dismiss Ruby on Rails because it wasn’t fast enough for Twitter. Or dismiss PHP because Facebook had to replace it. But let’s be honest — they were fast enough for them to become Facebook and Twitter, right? And as we all know, only MongoDB is webscale.

For the most part, scaling (rather than scalability as a thing) should be a process, keeping an eye on what has near inexhaustible headroom and what’s under pressure. Systems rarely break from load suddenly. The key to technical scalability isn’t any some sort of magical architecture or approach — it’s monitoring.

Other kinds of scaling

Moving on from this type of performance and load scaling there are more important kinds of scalability.

Feature Addition

Scalability can also refer to your application being able to grow, to have new features added. This kind of scalability is much more significant than the former. You can’t just throw servers or caching over it. It’s typically a technology or architecture issue, and solutions are hard-fought and usually involve a lot of rewriting.

Designing for this (or against it, depending on your point of view) is difficult and requires experience. One big factor is technology choices. Certain technologies are inherently difficult to scale. A classic example is jQuery. While it’s great for a button that shows a box, these sorts of interactions multiply exponentially, eventually creating “spaghetti code” that inhibits further development.

This isn’t to say jQuery has to create spaghetti. Careful and disciplined use can indeed keep clean jQuery code, but there’s little inherent to the library to prevent the mess. Likewise, certain types of technologies and abstractions can help. As an example, a good ORM is a lot more scalable (in this respect) than raw SQL queries scattered through the application would be.

This type of scalability is an area where monolith frameworks like Rails or Laravel excel. The same applies to frontend tools such as Ember or Angular.

These frameworks provide architectural patterns and designs that help to maintain a relatively constant (if sometimes fairly high) level of complexity, regardless of the addition of features or the expansion of scope. This is where the term framework really becomes meaningful.

Company or Team Growth

Another type of scalability is more human. How does the application deal with new people? Can you grow from a startup to a tech giant? Can you go from a solo dev to a team? Can you onboard a new developer seamlessly?

Again this often comes down to technology choices. Learning a new commercial library or framework isn’t necessarily that difficult, even for a junior developer. What often makes things hard is learning your organisation’s idiosyncrasies. Your strange build tools. Your custom framework. Your undocumented dev tools setup or your “unique” testing approach.

Again, this is something monoliths help with. Being able to point to the documentation for Phoenix, .Net or Laravel and say “We do it like that” is a significant blessing.

New Markets and Products

Can your application be used by Australians? By Europeans? Are you able to serve the Japanese market? Would it be possible to do so? If so how would it work? This is another form of scalability, the ability to scale out to new markets and new users. A lot of that is planning ahead, and making sure you don’t paint yourself into any silly corners like assuming fixed languages, making RTL text impossible, rejecting “incorrect” addresses or names, etc. And again, these might be premature optimisations, but sometimes they’re also a preventative step against common misconceptions.

Premature Optimisation is the Root Of All Evil

When thinking about issues of scalability, you always need to factor in the next bottleneck, or the current problem. Not invent ones that you might or might not have in some distant point down the road. And the ones that you’re going to hit first are more about launching your software and getting some runs on the board. Being Facebook is not a problem you currently have. So don’t solve it.