Growing Your Monolith

Photo by Hulki Okan Tabak on Unsplash

So you have a great idea for a software project and have decide to make it into a website. You have a small team and you decide to keep it simple and just put the whole thing in one place. You build a small server-side rendered application talking directly to the database.

In a month or two, you develop rapidly, the code is all in one place, it can be debugged and read easily and your product is adored by your users. You hit the spot — congratulations! Now it’s time to grow…

Front and back end

Perhaps you hire a few more developers, you push out more features and your codebase is starting to grow. With more developers working on the same codebase, you find that more often than not two or sometimes more developers are working on the same code and merge conflicts become more regular.

You decide to split your codebase.

The most intuitive next step would be to split the codebase into a client application and an API. However, there is a tradeoff:

Advantages:

• Separation of concerns (the API is concerned with business logic and persistence while the client is concerned mostly with presentation and user interaction)
• Two repositories make conflicts less likely
• Intrinsic cognitive load reduced because developers can now focus on either front or the back end

Disadvantages:

• Entities, validation and some business rules are now likely duplicated in the client and API
• The client application now must keep track of API interfaces it communicates to
• You need to build and deploy the client and API separately

Photo by Asif Akbar from FreeImages

A common module

The codebase is still not too big and you keep on commiting and pushing code, now with a more flexible setup. There are now two teams, one taking care of the front end and the other taking care of the backend. The codebase is getting bigger, and the front and backend get more and more out of sync. This results in bugs and more (unnecessary) communication between team members (perhaps even some blaming), as well as the necessity to look over the implementation in the other codebase, and reflect the changes made.

Obviously you need some common language between front and the backend. So you decide to make a package that will hold logic that both the client and the API will consume. There is, unfortunately a hefty price to pay for this, too.

Advantages:

• A good chunk of the code is no longer duplicated
• Business logic can be kept safe and tested
• Changes made to the common package are now reflected in both client and API

Disadvantages:

• You have a third repository, likely maintained by both teams
• The client and API need to keep up to date with the latest changes on the common module
• The common module has different rules for writing and building code than both the front and back end
• You need to create, maintain and possibly pay for a repository for the common module packages
• Developers need to understand semantic versioning and adhere to it

Photo by Christina Papadopoullou from FreeImages

As you can see, it already becomes less clear whether or not you are gaining value from this change.

The services

By now your small company has grown to three or four development teams, and the codebase has grown accordingly. The software now has many features, many entities and business rules.

It has become obvious that there are separate portals on the front end, and that these portals communicate with specific endpoints more than others.

The knowledge of the whole business and how different parts of the software operate is becoming a burden too big for any developer to bear, so you decide to make the next leap.

Hopefully you have kept your code clean and your contexts bounded, and it is at least possible for you to make the transition — it’s time to split the back end into services, and at least logically split the front end into parts concerned with separate business domains. As you might have guessed, it’s not just going to be sunshine and roses. Let’s take a look at the tradeoffs:

Advantages:

• The API is split into several repositories containing similar business logic, reducing the cognitive load on the team
• There is once more less merge conflicts and less commits that any one developer is behind from the main branch
• There is less communication needed from different teams
• Ideally each team has a set of modules encapsulating business logic in entities and interfaces
• Each service has a separate, smaller database

Disadvantages:

• The code is harder to run and debug
• The service communication is harder to orchestrate in the infrastructure
• The services need to communicate with other services to maintain data consistency
• Authentication needs to be centralized
• The system as a whole is much harder to understand

So was it worth it? Well… maybe. Before we conclude. Let’s take another step.

The event bus

The inter-service dependence is becoming too big and the instability of different components of the system is causing the whole system to go down and incidentally, driving you mad.

To decouple the services and take care of all those nasty data inconsistency issues, as well as provide a more stable environment for your users, you decide to start using an event bus.

Advantages:

• Services no longer have to call other services
• Data is synced safely and quickly
• You know exactly how to communicate with other services — you emit events

Disadvantages:

• API developers now must learn how to use the chosen event bus
• Developers must learn to think in terms of events
• Services taking care of shared entities now must emit events and other services must consume them
• The application flow is extremely hard to understand in some cases
• Orchestration of events becomes a possible issue

Photo by Yannik Mika on Unsplash

When to take the step

I’m sure that complicating your decisions by presenting you with the tradeoffs you are going to make did not make your job easier. I will not even try to pretend to have the answer, but what I can do is suggest a rule of thumb:

Take the step when the cognitive load of the teams in the current state becomes greater than what you would have by taking the next step towards a more distributed system.

But how can you tell when that is? Listen to the people. Are they complaining? What are they complaining about? How often?

If you’ve heard from six out of ten developers that they hate it that they have to maintain the same thing in the front and back end, it’s more likely that extracting this logic into a module is going to yield a net benefit.

Still, if most of them need to learn to write unit tests, maybe reconsider adding more knowledge debt to the pool.

Which leads us nicely into the reason to write this piece in the first place.

Sustainability

Much too often we plan for big projects and rapid growth, almost never asking ourselves — will it be sustainable?

I would say that such grand plans rarely are. It’s nice if you’re successful and can afford this rapid growth, but throwing more people at the problem will only last you so long, until the people that built the thing and contain all of the siloed and undocumented knowledge start leaving for all kinds of reasons.

Achieving sustained growth is a bit more straightforward than measuring cognitive load. Here’s some of the things you might need to do:

• Build trust in your teams
• Follow proven industry best practices
• Control the intake and measure the flow of work
• Build loosely coupled components

When you decide to start on this journey of sustainability, don’t do it all at once. Introduce more good habits and practices, only when most of the people have incorporated the ones you already introduced.

Conclusion

More often than not people think that a new hot technology or architecture is the best thing since sliced bread, but the truth is — it’s always a trade off.

The real question is — do you need it?

The more I’m thinking about software and the people building it, the more I’m seeing the negatives that the complexity of these solutions bring.

I’m by no means saying go back to monoliths, but I am saying that you might want to reconsider a microservice-first approach if your team consists of three to four developers with limited work experience.

Weigh your options, and try to chose the strategy that your team can deliver on comfortably. Unless your team consists of seasoned Navy Seals, I will venture a guess that you would be better off starting from a monolith, learning to separate modules and split only when you have a good grasp of your system.