Switching to Monorepo Architecture

CloudSEK - Monorepo Architecture
  • Author: Rohan Luthra
  • Editor: Benila Susan Jacob

An Obstacle

We started a new project a year ago in CloudSEK, and changed our architecture from monolithic and service-oriented to proper Microservice architecture. With the microservices, each service had its own repository and CI CD pipeline connected to our Kubernetes service mesh.

Everything was going smoothly until the number of services started to increase, and then it became tough to manage the code base. There were as many MRs (Merged Requests) as there were repositories, multiple CICDs to manage, multiple access issues, and a lot of code duplication, which led to increased manual effort. Even though there were common npm packages, updating this package in all repositories was an engineer’s nightmare. This eventually became a bottleneck and started affecting the team’s velocity.

The Solution

Although it should have been anticipated from our microservice approach, one learns from their mistakes. The issue is only with maintainability, hence a monorepo would solve the problem.

Things in the market

Since the tech stack is TS, both frontend and backend, there were a couple of frameworks to look at. The following are the frameworks on which we did a POC before moving forward:

  • Turborepo
  • Lerna
  • Nx

We chose Nx, as it had out-of-the-box capabilities supporting ReactJS and NestJs. Plus it had a couple of advantages over other solutions, specifically with the ‘affected’ code commands.

Setting up NX

With Nx, it’s super simple, just install nx-cli globally and start adding apps.

$ npx create-nx-workspace –pm pnpm –interactive false –defaultBase main –name my-workspace

Since we are using NestJs, let’s create 2 apps and 2 libs

# install NestJs plugin
$ pnpm install -D @nrwl/nest# apps
$ pnx g @nrwl/nest:app test1
$ pnx g @nrwl/nest:app test1# libs
$ pnx g @nrwl/nest:lib lib1
$ pnx g @nrwl/nest:lib lib2# start the apps
$ pnpm run start:all

The folder structure should look like this:

Folder structure
Folder structure

 

Advantages of using Nx

Since maintaining a monorepo is a lot of hassle because of too much code, Nx provides a way to get only changes: Affected

We demonstrate with an example how it was used to our benefit. In the setup, we had created 2 apps and 2 libs and just imported the libraries in these apps. Lets see the overall graph:

Test 1
Test 1

 

Test 2
Test 2

 

$ pnx graph
Now if changes are made to only lib2:
Now if changes are made to only lib2:

 

$ pnx affected:graph –base=main
As expected it only showed changes to test1 app. Similarly, if changes are made to lib2:
As expected it only showed changes to test1 app. Similarly, if changes are made to lib2:

 

$ pnx affected:graph –base=main

Commands with Affected

The changes are reflected in both the apps. Using the affected we need not run lints, tests, builds and deployments for the whole codebase, but just for the affected parts.

Cherry on top

As this was integrated in our CI CD, we were able to lint, test, build and deploy our services. All our pipelines ran only for affected code changes only.

 

But the best finding was that the docker container size was almost half of our previous build. Reason is that Nx build and NestJs builds are very different and Nx makes a smaller build.

Nx Build –

NestJs build –

Conclusion

Nobody can know everything, we all are just learning from our mistakes. And Nx is one of the best monorepo frameworks out there for TS/JS. In this blog, I’ve shown how Nx has helped us in creating a better structure to maintain our code and enhance our pipeline. Some major benefits we got shifting to the Nx –

  • Easier to maintain our code – single repository
  • Scaleable CICD setups
  • Faster CI/CD pipelines – as jobs run only happens for affected code only
  • Smaller docker builds
  • Simple integration with NestJs and ReactJs, easy migration

Other important notes with NX

  • Moving to pnpm ( diff ) ( benchmark ), as pnpm is 3 times faster than npm/yarn
  • Can do e2e testing with Jest and cypress
  • We can also publish our libs or apps (optional)
Rohan Luthra
Rohan is a Full Stack Software Engineer and a leader with the passion to write the perfect code. He is inquisitive about everything and loves to try different cuisines.
×
Rohan Luthra
Rohan is a Full Stack Software Engineer and a leader with the passion to write the perfect code. He is inquisitive about everything and loves to try different cuisines.
Latest Posts
  • CloudSEK - Monorepo Architecture