Mono repository is a popular approach where some libraries or other fairly independent projects are colocated in one repository. One of the benefits you get is simpler control over changes that should be synchronized across all involved packages. As a downside, you get less independence of each package.
The vast majority of everyday monorepo management tasks boil down to running package scripts in a certain order and under certain conditions, both locally and on CI. This article explains one of many possible approaches. This approach worked fairly well in some of RingCentral’s monorepo projects. The size of projects varies: from less than a hundred files and a handful of devs up to thousands of files and 50+ developers.
Scripts should eliminate the possibility of error in critical tasks like test coverage collection and publishing
Full control mode when needed: ability to run scripts granularly or in a quicker fashion
Overall, any interaction with a repository, both locally and on the CI, can be presented as scenarios that can be split into some granular phases (or stages in Gitlab terminology). Scenarios may skip phases or have some phases substituted with other different phases. Scenarios usually have certain goals and expectations — here are the most common ones:
A developer checks out the repo and runs— the goal is to have a fully functional and ready-to-use dev setup at this point, the expectation is that right after this command the developer may run other commands with no additional actions related to getting things ready to work
A developer runs thescript — the goal is to run in all packages, the expectation is that it just starts working, with no need to pre-build anything
CI determines that a tag has been pushed, checks out the repo, runs linter, runs build scripts, runs tests, and collects coverage, and if all is good — publishes to NPM. The goal is the delivery of high-quality code. The expectation is that if any phase breaks it stops the process and there is zero possibility of any unwanted publishes of incomplete/broken code
Let’s dive deeper into those phases.
In this phase, CI or the developer sets up the environment to run other scripts.
CI should runso that we can control when and how to do bootstrapping in the CI environment, for example, the task does not require packages to be bootstrapped, but the task does (more about this in the next section)
Developers should use regular, which must run full bootstrap
This is the cheapest check that makes sure the codebase is in good condition.
CI should runright after as the first cheap check
CI must stop all further expensive checks if the cheapcheck failed
In this phase, dependencies of underlying packages are installed — this is a post install on the dev machine or a post lint phase in the CI environment, but it depends on the setup.
CI may not require demos to be built/tested and thus neither to be bootstrapped unless you use demos for e2e tests. In this case, we should bootstrap only the truly used packages by scoping thescript: (we usually have libraries in this scope). This is a nice optimization that saves CI time.
A phase needed to obtain an artifact, something that later will be published. Also, it may be a prerequisite for tests.
Thescript on package level should run the script beforehand to make sure no previous artifacts exist on subsequent builds
Thecan be used to build all libs that will be used in demos
A phase that does the majority of code quality verification and makes sure things work as expected, not just written as expected (linter phase).
Sometimes code has to be built before running tests. If there are multiple libs in monorepo which depend on each other it is safer to run tests once all dependencies are built instead of using magic to run tests using only sources.
CI runs tests by calling the with e2e tests and coverage collectionscript, which is a maximum set of all tests
Locally devs may run which does not collect coverage or does not run e2e tests
CI should also runscript to upload coverages after
Optionally there could also be ascript which will run quick tests in watch mode
This is a phase that occurs on dev machines when developers commit and push the code, some minor quick checks should happen on this phase.
Obviously, we don’t want to make CI run potentially invalid code so we can useand (or )scripts in pre-commit hooks
This delivers artifacts to some package management system that distributes them to end users. Code must be built and tested before publishing, which is CI responsibility.
Publish scripts actually do the publishing (usually on CI)
Canary means a pre-release (something you can run nightly)
Release means a regular versioned release
CI may addflag to scripts: it’s better to add them here in order to prevent unwanted local publishes
Alternatively, on always publish from CI only as it is the only way to ensure all proper checks., CI can skip regular flow ( , , ) and only run in packages, which should do , , to make sure that if is run locally from dev machine it still goes through all phases, but we recommend to
Developer-specific phase, day to day activities like website or library/demo/storybook development, on this phase codebase is constantly watched for changes and re-built once changes occur.
Assume the following setup:
simply runs in all packages which starts Webpack and Babel watchers
Developer-specific phase before builds or when switching branches.
Root’s artifacts), then remove node_modules in packages, then remove node_modules in the root, this will bring repository to ground zeroshould first run scripts in packages (e.g. clean
Cleaning is useful when devs switch branches, then commandmakes sure they have working setup after branch switch
Main package scripts
We prefer to run tests one by one in topological order (maintained by Lerna) to see the nice structured output. Technicallycan be replaced with which disregards topology and runs everything in parallel.
You can publish all packages together usingflag for simplicity.
Scriptsand should be part of pre-commit hook.
GitLab CI config
This setup assumes you have(and others) in GitLab ENV variables. Also note the flags for publish scripts.
Travis CI config
This setup also assumes you haveand in Travis ENV variables. Also, note the flags for scripts. Travis by default will do which will do , so there’s some room for optimization like we did in GitLab example.
Since we mention a lot of packages it does make sense to tell what they are doing.
This approach with minor variations (mostly alternate phases) was successfully implemented and tested in different projects and proven to be consistent and complete. I hope it will help you to build, test and interact with your projects more efficiently.
This whole thing definitely requires some library to make things easier :)