Building a GitLab CI/CD pipeline for a monorepo the easy way
Table of Contents
1. The ideal case: CI/CD in a monorepo
In a monorepo with multiple applications, you ideally want to have separate pipeline configurations for each application to run only when changes occur in that specific application's code. This can be achieved by including specific YAML files based on directory changes in the project-level pipeline configuration.
2. The legacy approach
Prior to GitLab 16.4, the approach taken was to include all application-specific pipeline files in the project's main pipeline file, and handle the logic within each application's specific file. Jobs were extended from hidden jobs named .java-common or .python-common, triggered by changes to the relevant directories.
However, this approach led to redundant code, potential for human errors, and limited flexibility in job definitions due to key collisions in extended jobs.
3. New approach with GitLab 16.4
With the introduction of GitLab 16.4, a new approach was made possible by using the with keyword in the project pipeline configuration. By defining monorepo rules using this keyword, you can now include specific jobs only when changes occur in the associated directories, offering more flexibility and reducing code duplication for engineers working with monorepos.
This new approach simplifies the pipeline setup by allowing developers to focus on application-specific jobs without the need for extending hidden jobs and dealing with key collisions.
By following this best practice, using GitLab CI with monorepos becomes more efficient and streamlined.