Matrix builds are a feature of GitHub Actions that allow you to run multiple jobs in parallel across different configurations, such as operating systems, language versions, and architectures. This allows you to test various configurations easily without redundant workflows.
In this blog, we aim to provide a comprehensive guide on implementing matrix builds in GitHub Actions. You’ll be able to optimize your workflows and improve testing coverage, leading to faster and more reliable software.
Matrix builds in GitHub Actions allow developers to run jobs multiple times with different configurations, streamlining the testing process across various environments. This is particularly useful for testing applications across various environments, ensuring that code functions correctly regardless of the operating system, programming language version, or other variables. By defining a matrix of parameters—like OS types and versions; GitHub Actions automatically creates and executes a job for each combination of these parameters. This eliminates the need for duplicate workflows, enhancing efficiency and coverage.
In many ways, matrix builds can be compared to a "for loop" in programming. Just as a for loop iterates over a set of values to execute a block of code multiple times, matrix builds iterate over defined configurations to run jobs concurrently.
For example, if you have a web application that needs to be tested on both Windows and macOS with different versions of Node.js, matrix builds will handle this efficiently. Instead of writing separate workflows for each configuration, you can define a single matrix that includes all necessary combinations. This not only saves time but also simplifies the workflow management process, allowing developers to focus on writing code rather than managing multiple test setups.
Let’s consider a real-world scenario where we want to test a web application across multiple operating systems and Node.js versions.
Imagine you have a web application that must be compatible with various environments. You want to ensure it works on:
1. Create a workflow file:
Start by creating a YAML file in your repository under the .github/workflows directory. This file will define your CI/CD pipeline.
├── .github
│ └── workflows
│ └── ci.yaml // Define your workflow here
├── src
│ └── app
├── tests
│ └── app.test.js
└── package.json
2. define the jobs section: In your workflow file, define the jobs section. Each job represents a task that GitHub Actions will execute.
3. Set up the strategy: You need to specify the strategy block within each job. This is where you will define how the jobs will run in parallel based on different configurations.
4. Define the matrix: Under the strategy block, use the matrix key to specify the parameters you want to test across different configurations. Each parameter can have multiple values, and GitHub Actions will create a job for each combination.
Here’s an example of what this looks like in a workflow file:
In this example:
jobs: This section defines a job named build.strategy: This block specifies how to run the job with different configurations.matrix: Here, we define two parameters: os (operating systems) and node-version. Each value specified in these arrays will create parallel jobs for every combination.This configuration will create 9 jobs:
blacksmith-4vcpu-ubuntu-2204 with Node.js 12.xblacksmith-4vcpu-ubuntu-2204 with Node.js 14.xblacksmith-4vcpu-ubuntu-2204 with Node.js 16.xwindows-latest with Node.js 12.xwindows-latest with Node.js 14.xwindows-latest with Node.js 16.xmacos-latest with Node.js 12.xmacos-latest with Node.js 14.xmacos-latest with Node.js 16.x
5. Push your changes:
After committing your changes and pushing them to GitHub, navigate to the "Actions" tab of your repository. You’ll see that GitHub Actions has automatically created jobs for each combination of operating system and Node version specified in your matrix.
6. Monitor job execution:
Each job will run in parallel, allowing you to quickly identify any compatibility issues across different environments. For instance, if there are problems with Node.js 14 on Windows but not on Ubuntu, you'll be able to identify this quickly.
In addition to testing across various language version (Node.js 12.x, Node.js 14.x, Node.js 16.x) and running builds on different operating systems (testing on windows, macOS, Linux.), matrix builds are can be applied to a range of scenarios related to testing. Below are some common use cases, each accompanied by practical examples: This revision emphasizes the versatility of matrix builds while transitioning into the section discussing specific use cases.
Sharding tests is a technique used to split a large test suite into smaller chunks that can be executed in parallel. This reduces the overall testing time and allows for a faster feedback loop when you create a change.
Example:
name: CI
on:
push:
branches:
- main
jobs:
test:
runs-on: blacksmith-4vcpu-ubuntu-2204
strategy:
matrix:
shard: [1, 2, 3]
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Run tests
run: npm test -- --shard=${{ matrix.shard }}Here, the tests are divided into three shards (1, 2, and 3). Each job runs a different shard of the test suite, allowing for parallel execution and faster completion.
When building Docker images, it’s often necessary to ensure compatibility across different architectures (e.g., x86 and ARM). Matrix builds allow you to define jobs that target multiple architectures efficiently.
Example:
name: Build Docker Images
on:
push:
branches:
- main
jobs:
build:
runs-on: ${{ matrix.runner }}
strategy:
matrix:
platform: [amd64, arm64]
include:
- platform: amd64
runner: blacksmith-2vcpu-ubuntu-2204
- platform: arm64
runner: blacksmith-2vcpu-ubuntu-2204-arm
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Login to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Build and push Docker image
uses: docker/build-push-action@v2
with:
context: .
file: Dockerfile
push: true
tags: my-app:${{ matrix.platform }}
platforms: ${{ matrix.platform }}The use of the exclude keyword allows you to skip unnecessary builds, while the include keyword can be used to add specific configurations without duplicating your workflow. This approach not only simplifies your setup but also enhances performance and ensures compatibility across systems. In the example above, the workflow builds Docker images for both linux/amd64 and linux/arm64 architectures. By using matrix builds, you can run the ARM build on blacksmith-2vcpu-ubuntu-2204-arm ARM runner and the AMD64 build on a blacksmith-2vcpu-ubuntu-2204 runner, eliminating the need for QEMU emulation.
By using matrix builds, you can ensure that your images are compatible with multiple systems without duplicating the workflow.
When developing applications that interact with databases, ensuring compatibility with various database versions is essential. Matrix builds allow you to test your application against multiple database setups, ensuring that queries and functionalities work as expected across different environments.
Example:
name: CI
on:
push:
branches:
- main
jobs:
test:
runs-on: blacksmith-4vcpu-ubuntu-2204
strategy:
matrix:
db: [mysql:5.7, mysql:8.0, postgres:12, postgres:14]
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Set up Database
run: |
docker run --name db -e MYSQL_ROOT_PASSWORD=root -d ${{ matrix.db }}
- name: Run tests
run: npm testIn this example, the workflow tests the application against multiple versions of MySQL and PostgreSQL databases to ensure compatibility.
Matrix builds can be utilized to run performance tests under different configurations or load conditions. This is particularly useful for identifying how your application behaves under varying loads or resource limits.
Example:
name: Performance Tests
on:
push:
branches:
- main
jobs:
performance:
runs-on: blacksmith-4vcpu-ubuntu-2204
strategy:
matrix:
load: [100, 500, 1000] # Different load levels
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Run performance tests
run: |
echo "Running performance tests with load level ${{ matrix.load }}"
# Command to run performance tests goes hereThis setup allows you to evaluate how well your application performs under different user loads.
For applications that expose APIs, it's crucial to ensure that all endpoints function correctly across different versions of the API. Matrix builds can facilitate testing against multiple API versions simultaneously.
name: API Tests
on:
push:
branches:
- main
jobs:
api-test:
runs-on: blacksmith-4vcpu-ubuntu-2204
strategy:
matrix:
api-version: [v1, v2, v3]
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Run API tests
run: |
echo "Testing API version ${{ matrix.api-version }}"
# Command to run API tests goes hereThis example ensures that all specified API versions are tested for functionality and compatibility.
For web applications, it’s essential to verify that the application behaves consistently across different web browsers. Matrix builds can help automate this process by running tests in various browser environments.
Example:
name: Cross-Browser Testing
on:
push:
branches:
- main
jobs:
browser-test:
runs-on: blacksmith-4vcpu-ubuntu-2204
strategy:
matrix:
browser: [chrome, firefox, safari]
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Run browser tests
run: |
echo "Running tests on ${{ matrix.browser }}"
# Command to run browser-specific tests goes hereThis setup allows you to ensure that your web application functions correctly across popular browsers.
One of the most compelling reasons to use matrix builds is the substantial efficiency gains they provide. By running multiple jobs in parallel, you can significantly reduce the total time required for testing. Instead of executing tests sequentially, which can be time-consuming, matrix builds allow you to run jobs for different configurations simultaneously.
In the example we saw in the section above when you need to test your application across three operating systems and three versions of a programming language, a sequential approach could take a considerable amount of time. However, with matrix builds, all combinations can be executed at once. This parallel execution not only speeds up the testing process but also enables faster feedback on code changes, allowing developers to address issues more quickly.
Matrix builds also enhance test coverage by ensuring that your application is validated across various environments and configurations and can catch regressions. This is crucial for identifying compatibility issues that may arise in different setups.
By covering a wider range of scenarios, you increase confidence that your application will function correctly in real-world conditions. This thorough testing helps catch bugs early in the development cycle, ultimately leading to a more stable and reliable product.
Another significant benefit of using matrix builds is the simplification they bring to workflow management. By defining a matrix strategy, you can eliminate redundancy in your workflow files. Instead of creating separate jobs for each configuration, you can define them all in one place using a matrix.
This not only makes your workflow files cleaner and easier to read but also simplifies maintenance. If you need to add or remove configurations, you can do so by updating just one section of your workflow file rather than modifying multiple job definitions. This streamlined approach reduces the likelihood of errors and makes it easier for teams to collaborate on CI/CD processes.
Matrix builds in GitHub Actions come with several advanced features that enhance their functionality and flexibility. Here’s an in-depth look at these features and how they can be utilized effectively.
Dynamic matrix generation allows you to create matrices based on the outputs of previous jobs or specific conditions. This feature is particularly useful for complex workflows where the configurations needed for testing may change based on earlier job results.
To implement dynamic matrix generation, you can use the set-output command to define a matrix dynamically. For example, you might have a build job that determines which versions of a dependency are compatible, and then pass those versions to a subsequent test job.
Here’s a simplified example:
jobs:
build:
runs-on: blacksmith-4vcpu-ubuntu-2204
steps:
- name: Determine compatible versions
id: version-check
run: |
echo "::set-output name=matrix::$(echo '[{"version": "1.0"}, {"version": "2.0"}]')"
test:
needs: build
runs-on: ubuntu-latest
strategy:
matrix:
version: ${{ fromJson(needs.build.outputs.matrix) }}
steps:
- name: Run tests
run: npm test --version ${{ matrix.version }}In this example, the build job determines which versions are compatible and sets them as output. The test job then uses that output to create a matrix of tests to run, allowing for more flexible and responsive workflows.
When working with matrix builds, you may encounter situations where certain combinations of configurations are not necessary or even incompatible. GitHub Actions provides the
exclude and include keywords to refine your job executions.
include keyword. This is useful when you want to test an experimental feature or a special case without duplicating your entire workflow:This approach enhances your control over which jobs are executed and ensures that your testing is both comprehensive and relevant.
The fail-fast strategy is a built-in behavior in GitHub Actions that automatically cancels all running jobs in a matrix if one job fails. This can save time and resources by preventing unnecessary executions once a failure has been identified, ultimately helping to reduce CI costs. However, there are scenarios where you might want to disable this behavior to gather more data from all jobs. By halting further job executions when a failure occurs, teams can avoid wasting compute resources on jobs that are likely to fail due to an identified issue, leading to more efficient resource management.
To disable fail-fast behavior, you can set it explicitly in your workflow:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
node-version: [12.x, 14.x]In this configuration, even if one job fails (e.g., Node.js 12.x on Windows), all other jobs will continue to run. This is particularly useful for debugging or when you need results from all configurations for analysis.
Managing resource consumption during job execution is crucial, especially when using self-hosted runners or when your CI/CD environment has limited resources. The max-parallel key allows you to control how many jobs run concurrently within a matrix.
For example, if you want to limit the number of concurrent jobs to two at any given time, you can configure it like this:
strategy:
max-parallel: 2
matrix:
os: [ubuntu-latest, windows-latest]
node-version: [12.x, 14.x]With this setup, even if there are multiple combinations in the matrix (e.g., four total jobs), only two will run simultaneously. This helps manage the load on your CI/CD infrastructure and can lead to more stable builds by avoiding resource contention. However, if you're using Blacksmith's runners (blacksmith-2vcpu-ubuntu-2204 ), you won’t need to worry about these concurrency limits. Blacksmith has no concurrency limits and can scale with your demand, whether it’s hundreds of jobs or thousands of vCPUs.
Optimizing your matrix builds in GitHub Actions can lead to significant improvements in build times, resource utilization, and overall workflow efficiency. Here are some best practices to consider:
One of the key advantages of using matrix builds is the ability to run multiple jobs in parallel. However, to maximize this benefit, it's essential to utilize resources efficiently. Here are some tips for caching dependencies and speeding up build times:
node_modules directory:- name: Cache node_modules
uses: useblacksmith/cache@v5
with:
path: node_modules
key: ${{ runner.os }}-node-${{ hashFiles('**/package-lock.json') }}This action checks if the cache exists based on the specified key. If it does, it restores the cached dependencies, significantly reducing installation time.
npm ci instead of npm install for faster installations when working with Node.js projects. The ci command installs dependencies based on the package-lock.json, which is generally faster and more reliable in CI environments.Implementing conditional logic within your matrix builds can help streamline your workflows by skipping unnecessary steps based on specific conditions or matrix values. This not only saves time but also reduces resource consumption.
if conditions: You can use the if keyword to control whether a step runs based on the current matrix configuration. For example, if you want to skip deployment steps for certain Node.js versions, you can do so like thisBy implementing these conditional steps, you can tailor your workflows to be more efficient and relevant to the specific configurations being tested.
Handling artifacts efficiently within matrix jobs is crucial for maintaining a clean workflow and ensuring that necessary files are available for subsequent jobs or deployments. Here are some best practices:
In this example, only the contents of the ./dist/ directory are uploaded as artifacts, keeping things streamlined.
actions/download-artifact action:This allows subsequent jobs to access necessary files without duplicating work.In conclusion, leveraging matrix builds in GitHub Actions significantly enhances testing efficiency and overall workflow management. Being able to run multiple jobs in parallel across various configurations not only reduces the time required for testing but also simplifies workflow files by eliminating redundancy. In addition, with features like dynamic matrix generation and the ability to include or exclude specific configurations, teams can adapt their CI/CD processes to meet evolving needs.
