A Guide to Disk Space Management with Docker: How to Clear Your Cache

If you've been using Docker extensively in your local development workflow, you've likely encountered a common challenge: rapidly diminishing disk space. One common issue you might encounter is the no space left on device error when pulling images or running containers. While Docker is an incredibly powerful tool for development and deployment, it can be quite the disk space hog if left unchecked due to the accumulation of Docker artifacts over time. Regular Docker cleanup is essential to prevent disk space issues and maintain system performance. In this post, we'll explore the intricacies of Docker disk usage and how you can optimize your local workflow to keep your machine running smoothly.

Understanding Docker's disk usage

Before we dive into solutions, it's crucial to understand what's eating up all that precious disk space. Docker's disk usage primarily comes from a few sources:

1. Images: Docker images are the blueprints for containers. They can be quite large, especially if you're working with multiple projects or versions.
2. Containers: Running and stopped containers take up space, as they maintain their state on disk.
3. Volumes: Docker volumes are used for persistent data storage that can be mounted into containers. These can accumulate over time.
4. Docker network: Managing Docker networks is essential for maintaining system efficiency. Unused Docker networks contribute to network bridges and routing table entries, which can affect performance even though they don't take up disk space. It's important to list, remove, and prune these networks regularly to keep the system running smoothly.
5. Build cache: Docker's build cache is a double-edged sword. It speeds up builds by reusing layers from previous builds when Dockerfile instructions haven't changed. This is especially useful for time-consuming steps like dependency installation. However, over time, particularly with frequent builds and Dockerfile changes, the cache can accumulate significantly. Cached layers from various projects and image versions can quietly occupy gigabytes of storage, creating a trade-off between build speed and disk usage.

We can get a breakdown of how much disk space is used by Docker by running:

> docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          188       37        51.98GB   39.13GB (75%)
Containers      60        6         7.026GB   7.026GB (99%)
Local Volumes   21        17        2.985GB   115.1MB (3%)
Build Cache     955       0         2.68GB    2.68GB

Let's break down the main offenders in more detail:

1. Unused Images: Old or unused images you've pulled or built can pile up over time.
2. Dangling Images: These are layers not associated with any tagged images, often leftovers from building new versions of an image.
3. Stopped Containers: Containers you're no longer using but haven't removed.
4. Unused Volumes: Volumes that are no longer associated with any container but still exist on disk.

Strategies for reclaiming risk space

Now that we've identified the space-hogging culprits, let's explore effective strategies to reclaim your disk space:

1. Remove unused images

docker image prune

This command removes dangling images, which are images that are not associated with any containers and don’t have tags. We can add the --all flag to remove all unused images, not just the dangling ones:

> docker image prune --all
WARNING! This will remove all images without at least one container associated to them.
Are you sure you want to continue? [y/N] y
...
deleted: sha256:aed56c19ab37760be1be44464cac4702014d5cdb2e9851d3def27aefc707246f
deleted: sha256:6f5537365f3cc25e226e339d9f17804f322e69271f25c879234e871271af8c43
deleted: sha256:72c89f3b65d07e8c9a6828aa3eef45179a5fafdc7fa36f8522823762d1fc9ef4

Total reclaimed space: 12.36GB

‍
2. Clean up stopped containers

docker container prune

This removes all stopped containers. Stopped containers persist their state on disk, so removing them should free up this space. In order to move all containers, we must stop all of them first and then run the same prune command.

docker stop $(docker ps -q) && docker container prune

‍
3. Remove unused volumes

docker volume prune

This command removes all unused volumes.‍

4. Clear the build cache

> docker builder prune
WARNING! This will remove all dangling build cache. Are you sure you want to continue? [y/N] y
ID                                              RECLAIMABLE     SIZE            LAST ACCESSED
7lizg81c20p2zmcoomyu6rt9t                       true            216MB           4 weeks ago
...
kfuzv2e0w1jpuc83u61f26ri4                       true    0B              Less than a second ago
tkrm4heav645tgg03a9wg2q7g                       true    0B              Less than a second ago
Total:  19.24GB

This clears the build cache, which can significantly reduce disk usage.

‍
5. Use multi-stage builds: While this doesn't directly free up space, it helps create smaller images, reducing overall disk usage over time.

‍
6. Leverage .dockerignore: Prevent unnecessary files from being copied into your image by using a .dockerignore file. For a Node.js project, an example of something you typically want in your .dockerignore is the node_modules directory, or the log directory in a Ruby on Rails project.

Best practices for ongoing management

To keep your Docker environment from getting bloated over time, some other best practices can help:

1. Use specific tags: Instead of using the latest tag, use specific version tags for your images. This makes it easier to identify and remove outdated images.
2. Clean as you go: Make it a habit to remove containers and images you no longer need immediately after use. Consider running some of the aforementioned commands on your system as a regular cron job.
3. Use Docker desktop: If you're on Mac or Windows, Docker Desktop provides a user-friendly interface for managing images, containers, and volumes. At Blacksmith, we’re big fans of OrbStack (a more lightweight and performant Docker Desktop alternative).
4. Monitor your disk space: Regularly check your disk space usage. Tools like docker system df can give you an overview of Docker's disk usage.
5. Optimize your Dockerfiles: Write efficient Dockerfiles that produce smaller images. Combine RUN commands and clean up within the same layer to reduce image size.

By implementing these strategies and best practices, you can maintain an efficient Docker environment that balances functionality with prudent disk space management. Regular maintenance and thoughtful Docker usage will help you make the most of your resources and keep your development workflow smooth and efficient.