Where Container Images Are Stored: Introduction to Skopeo

It would be quite boring to learn about OCI image spec without some real practice with container images. To work with container images, you don't need a complete container manager like Docker. Container images are nothing but simple packages of content and there are many tools that can work with such packages.

One of those tools is Skopeo - it's a small CLI utility that is capable of copying images between different registries and to and from your local machine. Skopeo can also inspect images, including remote repositories - this is useful if you need some more metadata about particular image. We will use Skopeo to inspect an existing container image and see image spec components.

Skopeo is available in standard repositories on many Linux distributions. If you are using Fedora VM as was suggested in the start of the course, simply run yum install skopeo.

In the next chapter, we will learn what image spec is about and examine some images with the help of Skopeo.

Skopeo

To use Skopeo we first need to understand where the container images are actually stored. When you are using a full-featured container manager like Docker, you normally never think about where the images are. You would do docker pull alpine and then see the alpine image in the output of docker images command - but where is the image on your filesystem?

There are multiple locations where the image could be found, and the location depends on which tool you are using to work with your images. Higher level tools, like Docker, do not expose those low level details to you - for good reasons, because on most cases you don’t need to care about the location of images in your system. But we are here exactly to look under the hood, demistify Docker and get a full picture of how containers are working.

Skopeo has a concept of “stores” - each store represents certain type of image storage. The store could be a remote container registry, or it could be a different kind of locations on your filesystem. You can’t pull images with Skopeo - what you do instead is copying images from source store to the destination store.

Let’s start by doing a docker pull, but with Skopeo. To do so, we need to specify remote docker registry as a source, and local docker daemon as a destination.

skopeo copy docker://docker.io/alpine:latest docker-daemon:docker.io/alpine:latest

When you run docker images command, you will see alpine image in the list. Docker stores images in it’s own custom format, that pre-dates the OCI standards. When stored by Docker-daemon, images are located in /var/lib/docker/ and their content is split across multiple directories. The content of the image is stored inside the /var/lib/docker/overlay2 directory (unless you are not using overlay2), and the different metadata can be found in the /var/lib/docker/image directory.

To see a complete image as a single package we can use another Skopeo store type, caled docker-archive. Let’s copy the image from Docker Daemon to the Docker Archive - this would be identical to running docker save command, you are familair with that one:

skopeo copy docker-daemon:docker.io/alpine:latest docker-archive:docker.io-alpine-latest.tar.gz

Once again we need to specify the source storage type which is the Docker Daemon in this case and the destination storage type which is the Docker Archive.

Resulting archive can be unpacked with the tar command:

tar -xvf docker.io-alpine-latest.tar.gz

Let’s briefly examine the contents of this archive.

The main file is called manifest.json - it defines which layers the image has and where to find the configuration of this image.

Configuration file specifies many default values used to run the container - like the environment variables, command to run and if it needs to be attached to the standard input. Additionally, there is a history of changes of this image, as well as a pointer to the layer that needs to be used when this container is running. The layer itself, in turn, is another tar archive that containts the full filesystem.

This structure is somethig very specific to how Docker stores images. But because big parts of OCI standard originate in Docker, we can easily convert images from Docker format to OCI format - it’s just another skopeo command. In the next lesson, we will look at the OCI image-spec.