Best Practices
On this page
- Use Public Schema for Validation and Auto-Completion of Component Descriptors
- Separate Build and Publish Processes
- Separation Between Build and Publish
- Building Multi-Architecture Images
- Using Makefiles
- Pipeline Integration
- Static and Dynamic Variable Substitution
- Debugging: Explain the Blobs Directory
- Self-Contained Transport Archives
- CICD Integration
This chapter contains guidelines for common scenarios how to work with the Open Component Model, focusing on using CI/CD, build and publishing processes.
- Use Public Schema for Validation and Auto-Completion of Component Descriptors
- Separate Build and Publish Processes
- Separation Between Build and Publish
- Building Multi-Architecture Images
- Using Makefiles
- Pipeline Integration
- Static and Dynamic Variable Substitution
- Debugging: Explain the Blobs Directory
- Self-Contained Transport Archives
- CICD Integration
Use Public Schema for Validation and Auto-Completion of Component Descriptors
The Open Component Model (OCM) provides a public schema to validate and offer auto-completion of component constructor files used to create component descriptors. This schema is available at https://ocm.software/schemas/configuration-schema.yaml.
To use this schema in your IDE, you can add the following line to your component constructor file:
# yaml-language-server: $schema=https://ocm.software/schemas/configuration-schema.yaml
This line tells the YAML language server to use the OCM schema for validation and auto-completion.
Separate Build and Publish Processes
Traditional automated builds often have unrestricted internet access, which can lead to several challenges in enterprise environments:
- Limited control over downloaded artifacts
- Potential unavailability of required resources
- Security risks associated with write permissions to external repositories
Best practice: Implement a two-step process: a) Build: Create artifacts in a controlled environment, using local mirrors when possible. b) Publish: Use a separate, secured process to distribute build results.
OCM supports this approach through filesystem-based OCM repositories, allowing you to generate Common Transport Format (CTF) archives for component versions. These archives can then be securely processed and distributed.
Separation Between Build and Publish
Typical automated builds have access to the complete internet ecosystem. This involves
downloading of content required for a build (e.g., go mod tidy
), but also the upload
of build results to repositories (e.g., OCI image registries).
For builds in enterprise environments this can lead to several challenges:
- Limited control over downloaded artifacts
- Potential unavailability of required resources
- Security risks associated with write permissions to external repositories
The first problem might be acceptable, because the build results may be analyzed by scanners later to figure out what has been packaged. Triaging the results could be done in an asynchronous step later.
The second problem could be solved by mirroring the required artifacts and instrument the build to use the artifacts from the local mirror. Such a mirror would also offer a solution for the first problem and act as target for various scanning tools.
The third problem might pose severe security risks, because the build procedure as well as the downloaded artifacts may be used to catch registry credentials or at least corrupt the content of those repositories.
This could be avoided by establishing a contract between the build procedure of a component/project and the build system, providing the build result as a local file or file-structure. This is then taken by the build system to push content wherever it should be pushed to. This way the execution of the build procedure does not need write permissions to any repository, because it never pushes build results.
The Open Component Model supports such processes by supporting filesystem based OCM repositories, which are able to host any type of content, regardless of its technology. The task of the build then is to provide such a CTF archive for the OCM component versions generated by the build. This archive can then be used by the build-system to do whatever is required to make the content accessible by others.
The composition of such archives is described in the Getting Started section.
To secure further processes, a certified build-system could even sign the content with its build system certificate to enable followup-processes to verify that involved component versions are provided by accepted and well-known processes.
Building Multi-Architecture Images
Note: This section provides information only on on building multi-arch images. Referencing a multi-arch image does not differ from images for just one platform, see the
ocm add resources
command for the OCM CLI
At the time of writing this guide Docker is not able to build multi-architecture (multi-arch / multi-platform)
images natively. Instead, the buildx
plugin is used. However, this implies building and pushing
images in one step to a remote container registry as the local Docker image store does not
support multi-arch images (for additional information, see the Multi-arch build and images, the simple way blogpost)
The OCM CLI has some built-in support for dealing with multi-arch images during the
component version composition (ocm add resources
).
This allows building all artifacts locally and push them in a separate step to a container registry. This
is done by building single-arch images in a first step (still using buildx
for cross-platform
building). In a second step all images are bundled into a multi-arch image, which is stored as
local artifact in a component (CA) or common transport (CTF)
archive. This archive can be processed as usual (e.g., for signing or transfer to other locations).
When pushed to an image registry, multi-arch images are generated with a multi-arch-image manifest.
The following steps illustrate this procedure. For a simple project with a Go binary and a helm-chart assume the following file structure:
tree .
.
├── Dockerfile
├── go.mod
├── helmchart
│ ├── Chart.yaml
│ ├── templates
│ │ ├── ...
│ └── values.yaml
└── main.go
The Dockerfile has the following content:
FROM golang:1.19 AS builder
WORKDIR /app
COPY go.mod ./
COPY main.go ./
# RUN go mod download
RUN go build -o /helloserver main.go
# Create a new release build stage
FROM gcr.io/distroless/base-debian10
# Set the working directory to the root directory path
WORKDIR /
# Copy over the binary built from the previous stage
COPY --from=builder /helloserver /helloserver
ENTRYPOINT ["/helloserver"]
Now we want to build images for two platforms using Docker and buildx. Note the --load
option for
buildx
to store the image in the local Docker store. Note the architecture suffix in the tag to be
able to distinguish the images for the different platforms. Also note that the tag has a different
syntax than the --platform
argument for buildx
as slashes are not allowed in tags.
$ TAG_PREFIX=eu.gcr.io/my-project/acme # path to you OCI registry
$ PLATFORM=linux/amd64
$ VERSION=0.1.0
$ docker buildx build --load -t ${TAG_PREFIX}/simpleserver:0.1.0-linux-amd64 --platform linux/amd64 .
[+] Building 54.4s (14/14) FINISHED
=> [internal] load build definition from dockerfile 0.0s
=> => transferring dockerfile: 660B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for gcr.io/distroless/base-debian10:latest 1.6s
=> [internal] load metadata for docker.io/library/golang:1.19 1.2s
=> [builder 1/5] FROM docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e851425 49.2s
=> => resolve docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e8514253 0.0s
=> => sha256:14a70245b07c7f5056bdd90a3d93e37417ec26542def5a37ac8f19e437562533 156B / 156B 0.2s
=> => sha256:a2b47720d601b6c6c6e7763b4851e25475118d80a76be466ef3aa388abf2defd 148.91MB / 148.91MB 46.3s
=> => sha256:52908dc1c386fab0271a2b84b6ef4d96205a98a8c8801169554767172e45d8c7 85.97MB / 85.97MB 42.9s
=> => sha256:195ea6a58ca87a18477965a6e6a8623112bde82c5b568a29c56ce4581b6e6695 54.59MB / 54.59MB 33.8s
=> => sha256:c85a0be79bfba309d1f05dc40b447aa82b604593531fed1e7e12e4bef63483a5 10.88MB / 10.88MB 3.4s
=> => sha256:e4e46864aba2e62ba7c75965e4aa33ec856ee1b1074dda6b478101c577b63abd 5.16MB / 5.16MB 1.5s
=> => sha256:a8ca11554fce00d9177da2d76307bdc06df7faeb84529755c648ac4886192ed1 55.04MB / 55.04MB 19.3s
=> => extracting sha256:a8ca11554fce00d9177da2d76307bdc06df7faeb84529755c648ac4886192ed1 1.1s
=> => extracting sha256:e4e46864aba2e62ba7c75965e4aa33ec856ee1b1074dda6b478101c577b63abd 0.1s
=> => extracting sha256:c85a0be79bfba309d1f05dc40b447aa82b604593531fed1e7e12e4bef63483a5 0.1s
=> => extracting sha256:195ea6a58ca87a18477965a6e6a8623112bde82c5b568a29c56ce4581b6e6695 1.1s
=> => extracting sha256:52908dc1c386fab0271a2b84b6ef4d96205a98a8c8801169554767172e45d8c7 1.5s
=> => extracting sha256:a2b47720d601b6c6c6e7763b4851e25475118d80a76be466ef3aa388abf2defd 2.8s
=> => extracting sha256:14a70245b07c7f5056bdd90a3d93e37417ec26542def5a37ac8f19e437562533 0.0s
=> [stage-1 1/3] FROM gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd 30.7s
=> => resolve gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd319 0.0s
=> => sha256:f291067d32d8d06c3b996ba726b9aa93a71f6f573098880e05d16660cfc44491 8.12MB / 8.12MB 30.6s
=> => sha256:2445dbf7678f5ec17f5654ac2b7ad14d7b1ea3af638423fc68f5b38721f25fa4 657.02kB / 657.02kB 1.3s
=> => extracting sha256:2445dbf7678f5ec17f5654ac2b7ad14d7b1ea3af638423fc68f5b38721f25fa4 0.1s
=> => extracting sha256:f291067d32d8d06c3b996ba726b9aa93a71f6f573098880e05d16660cfc44491 0.1s
=> [internal] load build context 0.1s
=> => transferring context: 575B 0.0s
=> [builder 2/5] WORKDIR /app 0.1s
=> [builder 3/5] COPY go.mod ./ 0.0s
=> [builder 4/5] COPY main.go ./ 0.0s
=> [builder 5/5] RUN go build -o /helloserver main.go 2.4s
=> [stage-1 2/3] COPY --from=builder /helloserver /helloserver 0.0s
=> exporting to oci image format 0.8s
=> => exporting layers 0.2s
=> => exporting manifest sha256:04d69fc3245757d327d96b1a83b7a64543d970953c61d1014ae6980ed8b3ba2a 0.0s
=> => exporting config sha256:08641d64f612661a711587b07cfeeb6d2804b97998cfad85864a392c1aabcd06 0.0s
=> => sending tarball 0.6s
=> importing to docker
Repeat this command for the second platform:
$ docker buildx build --load -t ${TAG_PREFIX}/simpleserver:0.1.0-linux-arm64 --platform linux/arm64 .
docker buildx build --load -t ${TAG_PREFIX}/simpleserver:0.1.0-linux-arm64 --platform linux/arm64 .
[+] Building 40.1s (14/14) FINISHED
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load build definition from dockerfile 0.0s
=> => transferring dockerfile: 660B 0.0s
=> [internal] load metadata for gcr.io/distroless/base-debian10:latest 1.0s
=> [internal] load metadata for docker.io/library/golang:1.19 1.1s
=> [builder 1/5] FROM docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e851425 37.7s
=> => resolve docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e8514253 0.0s
=> => sha256:cd807e8b483974845eabbdbbaa4bb3a66f74facd8c061e01e923e9f1da608271 157B / 157B 0.2s
=> => sha256:fecd6ba4b3f93b6c90f4058b512f1b0a44223ccb3244f0049b16fe2c1b41cf45 115.13MB / 115.13MB 35.6s
=> => sha256:4fb255e3f99867ec7a2286dfbbef990491cde0a5d226d92be30bad4f9e917fa4 81.37MB / 81.37MB 31.8s
=> => sha256:426e8acfed2a5373bd99b22b5a968d55a148e14bc0e0f51c5cf0d779afefe291 54.68MB / 54.68MB 26.7s
=> => sha256:3d7b1480fa4dae5cbbb7d091c46ae0ae52f501418d4cfeb849b87023364e2564 10.87MB / 10.87MB 3.0s
=> => sha256:a3e29af4daf3531efcc63588162e8bdcf3434aa5d72df4eabeb5e20c6695e303 5.15MB / 5.15MB 1.3s
=> => sha256:077c13527d405646e2f6bb426e04716ae4f8dd2fdd8966dcb0194564a2b57896 53.70MB / 53.70MB 13.3s
=> => extracting sha256:077c13527d405646e2f6bb426e04716ae4f8dd2fdd8966dcb0194564a2b57896 0.9s
=> => extracting sha256:a3e29af4daf3531efcc63588162e8bdcf3434aa5d72df4eabeb5e20c6695e303 0.3s
=> => extracting sha256:3d7b1480fa4dae5cbbb7d091c46ae0ae52f501418d4cfeb849b87023364e2564 0.1s
=> => extracting sha256:426e8acfed2a5373bd99b22b5a968d55a148e14bc0e0f51c5cf0d779afefe291 1.2s
=> => extracting sha256:4fb255e3f99867ec7a2286dfbbef990491cde0a5d226d92be30bad4f9e917fa4 1.4s
=> => extracting sha256:fecd6ba4b3f93b6c90f4058b512f1b0a44223ccb3244f0049b16fe2c1b41cf45 2.0s
=> => extracting sha256:cd807e8b483974845eabbdbbaa4bb3a66f74facd8c061e01e923e9f1da608271 0.0s
=> [stage-1 1/3] FROM gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd 25.7s
=> => resolve gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd319 0.0s
=> => sha256:21d6a6c3921f47fb0a96eb028b4c3441944a6e5a44b30cd058425ccc66279760 7.13MB / 7.13MB 25.5s
=> => sha256:7d441aeb75fe3c941ee4477191c6b19edf2ad8310bac7356a799c20df198265c 657.02kB / 657.02kB 1.3s
=> => extracting sha256:7d441aeb75fe3c941ee4477191c6b19edf2ad8310bac7356a799c20df198265c 0.1s
=> => extracting sha256:21d6a6c3921f47fb0a96eb028b4c3441944a6e5a44b30cd058425ccc66279760 0.1s
=> [internal] load build context 0.0s
=> => transferring context: 54B 0.0s
=> [builder 2/5] WORKDIR /app 0.2s
=> [builder 3/5] COPY go.mod ./ 0.0s
=> [builder 4/5] COPY main.go ./ 0.0s
=> [builder 5/5] RUN go build -o /helloserver main.go 0.3s
=> [stage-1 2/3] COPY --from=builder /helloserver /helloserver 0.0s
=> exporting to oci image format 0.5s
=> => exporting layers 0.2s
=> => exporting manifest sha256:267ed1266b2b0ed74966e72d4ae8a2dfcf77777425d32a9a46f0938c962d9600 0.0s
=> => exporting config sha256:67102364e254bf5a8e58fa21ea56eb40645851d844f5c4d9651b4af7a40be780 0.0s
=> => sending tarball 0.3s
=> importing to docker
Check that the images were created correctly:
$ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
eu.gcr.io/acme/simpleserver 0.1.0-linux-arm64 67102364e254 6 seconds ago 22.4MB
eu.gcr.io/acme/simpleserver 0.1.0-linux-amd64 08641d64f612 About a minute ago 25.7MB
In the next step we create a component archive and a transport archive
$ PROVIDER=acme
$ COMPONENT=github.com/$(PROVIDER)/simpleserver
$ VERSION=0.1.0
$ mkdir gen
$ ocm create ca ${COMPONENT} ${VERSION} --provider ${PROVIDER} --file gen/ca
Create the file resources.yaml
. Note the variants in the image input and the type dockermulti
:
---
name: chart
type: helmChart
input:
type: helm
path: helmchart
---
name: image
type: ociImage
version: 0.1.0
input:
type: dockermulti
repository: eu.gcr.io/acme/simpleserver
variants:
- "eu.gcr.io/acme/simpleserver:0.1.0-linux-amd64"
- "eu.gcr.io/acme/simpleserver:0.1.0-linux-arm64"
The input type dockermulti
adds a multi-arch image composed by the given dedicated images from the local Docker
image store as local artifact to the component archive.
Add the described resources to your component archive:
$ ocm add resources ./gen/ca resources.yaml
processing resources.yaml...
processing document 1...
processing index 1
processing document 2...
processing index 1
found 2 resources
adding resource helmChart: "name"="chart","version"="<componentversion>"...
adding resource ociImage: "name"="image","version"="0.1.0"...
image 0: eu.gcr.io/acme/simpleserver:0.1.0-linux-amd64
image 1: eu.gcr.io/acme/simpleserver:0.1.0-linux-arm64
image 2: INDEX
locator: github.com/acme/simpleserver, repo: eu.gcr.io/acme/simpleserver, version 0.1.0
What happened?
The input type dockermulti
is used to compose a multi-arch image on-the fly. Like
the input type docker
it reads images from the local Docker daemon. In contrast to
this command you can list multiple images, created for different platforms, for which
an OCI index manifest is created to describe a multi-arch image. The complete set
of blobs is then packaged as artifact set archive and put as single resource into
the component version.
The resulting component-descriptor.yaml
in gen/ca
is:
component:
componentReferences: []
name: github.com/acme/simpleserver
provider: acme
repositoryContexts: []
resources:
- access:
localReference: sha256.9dd0f2cbae3b8e6eb07fa947c05666d544c0419a6e44bd607e9071723186333b
mediaType: application/vnd.oci.image.manifest.v1+tar+gzip
referenceName: github.com/acme/simpleserver/helloserver:0.1.0
type: localBlob
name: chart
relation: local
type: helmChart
version: 0.1.0
- access:
localReference: sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c
mediaType: application/vnd.oci.image.index.v1+tar+gzip
referenceName: github.com/acme/simpleserver/eu.gcr.io/acme/simpleserver:0.1.0
type: localBlob
name: image
relation: local
type: ociImage
version: 0.1.0
sources: []
version: 0.1.0
meta:
schemaVersion: v2
Note that there is only one resource of type image
with media-type application/vnd.oci.image.index.v1+tar+gzip
which is the standard media type for multi-arch images.
$ ls -l gen/ca/blobs
total 24M
-rw-r--r-- 1 d058463 staff 24M Dec 1 09:50 sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c
-rw-r--r-- 1 d058463 staff 4.7K Dec 1 09:50 sha256.9dd0f2cbae3b8e6eb07fa947c05666d544c0419a6e44bd607e9071723186333b
The file sha256.4e26… contains the multi-arch image packaged as OCI artifact set:
$ tar tvf gen/ca/blobs/sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c
-rw-r--r-- 0 0 0 741 Jan 1 2022 index.json
-rw-r--r-- 0 0 0 38 Jan 1 2022 oci-layout
drwxr-xr-x 0 0 0 0 Jan 1 2022 blobs
-rw-r--r-- 0 0 0 3051520 Jan 1 2022 blobs/sha256.05ef21d763159987b9ec5cfb3377a61c677809552dcac3301c0bde4e9fd41bbb
-rw-r--r-- 0 0 0 723 Jan 1 2022 blobs/sha256.117f12f0012875471168250f265af9872d7de23e19f0d4ef05fbe99a1c9a6eb3
-rw-r--r-- 0 0 0 6264832 Jan 1 2022 blobs/sha256.1496e46acd50a8a67ce65bac7e7287440071ad8d69caa80bcf144892331a95d3
-rw-r--r-- 0 0 0 6507520 Jan 1 2022 blobs/sha256.66817c8096ad97c6039297dc984ebc17c5ac9325200bfa9ddb555821912adbe4
-rw-r--r-- 0 0 0 491 Jan 1 2022 blobs/sha256.75a096351fe96e8be1847a8321bd66535769c16b2cf47ac03191338323349355
-rw-r--r-- 0 0 0 3051520 Jan 1 2022 blobs/sha256.77192cf194ddc77d69087b86b763c47c7f2b0f215d0e4bf4752565cae5ce728d
-rw-r--r-- 0 0 0 1138 Jan 1 2022 blobs/sha256.91018e67a671bbbd7ab875c71ca6917484ce76cde6a656351187c0e0e19fe139
-rw-r--r-- 0 0 0 17807360 Jan 1 2022 blobs/sha256.91f7bcfdfda81b6c6e51b8e1da58b48759351fa4fae9e6841dd6031528f63b4a
-rw-r--r-- 0 0 0 1138 Jan 1 2022 blobs/sha256.992b3b72df9922293c05f156f0e460a220bf601fa46158269ce6b7d61714a084
-rw-r--r-- 0 0 0 14755840 Jan 1 2022 blobs/sha256.a83c9b56bbe0f6c26c4b1d86e6de3a4862755d208c9dfae764f64b210eafa58c
-rw-r--r-- 0 0 0 723 Jan 1 2022 blobs/sha256.e624040295fb78a81f4b4b08b43b4de419f31f21074007df8feafc10dfb654e6
$ tar xvf gen/ca/blobs/sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c -O - index.json | jq .
x index.json
{
"schemaVersion": 2,
"mediaType": "application/vnd.oci.image.index.v1+json",
"manifests": [
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:e624040295fb78a81f4b4b08b43b4de419f31f21074007df8feafc10dfb654e6",
"size": 723
},
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"digest": "sha256:117f12f0012875471168250f265af9872d7de23e19f0d4ef05fbe99a1c9a6eb3",
"size": 723
},
{
"mediaType": "application/vnd.oci.image.index.v1+json",
"digest": "sha256:75a096351fe96e8be1847a8321bd66535769c16b2cf47ac03191338323349355",
"size": 491,
"annotations": {
"org.opencontainers.image.ref.name": "0.1.0",
"software.ocm/tags": "0.1.0"
}
}
],
"annotations": {
"software.ocm/main": "sha256:75a096351fe96e8be1847a8321bd66535769c16b2cf47ac03191338323349355"
}
}
You can create a common transport archive from the component archive.
cm transfer ca gen/ca gen/ctf
transferring version "github.com/acme/simpleserver:0.1.0"...
...resource 0(github.com/acme/simpleserver/helloserver:0.1.0)...
...resource 1(github.com/acme/simpleserver/eu.gcr.io/acme/simpleserver:0.1.0)...
...adding component version...
Or you can push it directly to the OCM repository:
$ OCMREPO=ghcr.io/${PROVIDER}
$ ocm transfer ca gen/ca $OCMREPO
transferring version "github.com/acme/simpleserver:0.1.0"...
...resource 0(github.com/acme/simpleserver/helloserver:0.1.0)...
...resource 1(github.com/acme/simpleserver/eu.gcr.io/acme/simpleserver:0.1.0)...
...adding component version...
The repository now should contain three additional artifacts. Depending on the OCI registry and
the corresponding UI you may see that the uploaded OCI image is a multi-arch-image. For example on
GitHub packages under the attribute OS/Arch
you can see two platforms, linux/amd64
and
linux/arm64
For automation and reuse purposes you may consider templating resource files and Makefiles (see below).
Using Makefiles
Developing with the Open Component Model usually is an iterative process of building artifacts,
generating component descriptors, analyzing and finally publishing them. To simplify and speed up this
process it should be automated using a build tool. One option is to use a Makefile
.
The following example can be used as a starting point and can be modified according to your needs.
In this example we will automate the same example as in the sections before:
- Creating a multi-arch image from Go sources from a Git repository using the Docker CLI
- Packaging the image and a Helm chart into a common transport archive
- Signing and publishing the build result
Prerequisites
- The OCM CLI must be installed and be available in your PATH
- The Makefile is located in the top-level folder of a Git project
- Operating system is Unix/Linux
- A sub-directory
local
can be used for local settings e.g. environment varibles, RSA keys, … - A sub-directory
gen
will be used for generated artifacts from themake build
command - It is recommended to add
local/
andgen/
to the.gitignore
file
We use the following file system layout for the example:
$ tree .
.
├── Dockerfile
├── LICENSE
├── Makefile
├── README.md
├── go.mod
├── helmchart
│ ├── Chart.yaml
│ ├── templates
│ │ ├── NOTES.txt
│ │ ├── _helpers.tpl
│ │ ├── deployment.yaml
│ │ ├── hpa.yaml
│ │ ├── ingress.yaml
│ │ ├── service.yaml
│ │ ├── serviceaccount.yaml
│ │ └── tests
│ │ └── test-connection.yaml
│ └── values.yaml
├── local
│ └── env.sh
├── main.go
├── resources.yaml
└── VERSION
This Makefile can be used
NAME ?= simpleserver
PROVIDER ?= acme.org
GITHUBORG ?= acme
IMAGE = ghcr.io/$(GITHUBORG)/demo/$(NAME)
COMPONENT = $(PROVIDER)/demo/$(NAME)
OCMREPO ?= ghcr.io/$(GITHUBORG)/ocm
MULTI ?= true
PLATFORMS ?= linux/amd64 linux/arm64
REPO_ROOT = .
VERSION = $(shell git describe --tags --exact-match 2>/dev/null|| echo "$$(cat $(REPO_ROOT)/VERSION)")
COMMIT = $(shell git rev-parse HEAD)
EFFECTIVE_VERSION = $(VERSION)-$(COMMIT)
GIT_TREE_STATE := $(shell [ -z "$(git status --porcelain 2>/dev/null)" ] && echo clean || echo dirty)
GEN = ./gen
OCM = ocm
CHART_SRCS=$(shell find helmchart -type f)
GO_SRCS=$(shell find . -name \*.go -type f)
ifeq ($(MULTI),true)
FLAGSUF = .multi
endif
.PHONY: build
build: $(GEN)/build
.PHONY: version
version:
@echo $(VERSION)
.PHONY: ca
ca: $(GEN)/ca
$(GEN)/ca: $(GEN)/.exists $(GEN)/image.$(NAME)$(FLAGSUF) resources.yaml $(CHART_SRCS)
$(OCM) create ca -f $(COMPONENT) "$(VERSION)" --provider $(PROVIDER) --file $(GEN)/ca
$(OCM) add resources --templater spiff $(GEN)/ca COMMIT="$(COMMIT)" VERSION="$(VERSION)" \
IMAGE="$(IMAGE):$(VERSION)" PLATFORMS="$(PLATFORMS)" MULTI=$(MULTI) resources.yaml
@touch $(GEN)/ca
$(GEN)/build: $(GO_SRCS)
go build .
@touch $(GEN)/build
.PHONY: image
image: $(GEN)/image.$(NAME)
$(GEN)/image.$(NAME): $(GEN)/.exists Dockerfile $(OCMSRCS)
docker build -t $(IMAGE):$(VERSION) --file Dockerfile $(COMPONENT_ROOT) .;
@touch $(GEN)/image.$(NAME)
.PHONY: multi
multi: $(GEN)/image.$(NAME).multi
$(GEN)/image.$(NAME).multi: $(GEN)/.exists Dockerfile $(GO_SRCS)
echo "Building Multi $(PLATFORMS)"
for i in $(PLATFORMS); do \
tag=$$(echo $$i | sed -e s:/:-:g); \
echo "Building platform $$i with tag: $$tag"; \
docker buildx build --load -t $(IMAGE):$(VERSION)-$$tag --platform $$i .; \
done
@touch $(GEN)/image.$(NAME).multi
.PHONY: ctf
ctf: $(GEN)/ctf
$(GEN)/ctf: $(GEN)/ca
@rm -rf $(GEN)/ctf
$(OCM) transfer ca $(GEN)/ca $(GEN)/ctf
touch $(GEN)/ctf
.PHONY: push
push: $(GEN)/ctf $(GEN)/push.$(NAME)
$(GEN)/push.$(NAME): $(GEN)/ctf
$(OCM) transfer ctf -f $(GEN)/ctf $(OCMREPO)
@touch $(GEN)/push.$(NAME)
.PHONY: transport
transport:
ifneq ($(TARGETREPO),)
$(OCM) transfer component -Vc $(OCMREPO)//$(COMPONENT):$(VERSION) $(TARGETREPO)
else
@echo "Cannot transport no TARGETREPO defined as destination" && exit 1
endif
$(GEN)/.exists:
@mkdir -p $(GEN)
@touch $@
.PHONY: info
info:
@echo "VERSION: $(VERSION)"
@echo "COMMIT: $(COMMIT)"
@echo "TREESTATE: $(GIT_TREE_STATE)"
.PHONY: describe
describe: $(GEN)/ctf
ocm get resources --lookup $(OCMREPO) -r -o treewide $(GEN)/ctf
.PHONY: descriptor
descriptor: $(GEN)/ctf
ocm get component -S v3alpha1 -o yaml $(GEN)/ctf
.PHONY: clean
clean:
rm -rf $(GEN)
The Makefile supports the following targets:
build
(default) simple Go buildversion
show current VERSION of Github repositoryimage
build a local Docker imagemulti
build multi-arch images with Dockerca
execute build and create a component archivectf
create a common transport format archivepush
push the common transport archive to an OCI registryinfo
show variables used in Makefile (version, commit, etc.)describe
display the component version in a tree-formdescriptor
show the component descriptor of the component versiontransport
transport the component from the upload repository into another OCM repositoryclean
delete all generated files (but does not delete Docker images)
The variables assigned with ?=
at the beginning can be set from outside and override the default
declared in the Makefile. Use either an environment variable or an argument when calling make
.
Example:
$ PROVIDER=foo make ca
Templating the Resources
The Makefile uses a dynamic list of generated platforms for the images. You can just set the PLATFORMS
variable:
MULTI ?= true
PLATFORMS ?= linux/amd64 linux/arm64
If MULTI
is set to true
, the variable PLATFORMS
will be evaluated to decide which image variants
will be built. This has to be reflected in the resources.yaml
. It has to use the input type
dockermulti
and list all the variants which should be packaged into a multi-arch image. This list
depends on the content of the Make variable.
The OCM CLI supports this by enabling templating mechanisms for the content by selecting a templater
using the option --templater ...
. The example uses the Spiff templater.
$(GEN)/ca: $(GEN)/.exists $(GEN)/image.$(NAME)$(FLAGSUF) resources.yaml $(CHART_SRCS)
$(OCM) create ca -f $(COMPONENT) "$(VERSION)" --provider $(PROVIDER) --file $(GEN)/ca
$(OCM) add resources --templater spiff $(GEN)/ca COMMIT="$(COMMIT)" VERSION="$(VERSION)" \
IMAGE="$(IMAGE):$(VERSION)" PLATFORMS="$(PLATFORMS)" MULTI=$(MULTI) resources.yaml
@touch $(GEN)/ca
The variables given to the add resources
command are passed to the templater. The template looks
like:
name: image
type: ociImage
version: (( values.VERSION ))
input:
type: (( bool(values.MULTI) ? "dockermulti" :"docker" ))
repository: (( index(values.IMAGE, ":") >= 0 ? substr(values.IMAGE,0,index(values.IMAGE,":")) :values.IMAGE ))
variants: (( bool(values.MULTI) ? map[split(" ", values.PLATFORMS)|v|-> values.IMAGE "-" replace(v,"/","-")] :~~ ))
path: (( bool(values.MULTI) ? ~~ :values.IMAGE ))
By using a variable values.MULTI
, the command distinguishes between a single Docker image and a multi-arch image.
With map[]
, the platform list from the Makefile is mapped to a list of tags created by the
docker buildx
command used in the Makefile. The value ~~
is used to undefine the yaml fields not
required for the selected case (the template can be used for multi- and single-arch builds).
$(GEN)/image.$(NAME).multi: $(GEN)/.exists Dockerfile $(GO_SRCS)
echo "Building Multi $(PLATFORMS)"
for i in $(PLATFORMS); do \
tag=$$(echo $$i | sed -e s:/:-:g); \
echo "Building platform $$i with tag: $$tag"; \
docker buildx build --load -t $(IMAGE):$(VERSION)-$$tag --platform $$i .; \
done
@touch $(GEN)/image.$(NAME).multi
Pipeline Integration
Pipeline infrastructures are heterogenous, so there is no universal answer how to integrate a build pipeline with OCM. Usually, the simplest way is using the OCM command line interface. Following you will find an example using GitHub actions.
There are two repositories dealing with GitHub actions: The first one provides various actions that can be called from a workflow. The second one provides the required installations of the OCM parts into the container.
An typical workflow for a build step will create a component version and a transport archive:
jobs:
create-ocm:
runs-on: ubuntu-latest
steps:
...
- name: setup OCM
uses: open-component-model/ocm-setup-action@main
...
- name: create OCM component version
uses: open-component-model/ocm-action@main
with:
action: create_component
component: acme.org/demo/simpleserver
provider: ${{ env.PROVIDER }}
version: github.com/jensh007
...
This creates a component version for the current build. Additionally, a transport archive may be created or the component version along with the built container images may be uploaded to an OCI registry, etc.
More documentation is available here. A full example can be found in the sample Github repository.
Static and Dynamic Variable Substitution
Looking at the settings file shows that
some variables like the version
or the commit
change with every build
or release. In many cases, these variables will be auto-generated during the build.
Other variables like the version of 3rd-party components will just change from time to time and are often set manually by an engineer or release manager. It is useful to separate between static and dynamic variables. Static files can be checked-in into the source control system and are maintained manually. Dynamic variables can be generated during build.
Example: manually maintained:
NAME: microblog
COMPONENT_NAME_PREFIX: github.com/acme.org/microblog
PROVIDER: ocm.software
ELASTIC_VERSION: 8.5.1
MARIADB_VERSION: 10.6.11
MARIADB_CHART_VERSION: 11.4.2
NGINX_VERSION: 1.5.1
NGINX_CHART_VERSION: 4.4.2
auto-generated from a build script:
VERSION: 0.23.1
COMMIT: 5f03021059c7dbe760ac820a014a8a84166ef8b4
ocm add componentversions --create --file ../gen/ctf --settings ../gen/dynamic_settings.yaml --settings static_settings.yaml component-constructor.yaml
Debugging: Explain the Blobs Directory
For analyzing and debugging the content of a transport archive, there are some supportive commands to analyze what is contained in the archive and what is stored in which blob:
tree ../gen/ctf
../gen/ctf
├── artifact-index.json
└── blobs
├── ...
├── sha256.59ff88331c53a2a94cdd98df58bc6952f056e4b2efc8120095fbc0a870eb0b67
├── ...
ocm get resources -r -o wide ../gen/ctf
...
---
REFERENCEPATH: github.com/acme.org/microblog/nginx-controller:1.5.1
NAME : nginx-controller-chart
VERSION : 1.5.1
IDENTITY :
TYPE : helmChart
RELATION : local
ACCESSTYPE : localBlob
ACCESSSPEC : {"localReference":"sha256:59ff88331c53a2a94cdd98df58bc6952f056e4b2efc8120095fbc0a870eb0b67","mediaType":"application/vnd.oci.image.manifest.v1+tar+gzip","referenceName":"github.com/acme.org/microblog/nginx-controller/ingress-nginx:4.4.2"}
...
Self-Contained Transport Archives
The transport archive created from a component-constructor file, using the command ocm add componentversions --create ...
, does not automatically resolve image references to external OCI registries and stores them in the archive. If you want to create a self-contained transport archive with all images stored as local artifacts, you need to use the --copy-resources
option of the ocm transfer ctf
command. This will copy all external images to the blobs directory of the archive.
ocm transfer ctf --copy-resources <ctf-dir> <new-ctf-dir-or-oci-repo-url>
Note that this archive can become huge if there an many external images involved!
CICD Integration
Configure rarely changing variables in a static file and generate dynamic variables during the build from the environment. See the Static and Dynamic Variable Substitution section above.