Best Practices

This chapter contains guidelines for common scenarios how to work with the Open Component Model, focusing on using CI/CD, build and publishing processes.

Use Public Schema for Validation and Auto-Completion of Component Descriptors
Separate Build and Publish Processes
Separation Between Build and Publish
Building Multi-Architecture Images
Using Makefiles
- Prerequisites
- Templating the Resources
Pipeline Integration
Static and Dynamic Variable Substitution
Debugging: Explain the Blobs Directory
Self-Contained Transport Archives
CICD Integration

Use Public Schema for Validation and Auto-Completion of Component Descriptors

The Open Component Model (OCM) provides a public schema to validate and offer auto-completion of component constructor files used to create component descriptors. This schema is available at https://ocm.software/schemas/configuration-schema.yaml.

To use this schema in your IDE, you can add the following line to your component constructor file:

# yaml-language-server: $schema=https://ocm.software/schemas/configuration-schema.yaml

This line tells the YAML language server to use the OCM schema for validation and auto-completion.

Separate Build and Publish Processes

Traditional automated builds often have unrestricted internet access, which can lead to several challenges in enterprise environments:

Limited control over downloaded artifacts
Potential unavailability of required resources
Security risks associated with write permissions to external repositories

Best practice: Implement a two-step process: a) Build: Create artifacts in a controlled environment, using local mirrors when possible. b) Publish: Use a separate, secured process to distribute build results.

OCM supports this approach through filesystem-based OCM repositories, allowing you to generate Common Transport Format (CTF) archives for component versions. These archives can then be securely processed and distributed.

Separation Between Build and Publish

Typical automated builds have access to the complete internet ecosystem. This involves downloading of content required for a build (e.g., go mod tidy), but also the upload of build results to repositories (e.g., OCI image registries).

For builds in enterprise environments this can lead to several challenges:

Limited control over downloaded artifacts
Potential unavailability of required resources
Security risks associated with write permissions to external repositories

The first problem might be acceptable, because the build results may be analyzed by scanners later to figure out what has been packaged. Triaging the results could be done in an asynchronous step later.

The second problem could be solved by mirroring the required artifacts and instrument the build to use the artifacts from the local mirror. Such a mirror would also offer a solution for the first problem and act as target for various scanning tools.

The third problem might pose severe security risks, because the build procedure as well as the downloaded artifacts may be used to catch registry credentials or at least corrupt the content of those repositories.

This could be avoided by establishing a contract between the build procedure of a component/project and the build system, providing the build result as a local file or file-structure. This is then taken by the build system to push content wherever it should be pushed to. This way the execution of the build procedure does not need write permissions to any repository, because it never pushes build results.

The Open Component Model supports such processes by supporting filesystem based OCM repositories, which are able to host any type of content, regardless of its technology. The task of the build then is to provide such a CTF archive for the OCM component versions generated by the build. This archive can then be used by the build-system to do whatever is required to make the content accessible by others.

The composition of such archives is described in the Getting Started Section.

To secure further processes, a certified build-system could even sign the content with its build system certificate to enable followup-processes to verify that involved component versions are provided by accepted and well-known processes.

Building Multi-Architecture Images

Note: This section provides information only on on building multi-arch images. Referencing a multi-arch image does not differ from images for just one platform, see the ocm add resources command for the OCM CLI

At the time of writing this guide Docker is not able to build multi-architecture (multi-arch / multi-platform) images natively. Instead, the buildx plugin is used. However, this implies building and pushing images in one step to a remote container registry as the local Docker image store does not support multi-arch images (for additional information, see the Multi-arch build and images, the simple way blogpost)

The OCM CLI has some built-in support for dealing with multi-arch images during the component version composition (ocm add resources). This allows building all artifacts locally and push them in a separate step to a container registry. This is done by building single-arch images in a first step (still using buildx for cross-platform building). In a second step all images are bundled into a multi-arch image, which is stored as local artifact in a component (CA) or common transport (CTF) archive. This archive can be processed as usual (e.g., for signing or transfer to other locations). When pushed to an image registry, multi-arch images are generated with a multi-arch-image manifest.

The following steps illustrate this procedure. For a simple project with a Go binary and a helm-chart assume the following file structure:

 tree .
.
├── Dockerfile
├── go.mod
├── helmchart
│   ├── Chart.yaml
│   ├── templates
│   │   ├── ...
│   └── values.yaml
└── main.go

The Dockerfile has the following content:

FROM golang:1.19 AS builder

WORKDIR /app
COPY go.mod ./
COPY main.go ./
# RUN go mod download
RUN go build -o /helloserver main.go

# Create a new release build stage
FROM gcr.io/distroless/base-debian10

# Set the working directory to the root directory path
WORKDIR /
# Copy over the binary built from the previous stage
COPY --from=builder /helloserver /helloserver

ENTRYPOINT ["/helloserver"]

Now we want to build images for two platforms using Docker and buildx. Note the --load option for buildx to store the image in the local Docker store. Note the architecture suffix in the tag to be able to distinguish the images for the different platforms. Also note that the tag has a different syntax than the --platform argument for buildx as slashes are not allowed in tags.

$ TAG_PREFIX=eu.gcr.io/my-project/acme # path to you OCI registry
$ PLATFORM=linux/amd64
$ VERSION=0.1.0

$ docker buildx build --load -t ${TAG_PREFIX}/simpleserver:0.1.0-linux-amd64 --platform linux/amd64 .
[+] Building 54.4s (14/14) FINISHED
 => [internal] load build definition from dockerfile                                                                          0.0s
 => => transferring dockerfile: 660B                                                                                          0.0s
 => [internal] load .dockerignore                                                                                             0.0s
 => => transferring context: 2B                                                                                               0.0s
 => [internal] load metadata for gcr.io/distroless/base-debian10:latest                                                       1.6s
 => [internal] load metadata for docker.io/library/golang:1.19                                                                1.2s
 => [builder 1/5] FROM docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e851425  49.2s
 => => resolve docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e8514253          0.0s
 => => sha256:14a70245b07c7f5056bdd90a3d93e37417ec26542def5a37ac8f19e437562533 156B / 156B                                    0.2s
 => => sha256:a2b47720d601b6c6c6e7763b4851e25475118d80a76be466ef3aa388abf2defd 148.91MB / 148.91MB                           46.3s
 => => sha256:52908dc1c386fab0271a2b84b6ef4d96205a98a8c8801169554767172e45d8c7 85.97MB / 85.97MB                             42.9s
 => => sha256:195ea6a58ca87a18477965a6e6a8623112bde82c5b568a29c56ce4581b6e6695 54.59MB / 54.59MB                             33.8s
 => => sha256:c85a0be79bfba309d1f05dc40b447aa82b604593531fed1e7e12e4bef63483a5 10.88MB / 10.88MB                              3.4s
 => => sha256:e4e46864aba2e62ba7c75965e4aa33ec856ee1b1074dda6b478101c577b63abd 5.16MB / 5.16MB                                1.5s
 => => sha256:a8ca11554fce00d9177da2d76307bdc06df7faeb84529755c648ac4886192ed1 55.04MB / 55.04MB                             19.3s
 => => extracting sha256:a8ca11554fce00d9177da2d76307bdc06df7faeb84529755c648ac4886192ed1                                     1.1s
 => => extracting sha256:e4e46864aba2e62ba7c75965e4aa33ec856ee1b1074dda6b478101c577b63abd                                     0.1s
 => => extracting sha256:c85a0be79bfba309d1f05dc40b447aa82b604593531fed1e7e12e4bef63483a5                                     0.1s
 => => extracting sha256:195ea6a58ca87a18477965a6e6a8623112bde82c5b568a29c56ce4581b6e6695                                     1.1s
 => => extracting sha256:52908dc1c386fab0271a2b84b6ef4d96205a98a8c8801169554767172e45d8c7                                     1.5s
 => => extracting sha256:a2b47720d601b6c6c6e7763b4851e25475118d80a76be466ef3aa388abf2defd                                     2.8s
 => => extracting sha256:14a70245b07c7f5056bdd90a3d93e37417ec26542def5a37ac8f19e437562533                                     0.0s
 => [stage-1 1/3] FROM gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd  30.7s
 => => resolve gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd319        0.0s
 => => sha256:f291067d32d8d06c3b996ba726b9aa93a71f6f573098880e05d16660cfc44491 8.12MB / 8.12MB                               30.6s
 => => sha256:2445dbf7678f5ec17f5654ac2b7ad14d7b1ea3af638423fc68f5b38721f25fa4 657.02kB / 657.02kB                            1.3s
 => => extracting sha256:2445dbf7678f5ec17f5654ac2b7ad14d7b1ea3af638423fc68f5b38721f25fa4                                     0.1s
 => => extracting sha256:f291067d32d8d06c3b996ba726b9aa93a71f6f573098880e05d16660cfc44491                                     0.1s
 => [internal] load build context                                                                                             0.1s
 => => transferring context: 575B                                                                                             0.0s
 => [builder 2/5] WORKDIR /app                                                                                                0.1s
 => [builder 3/5] COPY go.mod ./                                                                                              0.0s
 => [builder 4/5] COPY main.go ./                                                                                             0.0s
 => [builder 5/5] RUN go build -o /helloserver main.go                                                                        2.4s
 => [stage-1 2/3] COPY --from=builder /helloserver /helloserver                                                               0.0s
 => exporting to oci image format                                                                                             0.8s
 => => exporting layers                                                                                                       0.2s
 => => exporting manifest sha256:04d69fc3245757d327d96b1a83b7a64543d970953c61d1014ae6980ed8b3ba2a                             0.0s
 => => exporting config sha256:08641d64f612661a711587b07cfeeb6d2804b97998cfad85864a392c1aabcd06                               0.0s
 => => sending tarball                                                                                                        0.6s
 => importing to docker

Repeat this command for the second platform:

$ docker buildx build --load -t ${TAG_PREFIX}/simpleserver:0.1.0-linux-arm64 --platform linux/arm64 .
docker buildx build --load -t ${TAG_PREFIX}/simpleserver:0.1.0-linux-arm64 --platform linux/arm64 .
[+] Building 40.1s (14/14) FINISHED
 => [internal] load .dockerignore                                                                                             0.0s
 => => transferring context: 2B                                                                                               0.0s
 => [internal] load build definition from dockerfile                                                                          0.0s
 => => transferring dockerfile: 660B                                                                                          0.0s
 => [internal] load metadata for gcr.io/distroless/base-debian10:latest                                                       1.0s
 => [internal] load metadata for docker.io/library/golang:1.19                                                                1.1s
 => [builder 1/5] FROM docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e851425  37.7s
 => => resolve docker.io/library/golang:1.19@sha256:dc76ef03e54c34a00dcdca81e55c242d24b34d231637776c4bb5c1a8e8514253          0.0s
 => => sha256:cd807e8b483974845eabbdbbaa4bb3a66f74facd8c061e01e923e9f1da608271 157B / 157B                                    0.2s
 => => sha256:fecd6ba4b3f93b6c90f4058b512f1b0a44223ccb3244f0049b16fe2c1b41cf45 115.13MB / 115.13MB                           35.6s
 => => sha256:4fb255e3f99867ec7a2286dfbbef990491cde0a5d226d92be30bad4f9e917fa4 81.37MB / 81.37MB                             31.8s
 => => sha256:426e8acfed2a5373bd99b22b5a968d55a148e14bc0e0f51c5cf0d779afefe291 54.68MB / 54.68MB                             26.7s
 => => sha256:3d7b1480fa4dae5cbbb7d091c46ae0ae52f501418d4cfeb849b87023364e2564 10.87MB / 10.87MB                              3.0s
 => => sha256:a3e29af4daf3531efcc63588162e8bdcf3434aa5d72df4eabeb5e20c6695e303 5.15MB / 5.15MB                                1.3s
 => => sha256:077c13527d405646e2f6bb426e04716ae4f8dd2fdd8966dcb0194564a2b57896 53.70MB / 53.70MB                             13.3s
 => => extracting sha256:077c13527d405646e2f6bb426e04716ae4f8dd2fdd8966dcb0194564a2b57896                                     0.9s
 => => extracting sha256:a3e29af4daf3531efcc63588162e8bdcf3434aa5d72df4eabeb5e20c6695e303                                     0.3s
 => => extracting sha256:3d7b1480fa4dae5cbbb7d091c46ae0ae52f501418d4cfeb849b87023364e2564                                     0.1s
 => => extracting sha256:426e8acfed2a5373bd99b22b5a968d55a148e14bc0e0f51c5cf0d779afefe291                                     1.2s
 => => extracting sha256:4fb255e3f99867ec7a2286dfbbef990491cde0a5d226d92be30bad4f9e917fa4                                     1.4s
 => => extracting sha256:fecd6ba4b3f93b6c90f4058b512f1b0a44223ccb3244f0049b16fe2c1b41cf45                                     2.0s
 => => extracting sha256:cd807e8b483974845eabbdbbaa4bb3a66f74facd8c061e01e923e9f1da608271                                     0.0s
 => [stage-1 1/3] FROM gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd  25.7s
 => => resolve gcr.io/distroless/base-debian10@sha256:101798a3b76599762d3528635113f0466dc9655ecba82e8e33d410e2bf5cd319        0.0s
 => => sha256:21d6a6c3921f47fb0a96eb028b4c3441944a6e5a44b30cd058425ccc66279760 7.13MB / 7.13MB                               25.5s
 => => sha256:7d441aeb75fe3c941ee4477191c6b19edf2ad8310bac7356a799c20df198265c 657.02kB / 657.02kB                            1.3s
 => => extracting sha256:7d441aeb75fe3c941ee4477191c6b19edf2ad8310bac7356a799c20df198265c                                     0.1s
 => => extracting sha256:21d6a6c3921f47fb0a96eb028b4c3441944a6e5a44b30cd058425ccc66279760                                     0.1s
 => [internal] load build context                                                                                             0.0s
 => => transferring context: 54B                                                                                              0.0s
 => [builder 2/5] WORKDIR /app                                                                                                0.2s
 => [builder 3/5] COPY go.mod ./                                                                                              0.0s
 => [builder 4/5] COPY main.go ./                                                                                             0.0s
 => [builder 5/5] RUN go build -o /helloserver main.go                                                                        0.3s
 => [stage-1 2/3] COPY --from=builder /helloserver /helloserver                                                               0.0s
 => exporting to oci image format                                                                                             0.5s
 => => exporting layers                                                                                                       0.2s
 => => exporting manifest sha256:267ed1266b2b0ed74966e72d4ae8a2dfcf77777425d32a9a46f0938c962d9600                             0.0s
 => => exporting config sha256:67102364e254bf5a8e58fa21ea56eb40645851d844f5c4d9651b4af7a40be780                               0.0s
 => => sending tarball                                                                                                        0.3s
 => importing to docker

Check that the images were created correctly:

$ docker image ls
REPOSITORY                                              TAG                 IMAGE ID       CREATED              SIZE
eu.gcr.io/acme/simpleserver   0.1.0-linux-arm64   67102364e254   6 seconds ago        22.4MB
eu.gcr.io/acme/simpleserver   0.1.0-linux-amd64   08641d64f612   About a minute ago   25.7MB

In the next step we create a component archive and a transport archive

$ PROVIDER=acme
$ COMPONENT=github.com/$(PROVIDER)/simpleserver
$ VERSION=0.1.0
$ mkdir gen
$ ocm create ca ${COMPONENT} ${VERSION} --provider ${PROVIDER} --file gen/ca

Create the file resources.yaml. Note the variants in the image input and the type dockermulti:

---
name: chart
type: helmChart
input:
  type: helm
  path: helmchart
---
name: image
type: ociImage
version: 0.1.0
input:
  type: dockermulti
  repository: eu.gcr.io/acme/simpleserver
  variants:
  - "eu.gcr.io/acme/simpleserver:0.1.0-linux-amd64"
  - "eu.gcr.io/acme/simpleserver:0.1.0-linux-arm64"

The input type dockermulti adds a multi-arch image composed by the given dedicated images from the local Docker image store as local artifact to the component archive.

Add the described resources to your component archive:

$ ocm add resources ./gen/ca resources.yaml
processing resources.yaml...
  processing document 1...
    processing index 1
  processing document 2...
    processing index 1
found 2 resources
adding resource helmChart: "name"="chart","version"="<componentversion>"...
adding resource ociImage: "name"="image","version"="0.1.0"...
  image 0: eu.gcr.io/acme/simpleserver:0.1.0-linux-amd64
  image 1: eu.gcr.io/acme/simpleserver:0.1.0-linux-arm64
  image 2: INDEX
locator: github.com/acme/simpleserver, repo: eu.gcr.io/acme/simpleserver, version 0.1.0

What happened?

The input type dockermulti is used to compose a multi-arch image on-the fly. Like the input type docker it reads images from the local Docker daemon. In contrast to this command you can list multiple images, created for different platforms, for which an OCI index manifest is created to describe a multi-arch image. The complete set of blobs is then packaged as artifact set archive and put as single resource into the component version.

The resulting component-descriptor.yaml in gen/ca is:

component:
  componentReferences: []
  name: github.com/acme/simpleserver
  provider: acme
  repositoryContexts: []
  resources:
  - access:
      localReference: sha256.9dd0f2cbae3b8e6eb07fa947c05666d544c0419a6e44bd607e9071723186333b
      mediaType: application/vnd.oci.image.manifest.v1+tar+gzip
      referenceName: github.com/acme/simpleserver/helloserver:0.1.0
      type: localBlob
    name: chart
    relation: local
    type: helmChart
    version: 0.1.0
  - access:
      localReference: sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c
      mediaType: application/vnd.oci.image.index.v1+tar+gzip
      referenceName: github.com/acme/simpleserver/eu.gcr.io/acme/simpleserver:0.1.0
      type: localBlob
    name: image
    relation: local
    type: ociImage
    version: 0.1.0
  sources: []
  version: 0.1.0
meta:
  schemaVersion: v2

Note that there is only one resource of type image with media-type application/vnd.oci.image.index.v1+tar+gzip which is the standard media type for multi-arch images.

$ ls -l gen/ca/blobs
total 24M
-rw-r--r-- 1 d058463 staff  24M Dec  1 09:50 sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c
-rw-r--r-- 1 d058463 staff 4.7K Dec  1 09:50 sha256.9dd0f2cbae3b8e6eb07fa947c05666d544c0419a6e44bd607e9071723186333b

The file sha256.4e26… contains the multi-arch image packaged as OCI artifact set:

$ tar tvf gen/ca/blobs/sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c
-rw-r--r--  0 0      0         741 Jan  1  2022 index.json
-rw-r--r--  0 0      0          38 Jan  1  2022 oci-layout
drwxr-xr-x  0 0      0           0 Jan  1  2022 blobs
-rw-r--r--  0 0      0     3051520 Jan  1  2022 blobs/sha256.05ef21d763159987b9ec5cfb3377a61c677809552dcac3301c0bde4e9fd41bbb
-rw-r--r--  0 0      0         723 Jan  1  2022 blobs/sha256.117f12f0012875471168250f265af9872d7de23e19f0d4ef05fbe99a1c9a6eb3
-rw-r--r--  0 0      0     6264832 Jan  1  2022 blobs/sha256.1496e46acd50a8a67ce65bac7e7287440071ad8d69caa80bcf144892331a95d3
-rw-r--r--  0 0      0     6507520 Jan  1  2022 blobs/sha256.66817c8096ad97c6039297dc984ebc17c5ac9325200bfa9ddb555821912adbe4
-rw-r--r--  0 0      0         491 Jan  1  2022 blobs/sha256.75a096351fe96e8be1847a8321bd66535769c16b2cf47ac03191338323349355
-rw-r--r--  0 0      0     3051520 Jan  1  2022 blobs/sha256.77192cf194ddc77d69087b86b763c47c7f2b0f215d0e4bf4752565cae5ce728d
-rw-r--r--  0 0      0        1138 Jan  1  2022 blobs/sha256.91018e67a671bbbd7ab875c71ca6917484ce76cde6a656351187c0e0e19fe139
-rw-r--r--  0 0      0    17807360 Jan  1  2022 blobs/sha256.91f7bcfdfda81b6c6e51b8e1da58b48759351fa4fae9e6841dd6031528f63b4a
-rw-r--r--  0 0      0        1138 Jan  1  2022 blobs/sha256.992b3b72df9922293c05f156f0e460a220bf601fa46158269ce6b7d61714a084
-rw-r--r--  0 0      0    14755840 Jan  1  2022 blobs/sha256.a83c9b56bbe0f6c26c4b1d86e6de3a4862755d208c9dfae764f64b210eafa58c
-rw-r--r--  0 0      0         723 Jan  1  2022 blobs/sha256.e624040295fb78a81f4b4b08b43b4de419f31f21074007df8feafc10dfb654e6

$ tar xvf gen/ca/blobs/sha256.4e26c7dd46e13c9b1672e4b28a138bdcb086e9b9857b96c21e12839827b48c0c -O - index.json | jq .
x index.json
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.index.v1+json",
  "manifests": [
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:e624040295fb78a81f4b4b08b43b4de419f31f21074007df8feafc10dfb654e6",
      "size": 723
    },
    {
      "mediaType": "application/vnd.oci.image.manifest.v1+json",
      "digest": "sha256:117f12f0012875471168250f265af9872d7de23e19f0d4ef05fbe99a1c9a6eb3",
      "size": 723
    },
    {
      "mediaType": "application/vnd.oci.image.index.v1+json",
      "digest": "sha256:75a096351fe96e8be1847a8321bd66535769c16b2cf47ac03191338323349355",
      "size": 491,
      "annotations": {
        "org.opencontainers.image.ref.name": "0.1.0",
        "software.ocm/tags": "0.1.0"
      }
    }
  ],
  "annotations": {
    "software.ocm/main": "sha256:75a096351fe96e8be1847a8321bd66535769c16b2cf47ac03191338323349355"
  }
}

You can create a common transport archive from the component archive.

cm transfer ca gen/ca gen/ctf
transferring version "github.com/acme/simpleserver:0.1.0"...
...resource 0(github.com/acme/simpleserver/helloserver:0.1.0)...
...resource 1(github.com/acme/simpleserver/eu.gcr.io/acme/simpleserver:0.1.0)...
...adding component version...

Or you can push it directly to the OCM repository:

$ OCMREPO=ghcr.io/${PROVIDER}
$ ocm transfer ca gen/ca $OCMREPO
transferring version "github.com/acme/simpleserver:0.1.0"...
...resource 0(github.com/acme/simpleserver/helloserver:0.1.0)...
...resource 1(github.com/acme/simpleserver/eu.gcr.io/acme/simpleserver:0.1.0)...
...adding component version...

The repository now should contain three additional artifacts. Depending on the OCI registry and the corresponding UI you may see that the uploaded OCI image is a multi-arch-image. For example on GitHub packages under the attribute OS/Arch you can see two platforms, linux/amd64 and linux/arm64

For automation and reuse purposes you may consider templating resource files and Makefiles (see below).

Using Makefiles

Developing with the Open Component Model usually is an iterative process of building artifacts, generating component descriptors, analyzing and finally publishing them. To simplify and speed up this process it should be automated using a build tool. One option is to use a Makefile. The following example can be used as a starting point and can be modified according to your needs.

In this example we will automate the same example as in the sections before:

Creating a multi-arch image from Go sources from a Git repository using the Docker CLI
Packaging the image and a Helm chart into a common transport archive
Signing and publishing the build result

Prerequisites

The OCM CLI must be installed and be available in your PATH
The Makefile is located in the top-level folder of a Git project
Operating system is Unix/Linux
A sub-directory local can be used for local settings e.g. environment varibles, RSA keys, …
A sub-directory gen will be used for generated artifacts from the make buildcommand
It is recommended to add local/ and gen/ to the .gitignore file

We use the following file system layout for the example:

$ tree .
.
├── Dockerfile
├── LICENSE
├── Makefile
├── README.md
├── go.mod
├── helmchart
│   ├── Chart.yaml
│   ├── templates
│   │   ├── NOTES.txt
│   │   ├── _helpers.tpl
│   │   ├── deployment.yaml
│   │   ├── hpa.yaml
│   │   ├── ingress.yaml
│   │   ├── service.yaml
│   │   ├── serviceaccount.yaml
│   │   └── tests
│   │       └── test-connection.yaml
│   └── values.yaml
├── local
│   └── env.sh
├── main.go
├── resources.yaml
└── VERSION

This Makefile can be used

NAME      ?= simpleserver
PROVIDER  ?= acme.org
GITHUBORG ?= acme
IMAGE     = ghcr.io/$(GITHUBORG)/demo/$(NAME)
COMPONENT = $(PROVIDER)/demo/$(NAME)
OCMREPO   ?= ghcr.io/$(GITHUBORG)/ocm
MULTI     ?= true
PLATFORMS ?= linux/amd64 linux/arm64
REPO_ROOT           = .
VERSION             = $(shell git describe --tags --exact-match 2>/dev/null|| echo "$$(cat $(REPO_ROOT)/VERSION)")
COMMIT              = $(shell git rev-parse HEAD)
EFFECTIVE_VERSION   = $(VERSION)-$(COMMIT)
GIT_TREE_STATE      := $(shell [ -z "$(git status --porcelain 2>/dev/null)" ] && echo clean || echo dirty)
GEN = ./gen
OCM = ocm

CHART_SRCS=$(shell find helmchart -type f)
GO_SRCS=$(shell find . -name \*.go -type f)

ifeq ($(MULTI),true)
FLAGSUF     = .multi
endif

.PHONY: build
build: $(GEN)/build

.PHONY: version
version:
	@echo $(VERSION)

.PHONY: ca
ca: $(GEN)/ca

$(GEN)/ca: $(GEN)/.exists $(GEN)/image.$(NAME)$(FLAGSUF) resources.yaml $(CHART_SRCS)
	$(OCM) create ca -f $(COMPONENT) "$(VERSION)" --provider $(PROVIDER) --file $(GEN)/ca
	$(OCM) add resources --templater spiff $(GEN)/ca COMMIT="$(COMMIT)" VERSION="$(VERSION)" \
		IMAGE="$(IMAGE):$(VERSION)" PLATFORMS="$(PLATFORMS)" MULTI=$(MULTI) resources.yaml
	@touch $(GEN)/ca

$(GEN)/build: $(GO_SRCS)
	go build .
	@touch $(GEN)/build

.PHONY: image
image: $(GEN)/image.$(NAME)

$(GEN)/image.$(NAME): $(GEN)/.exists Dockerfile $(OCMSRCS)
	docker build -t $(IMAGE):$(VERSION) --file Dockerfile $(COMPONENT_ROOT) .;
	@touch $(GEN)/image.$(NAME)

.PHONY: multi
multi: $(GEN)/image.$(NAME).multi

$(GEN)/image.$(NAME).multi: $(GEN)/.exists Dockerfile $(GO_SRCS)
	echo "Building Multi $(PLATFORMS)"
	for i in $(PLATFORMS); do \
	tag=$$(echo $$i | sed -e s:/:-:g); \
	echo "Building platform $$i with tag: $$tag"; \
	docker buildx build --load -t $(IMAGE):$(VERSION)-$$tag --platform $$i .; \
	done
	@touch $(GEN)/image.$(NAME).multi

.PHONY: ctf
ctf: $(GEN)/ctf

$(GEN)/ctf: $(GEN)/ca
	@rm -rf $(GEN)/ctf
	$(OCM) transfer ca $(GEN)/ca $(GEN)/ctf
	touch $(GEN)/ctf

.PHONY: push
push: $(GEN)/ctf $(GEN)/push.$(NAME)

$(GEN)/push.$(NAME): $(GEN)/ctf
	$(OCM) transfer ctf -f $(GEN)/ctf $(OCMREPO)
	@touch $(GEN)/push.$(NAME)

.PHONY: transport
transport:
ifneq ($(TARGETREPO),)
	$(OCM) transfer component -Vc  $(OCMREPO)//$(COMPONENT):$(VERSION) $(TARGETREPO)
else
	@echo "Cannot transport no TARGETREPO defined as destination" && exit 1
endif

$(GEN)/.exists:
	@mkdir -p $(GEN)
	@touch $@

.PHONY: info
info:
	@echo "VERSION:  $(VERSION)"
	@echo "COMMIT:   $(COMMIT)"
	@echo "TREESTATE:   $(GIT_TREE_STATE)"

.PHONY: describe
describe: $(GEN)/ctf
	ocm get resources --lookup $(OCMREPO) -r -o treewide $(GEN)/ctf

.PHONY: descriptor
descriptor: $(GEN)/ctf
	ocm get component -S v3alpha1 -o yaml $(GEN)/ctf

.PHONY: clean
clean:
	rm -rf $(GEN)

The Makefile supports the following targets:

build (default) simple Go build
version show current VERSION of Github repository
image build a local Docker image
multi build multi-arch images with Docker
ca execute build and create a component archive
ctf create a common transport format archive
push push the common transport archive to an OCI registry
info show variables used in Makefile (version, commit, etc.)
describe display the component version in a tree-form
descriptor show the component descriptor of the component version
transport transport the component from the upload repository into another OCM repository
clean delete all generated files (but does not delete Docker images)

The variables assigned with ?= at the beginning can be set from outside and override the default declared in the Makefile. Use either an environment variable or an argument when calling make.

Example:

$ PROVIDER=foo make ca

Templating the Resources

The Makefile uses a dynamic list of generated platforms for the images. You can just set the PLATFORMS variable:

MULTI     ?= true
PLATFORMS ?= linux/amd64 linux/arm64

If MULTI is set to true, the variable PLATFORMS will be evaluated to decide which image variants will be built. This has to be reflected in the resources.yaml. It has to use the input type dockermulti and list all the variants which should be packaged into a multi-arch image. This list depends on the content of the Make variable.

The OCM CLI supports this by enabling templating mechanisms for the content by selecting a templater using the option --templater .... The example uses the Spiff templater.

$(GEN)/ca: $(GEN)/.exists $(GEN)/image.$(NAME)$(FLAGSUF) resources.yaml $(CHART_SRCS)
	$(OCM) create ca -f $(COMPONENT) "$(VERSION)" --provider $(PROVIDER) --file $(GEN)/ca
	$(OCM) add resources --templater spiff $(GEN)/ca COMMIT="$(COMMIT)" VERSION="$(VERSION)" \
		IMAGE="$(IMAGE):$(VERSION)" PLATFORMS="$(PLATFORMS)" MULTI=$(MULTI) resources.yaml
	@touch $(GEN)/ca

The variables given to the add resources command are passed to the templater. The template looks like:

name: image
type: ociImage
version: (( values.VERSION ))
input:
  type: (( bool(values.MULTI) ? "dockermulti" :"docker" ))
  repository:  (( index(values.IMAGE, ":") >= 0 ? substr(values.IMAGE,0,index(values.IMAGE,":")) :values.IMAGE ))
  variants: (( bool(values.MULTI) ? map[split(" ", values.PLATFORMS)|v|-> values.IMAGE "-" replace(v,"/","-")] :~~ ))
  path: (( bool(values.MULTI) ? ~~ :values.IMAGE ))

By using a variable values.MULTI, the command distinguishes between a single Docker image and a multi-arch image. With map[], the platform list from the Makefile is mapped to a list of tags created by the docker buildx command used in the Makefile. The value ~~ is used to undefine the yaml fields not required for the selected case (the template can be used for multi- and single-arch builds).

$(GEN)/image.$(NAME).multi: $(GEN)/.exists Dockerfile $(GO_SRCS)
	echo "Building Multi $(PLATFORMS)"
	for i in $(PLATFORMS); do \
	tag=$$(echo $$i | sed -e s:/:-:g); \
	echo "Building platform $$i with tag: $$tag"; \
	docker buildx build --load -t $(IMAGE):$(VERSION)-$$tag --platform $$i .; \
	done
	@touch $(GEN)/image.$(NAME).multi

Pipeline Integration

Pipeline infrastructures are heterogenous, so there is no universal answer how to integrate a build pipeline with OCM. Usually, the simplest way is using the OCM command line interface. Following you will find an example using GitHub actions.

There are two repositories dealing with GitHub actions: The first one provides various actions that can be called from a workflow. The second one provides the required installations of the OCM parts into the container.

An typical workflow for a build step will create a component version and a transport archive:

jobs:
  create-ocm:
    runs-on: ubuntu-latest
    steps:
      ...
      - name: setup OCM
        uses: open-component-model/ocm-setup-action@main
      ...
      - name: create OCM component version
        uses: open-component-model/ocm-action@main
        with:
          action: create_component
          component: acme.org/demo/simpleserver
          provider: ${{ env.PROVIDER }}
          version: github.com/jensh007
      ...

This creates a component version for the current build. Additionally, a transport archive may be created or the component version along with the built container images may be uploaded to an OCI registry, etc.

More documentation is available here. A full example can be found in the sample Github repository.

Static and Dynamic Variable Substitution

Looking at the settings file shows that some variables like the version or the commit change with every build or release. In many cases, these variables will be auto-generated during the build.

Other variables like the version of 3rd-party components will just change from time to time and are often set manually by an engineer or release manager. It is useful to separate between static and dynamic variables. Static files can be checked-in into the source control system and are maintained manually. Dynamic variables can be generated during build.

Example: manually maintained:

NAME: microblog
COMPONENT_NAME_PREFIX: github.com/acme.org/microblog
PROVIDER: ocm.software
ELASTIC_VERSION: 8.5.1
MARIADB_VERSION: 10.6.11
MARIADB_CHART_VERSION: 11.4.2
NGINX_VERSION: 1.5.1
NGINX_CHART_VERSION: 4.4.2

auto-generated from a build script:

VERSION: 0.23.1
COMMIT: 5f03021059c7dbe760ac820a014a8a84166ef8b4

ocm add componentversions --create --file ../gen/ctf --settings ../gen/dynamic_settings.yaml --settings static_settings.yaml component-constructor.yaml

Debugging: Explain the Blobs Directory

For analyzing and debugging the content of a transport archive, there are some supportive commands to analyze what is contained in the archive and what is stored in which blob:

tree ../gen/ctf
../gen/ctf
├── artifact-index.json
└── blobs
    ├── ...
    ├── sha256.59ff88331c53a2a94cdd98df58bc6952f056e4b2efc8120095fbc0a870eb0b67
    ├── ...

ocm get resources -r -o wide ../gen/ctf
...
---
REFERENCEPATH: github.com/acme.org/microblog/nginx-controller:1.5.1
NAME         : nginx-controller-chart
VERSION      : 1.5.1
IDENTITY     :
TYPE         : helmChart
RELATION     : local
ACCESSTYPE   : localBlob
ACCESSSPEC   : {"localReference":"sha256:59ff88331c53a2a94cdd98df58bc6952f056e4b2efc8120095fbc0a870eb0b67","mediaType":"application/vnd.oci.image.manifest.v1+tar+gzip","referenceName":"github.com/acme.org/microblog/nginx-controller/ingress-nginx:4.4.2"}
...

Self-Contained Transport Archives

The transport archive created from a component-constructor file, using the command ocm add componentversions --create ..., does not automatically resolve image references to external OCI registries and stores them in the archive. If you want to create a self-contained transport archive with all images stored as local artifacts, you need to use the --copy-resources option of the ocm transfer ctf command. This will copy all external images to the blobs directory of the archive.

ocm transfer ctf --copy-resources <ctf-dir> <new-ctf-dir-or-oci-repo-url>

Note that this archive can become huge if there an many external images involved!

CICD Integration

Configure rarely changing variables in a static file and generate dynamic variables during the build from the environment. See the Static and Dynamic Variable Substitution section above.

Air-gapped GitOps with OCM & Flux

Build & Deploy Infrastructure via Helm Charts with OCM

Docs

Open Component Model

Title here

Best Practices

Use Public Schema for Validation and Auto-Completion of Component Descriptors

Separate Build and Publish Processes

Separation Between Build and Publish

Building Multi-Architecture Images

Using Makefiles

Prerequisites

Templating the Resources

Pipeline Integration

Static and Dynamic Variable Substitution

Debugging: Explain the Blobs Directory

Self-Contained Transport Archives

CICD Integration

Best Practices

Use Public Schema for Validation and Auto-Completion of Component Descriptors#

Separate Build and Publish Processes#

Separation Between Build and Publish#

Building Multi-Architecture Images#

Using Makefiles#

Prerequisites#

Templating the Resources#

Pipeline Integration#

Static and Dynamic Variable Substitution#

Debugging: Explain the Blobs Directory#

Self-Contained Transport Archives#

CICD Integration#

Use Public Schema for Validation and Auto-Completion of Component Descriptors

Separate Build and Publish Processes

Separation Between Build and Publish

Building Multi-Architecture Images

Using Makefiles

Prerequisites

Templating the Resources

Pipeline Integration

Static and Dynamic Variable Substitution

Debugging: Explain the Blobs Directory

Self-Contained Transport Archives

CICD Integration